| Message ID | 20200616122745.25056-1-michal.lowas-rzechonek@silvair.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | mesh: Deliver mesh packets over datagram socket | expand |
Hi Michał, On Tue, 2020-06-16 at 14:27 +0200, Michał Lowas-Rzechonek wrote: > This patchset adds another D-Bus API for attaching applications, called > AttachFD. We have talked a little about this, but I am personally opposed to switching the daemon--> App message delivery system to socket based. This has the feeling of something that has been developed as the result of a stress test to see how much data can be pushed through the system as fast as possible, which should not be a common mesh use case. I also worry about the increase in system socket resorces... Currently the daemon consumes 1 socket for App <--> daemon communication (the one to the dbus daemon), and each App currently being supported uses one more (again, to dbus daemon) And all MUXing, marshaling and unmarshaling of messages is handled by DBUS... the very reason for it's existance. While creating a new socket between App and daemon would make the messages flow faster, it comes at the cost of re-inventing MUXing, marshaling, unmarshaling plus all the additional sockets. A larger code footprint, and a *much* larger resource footprint. This just feels to me like the kind of customization that is fine for a vendor to do for a specialized high-flow mesh, but not something that we want to impose on everyone who uses Mesh in the open source community. > When application uses that API, it receives one end of a datagram socket > pair, which it's supposed to recv() from in order to receive mesh > packets, instead of wating for *MessageReceived() calls over D-Bus. > > This significantly reduces system load for high traffic environment > (e.g. an application that subscribes to a large number of publications > in a big network). > > Message delivery is one way only: application is still supposed to call > *Send methods via D-Bus, although the socket pair is bidirectional, so > it would be possible to add sending as well. > > Michał Lowas-Rzechonek (1): > mesh: Implement AttachFD method > > Przemysław Fierek (1): > mesh: Add documentation for AttachFD > > doc/mesh-api.txt | 40 +++++++++++++++ > mesh/mesh.c | 12 ++++- > mesh/model.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++- > mesh/node.c | 83 ++++++++++++++++++++++++++++++- > mesh/node.h | 4 +- > 5 files changed, 259 insertions(+), 6 deletions(-) >
Brian, On 06/16, Gix, Brian wrote: > This has the feeling of something that has been developed as the > result of a stress test to see how much data can be pushed through > the system as fast as possible, which should not be a common mesh use > case. Well, not really. We came up with this mechanism because of performance issues with *real world* networks on embedded platforms. And by "real world" I mean a single network of 200+ nodes. We need a way to monitor the nodes in such networks, and potentially much bigger ones too - mesh addess space is 32k+ nodes, and we already have runnin installations with ~1k nodes, all of which are publishing sensor data. Without this, bluetooth-meshd is *unusable*. > I also worry about the increase in system socket resorces... > Currently the daemon consumes 1 socket for App <--> daemon > communication (the one to the dbus daemon), and each App currently > being supported uses one more (again, to dbus daemon) And all > MUXing, marshaling and unmarshaling of messages is handled by > DBUS... the very reason for it's existance. While creating a > new socket between App and daemon would make the messages flow > faster, it comes at the cost of re-inventing MUXing, marshaling, > unmarshaling plus all the additional sockets. A larger code > footprint, and a *much* larger resource footprint. As you can see in the patch, serialization format is very simple (*much* simpler than D-Bus marshalling). As for MUXing, I don't really understand what you mean? A node can have only one application attached to it, so the daemon knows exactly which socket to use. Same for the application. About file descriptors: this patch targets primarily an embedded platform. While I've seen issues with number of open files on busy network servers, I don't think an IoT device is going to face such problems. As for the "resource footprint", it's a tradeoff between open files vs. CPU. By using D-Bus calls to deliver network packets, you need to: - copy the packet from the mesh daemon to dbus daemon - have the daemon look up the application by D-Bus address - have the daemon deserialize the message header and check permissions - copy the packet from dbus daemon to application This requires an additional copy and an additional context switch, and a potentially costly permission check (I've mentioned AppArmor in the previous mail). So this delivery method in fact consumes *more* resources. > This just feels to me like the kind of customization that is fine > for a vendor to do for a specialized high-flow mesh, but not > something that we want to impose on everyone who uses Mesh in the > open source community. I don't want to "impose" anything... the API is entirely optional and the "regular" Attach() works as it used to, and might be perfectly fine for simpler applications. Note that GATT API does the same thing wrt. AquireNotify() call.