Skip to content

Developer Notes for LCM Bridge Application

Introduction

The idea behind this works was that it would be great to be able to have an application that monitors a configuration file (JSON likely) and when the file is updated, the application will spin up and down LCM subscribers that will convert the message to JSON and then 'bridge' those messages to other systems. What 'bridging' means is different based on the target system. Some possibilities to think about are:

  1. ZeroMQ
  2. AMQP (RabbitMQ)
  3. MQTT
  4. Kafka
  5. Cassandra
  6. TimescaleDB
  7. Elasticsearch
  8. MongoDB
  9. Websocket

I know Kevin Barnard has a Python script that Carlos is using for the WebSocket purpose, but the idea was to extend this to other application targets and it only makes sense to do it for WebSockets too (maybe but this will NOT be the primary target of this work).

Architecture

--- title: High Level Flow --- flowchart LR subgraph LCM-Messages lcm1(Channel/Type 1) lcm2(Channel/Type 2) lcm3(Channel/Type 3) lcm4(Channel/Type 4) lcm5(Channel/Type 5) lcm6(Channel/Type 6) end subgraph LCM-Bridge-Controller lcm(LCM) zmq-bridge(ZeroMQBridge) --> zmq-socket(Socket/Address) message-handler-1(MessageHandler 1) message-handler-2(MessageHandler 2) message-handler-3(MessageHandler 3) message-handler-4(MessageHandler 4) message-handler-5(MessageHandler 5) message-handler-6(MessageHandler 6) rabbitmq-bridge(RabbitMQ Bridge) mqtt-bridge(MQTT Bridge) kafka-bridge(Kafka Bridge) message-type-1-handler(MessageType 1 Handler) message-type-2-handler(MessageType 2 Handler) message-type-3-handler(MessageType 3 Handler) message-type-4-handler(MessageType 4 Handler) message-type-5-handler(MessageType 5 Handler) message-type-6-handler(MessageType 6 Handler) end message-handler-1 --> message-type-1-handler message-handler-2 --> message-type-2-handler message-handler-3 --> message-type-3-handler message-handler-4 --> message-type-4-handler message-handler-5 --> message-type-5-handler message-handler-6 --> message-type-6-handler lcm1 --> message-type-1-handler --> zmq-bridge message-type-1-handler --> kafka-bridge lcm2 --> message-type-2-handler --> zmq-bridge lcm3 --> message-type-3-handler --> zmq-bridge message-type-3-handler --> rabbitmq-bridge lcm4 --> message-type-4-handler --> zmq-bridge message-type-4-handler --> rabbitmq-bridge lcm5 --> message-type-5-handler --> zmq-bridge lcm6 --> message-type-6-handler --> zmq-bridge message-type-6-handler --> mqtt-bridge
--- title: LCM Bridge Controller Classes --- classDiagram LCMBridgeController --o LCM LCMBridgeController "1" --o "0..1" ZeroMQBridge LCMBridgeController "1" --o "0..*" MessageHandler Bridge <|-- ZeroMQBridge ZeroMQBridge --o socket_t MessageHandler "1" --o "1" MessageTypeHandler MessageHandler "1" --o "0..*" Bridge MessageHandler --> LCMToJSON MessageTypeHandler -- LCM class LCMBridgeController{ -LCM lcm_core -ZeroMQBridge *zero_mq_bridge -vector~MessageHandler~ handlers +main() } class LCM{ +subscribe() } class Bridge{ +forward_json(std::string channel, std::string message_type, std::string json_message) } class ZeroMQBridge{ ZeroMQBridge(std::string bind_address) +forward_json(std::string channel, std::string message_type, std::string json_message) } class socket_t { +send(zmq::buff(json_message)) } class MessageHandler{ -std::string lcm_type -vector~Bridge*~ bridges -vector~int~ sub_sample_rates MessageHandler(const std::string &channel, const std::string &type_name, lcm::LCM &lcm, vector~Bridge*~ &bridges, vector~int~ &sub_sample_rates) } class MessageTypeHandler { -std::string lcm_type -vector~Bridge*~ bridges -vector~int~ sub_sample_rates +onMessage(const lcm::ReceiveBuffer*rbuf, const std::string& channel, const ~MessageType~* msg) } class LCMToJSON{ +convertLCMToJSON(~MessageType~lcm_message) }

ASSUMPTION: We are making the assumption that there is a 1-1 association between LCM channel and type. In other words, we expect there to only be one message type per channel (similar to LCM Spy).

Classes

  1. LCMBridgeController: This is actually not a class, but is the main entry point for an LCM bridge process. It basically is responsible for constructing an LCM object, then reading in and monitoring (on the main thread) a configuration JSON file. Based on the JSON, it will configuring a logging utility (spdlog) and then construct any Bridges that are defined in the JSON. For example, the JSON might spell out the contruction parameters for a ZeroMQ Bridge, a RabbitMQ Bridge, and a Kafka bridge. Once the bridges are constructed, it then looks at the JSON configuration for channels. Basically, each channel in the JSON can be associated with one or more bridges and each channel-bridge combination can have a custom sub-sample rate. In other words, one LCM channel can be set up to push messages across a ZeroMQ bridge at rull rate, while maybe sending only every 10th message across a RabbitMQ bridge. It will create a vector of Bridges that are to be linked to an LCM channel and also a vector of the sub-sample rates for those bridges. It then hands the channel name, lcm-type name, LCM object and bridge and sub-sample rate vectors to a MessageHandler constructor.
  2. MessageHandler: The MessageHandler code is actually written by a script called 'generate-message-handler.sh'. That script is run during the build and basically looks at all the LCM types that were built by the CMake/Make process and then generates one class for each message type that implements the onMessage method for that specific type. This class can then be used as the handler for the LCM messages of that type. The MessageHandler class itself simply takes in the channel name, lcm-type name, the LCM object, the vector of Bridges and the vector of subsample rates and then constructs the proper message handling class (based on tahe lcm-type name) and then calls LCM.subscribe which links up the LCM channel to the class with the onMessage method. Each onMessage method does the implicit conversion to the correct LCM type and then the method uses the static LCMToJSON methods to convert the LCM type to a JSON message. The onMessage method then basically loops over each bridge and if the message counter indicates that it matches a sub-sample rate, it calls forward_json on that bridge.

Target Notes and Bridge Architecture

ZeroMQ

The best way to map these messaging systems is to think of a 1-to-1 link between LCM channel/type to an ZeroMQ pub topic. In other words, there should be one object that contains all the information about the ZeroMQ publisher (like port number) and then individual objects/threads for each of the bridges between LCM and the ZeroMQ topics. So this might look like:

flowchart LR subgraph LCM Traffic lcm-channel-1(Channel/Type 1
process) lcm-channel-2(Channel/Type 2
process) lcm-channel-3(Channel/Type 3
process) lcm-channel-4(Channel/Type 4
process) lcm-channel-5(Channel/Type 5
process) lcm-channel-6(Channel/Type 6
process) end subgraph LCM Bridge config-file --> lcm-bridge-controller(LCM Bridge Controller
main thread) --> zeromq-manager(ZeroMQ Bridge Manager
main thread) --> zeromq-bridge-1(ZeroMQ Bridge 1
std::thread) --> zeromq-topic-1 zeromq-manager --> zeromq-bridge-2(ZeroMQ Bridge 2
std::thread) --> zeromq-topic-2 zeromq-manager --> zeromq-bridge-3(ZeroMQ Bridge 3
std::thread) --> zeromq-topic-3 zeromq-manager --> zeromq-bridge-4(ZeroMQ Bridge 4
std::thread) --> zeromq-topic-4 zeromq-manager --> zeromq-bridge-5(ZeroMQ Bridge 5
std::thread) --> zeromq-topic-5 zeromq-manager --> zeromq-bridge-6(ZeroMQ Bridge 6
std::thread) --> zeromq-topic-6 end lcm-channel-1 --> zeromq-bridge-1 lcm-channel-2 --> zeromq-bridge-2 lcm-channel-3 --> zeromq-bridge-3 lcm-channel-4 --> zeromq-bridge-4 lcm-channel-5 --> zeromq-bridge-5 lcm-channel-6 --> zeromq-bridge-6

So, what do the interfaces between the different components need to look like? What commands and responsibilities does each piece have?

  1. LCM Bridge Controller
    1. Responsible for monitoring the config file for changes
    2. When a ZeroMQ bridge is defined in the config file:
      1. The controller should check to see if a ZeroMQ bridge manager is already in place and if not, create one
      2. Hand the bridge manager the channel configurations.
    3. When the Bridge Controller is shutdown, it should tell the ZeroMQ Bridge Manager to shutdown too.
  2. ZeroMQ Bridge Manager
    1. There isn't much to do for the manager when first constructed.
    2. Once the bridge is constructed.

AMQP

Under construction

MQTT

Under construction

Kafka

Under construction

Cassandra

Under construction

TimescaleDB

Under construction

Elasticsearch

Under construction

MongoDB

Under construction

Websocket

Under construction

Processes

  1. lcm-bridge-controller
    1. Monitor changes to a configuration file. Since it's really difficult to have a cross-platform way to be notified when a file changes, we can use polling since it can be very low update rates. I did look into inotify and fswatch, but the c++ interface in either case was difficult to make cross platform and the cost just wasn't work the small performance enhancement.