What is LCM?

LCM stands for Lightweight Communications and Marshalling. The communications aspect of LCM is that it allows multiple senders and receivers to use a network for sending and receiving messages to each other using a shared channel. The marshalling aspect of LCM is that it provides you the ability to define your own data types that are automatically encoded into data packets in a platform- and language-independent fashion. Most users will use both of these capabilities at the same time even though they can be separated if desired. LCM also includes a logging facility, allowing LCM traffic to be recorded and replayed.

The design goals for the communications system are:

The design goals for the marshalling system are:

These are some properties that LCM does not have:

Communications System

LCM uses UDP multicast for its underlying communications mechanism. The primary reason for this is so that multiple senders and receivers can connect to the same data channel without requiring a central server of some type that copies packets or maintains separate point-to-point links. Instead, this maintainance is performed directly by the network stacks of the hosts, switches, and routers on the network.

Another advantage of UDP multicast is that multiple applications on the same host behave much the same as multiple applications spread across multiple hosts. This is in contrast to UDP broadcast, in which generally only one application per host can listen on a specific port. UDP multicast also has the advantage of routability across multiple subnets as long as the routers are configured properly (generally possible within an organization but not Internet-wide).

That said, UDP multicast can have quirks in getting it to work properly on a LAN, or even a single host. These issues are discussed in the Section UDP Multicast Setup.

Using LCM requires you to pick an IP address to serve as the multicast group for all messages. This IP address is distinct from any one host's address and is used by every machine to subscribe to the group. You must also pick a UDP port over which messages will be exchanged. The default IP is 239.255.76.67 and port is 7667. This IP is in the special administrative range that is not routable between networks. You must also choose a time-to-live (TTL) which represents the maximum number of routers that a packet can pass through. The default of 0 prevents your packets from leaving a single host. Using 1 will allow your packets to be received anywhere on a single subnet. Note that multicast traffic is treated like broadcast traffic within a subnet, in that every machine on the subnet will receive the traffic, even if it is not subscribed to the multicast group. This can be prevented using a feature called IGMP snooping which is available on higher-end managed switches. See Section UDP Multicast Setup.

After choosing the multicast group, LCM subscribe to the multicast group during initialization of the library by sending out the necessary IGMP requests onto the subnet. After this subscription, that host will receive packets destined to the IP address assigned to the group. The network stack of the host will assure that any applications listening on the correct UDP port receive the traffic. Unlike normal UDP or TCP connections, multiple applications on the same host are allowed to listen to the same multicast port.

The format of LCM packets is very simple. A packet consists of a variable-length ASCII string identifier followed by variable-length buffer of opaque data. This opaque data will generally be a marshalled LCM data type. The string identifier is called the channel name and is the basis for the subscription mechanism in LCM. At the LCM library level, packets are filtered out that do not contain channel names to which the application has subscribed. The packets corresponding to subscribed channels are passed through to the application, either with the opaque buffer intact or already unmarshalled, depending on which API is being used.

Since UDP has a packet-size limit of 65K, LCM will automatically perform high-level fragmentation and reassembly if a packet requires it.

Choosing UDP as the transport mechanism for LCM has some implications for the practical use of the library. Most importantly, since packets could be lost, it is important to design your applications and protocols so this is not a problem. It is advisable to transmit time-sensitive data at a fixed rate so that if a packet is lost, the receiving application will immediately recover with the most recent data as it is received. The frequency of your transmit rate will be determined by how sensitive your application is to latency. Transmitting at a fixed rate also has the advantage that your network will be operating under a steady-state amount of bandwidth and you will not need to worry about the effects of momentary spikes in traffic.

Protocols requiring guaranteed request-response semantics can also be a challenge when using UDP. It is not practical to implement such a scheme for a many-to-many network, however, if you need one-to-one communications as part of your LCM network, guaranteed delivery of request-response protocols can be achieved as follows: Encode the state of the "handshake" using data fields in both the request and response data types. For example, the requester continues to send the request packet until the response packet contains confirmation of its receipt. The responder then continues to resend the response until the requestor confirms receipt of it. If your protocols are already designed around fixed data rates, this handshaking procedure is easy to implement.

Marshalling System

Sending messages from one application to another involves encoding and decoding data into a byte stream. This sort of programming tends to be straight-forward at first glance, but it is tedious, error-prone, and intolerant to mismatches in types. A mismatch in type can occur in two fundamental ways: a message of type "foo_t" was expected but a message of "bar_t" was received, or a message of type "foo_t" was expected but the underlying type specification of "foo_t" did not match on the transmitter and receiver.

Like other message specification systems, it addresses the tedious and error-prone aspects via automatic-generation of code, but provides first-class support for several languages (currently C, Java, and Python). Unlike other message specification systems, LCM detects both types of faults described above, preventing erratic (and often hard-to-isolate) failures. LCM also enforces safe use of variable-length arrays, helping to prevent illegal pointer operations in languages like C.

The formal type specification language for LCM follows standard C and Java syntax as much as possible, while providing several additional features. The syntax itself will also be familiar to users of External Data Representation (XDR), used by RPC. Notable deviations include a simplified notation for variable-length arrays, and a consistent naming scheme for primitive types (based on C99's stdint).

The tutorials illustrate all of the basic syntax and capabilities of the LCM marshalling system.

Logging

LCM includes a logging facility allowing LCM network traffic to be recorded and played back again. LCM's multicast-based network architecture means that adding a logger to the system requires no additional configuration: simply run the logger or log player. The result of a played-back log is indistinguishable from live network, aside from any timestamps that might be embedded in the user-defined LCM types.

The log file format is very simple: each LCM message is concatentated in the order they are received from the network stack. For each record, a synchronization sequence is written, followed by the time at which the message was received, followed by the packet data. The timestamps allow the log player to closely replicate the inter-message timing that occured when the log was created. The synchronization sequence is used to facilitate seeking in the log file: the Java-based log player, which features a flexible GUI, allows the both seeking and variable-speed playback.

The log file format itself has both C and Java implementations and can be accessed by users. This is useful for off-line processing of log files: messages can be filtered or otherwise altered. A similar result could be obtained by writing an LCM application and playing the original log while recording a new log, however accessing the log files directly is both faster (since the log file can be processed at the maximum speed your application is capable of) and more accurate (since timestamps in the logfile can be preserved exactly).

Allowing applications to operate correctly from logged data requires some careful consideration of timestamps. Specifically, timestamps in logged data will not agree with the computer's current time. However, it is relatively easy to write well-behaved LCM applications that are time-agnostic.