<refentry id="multicast-setup" revision="17 Jan 2008">
<refmeta>
    <refentrytitle>UDP Multicast Setup</refentrytitle>
</refmeta>

<refnamediv>
    <refname>UDP Multicast Setup</refname>
    <refpurpose>Getting maximum performance on your LAN or local host</refpurpose>
</refnamediv>

<refsect1>
    <title>Using LCM on a single host</title>

    <para>A quirk of multicast, at least on Linux, is that the loopback
    interface 127.0.0.1 cannot be used for multicast traffic.  This means
    that to use LCM, even for inter-application communication on a single
    host, <emphasis>you must have a network interface</emphasis>.  If
    this network interface is already connected to the Internet, LCM will
    generally "just work" because the default route
    will allow LCM to find the correct network interface for multicast
    traffic.</para>

    <para>If the network interface is not connected to anything, you
    must explicitly enable it for multicast traffic.  First, bring the
    interface up using a command like <command>ifconfig eth0 up</command>.
    It need not be assigned an IP address.  Then, you must add a route
    for multicast traffic.  For LCM's default multicast group, the
    command would be:</para>
    <blockquote>
    <command>route add -net 239.255.0.0/16 dev eth0</command>
    </blockquote>

    <para>Remember, you must always do this to use LCM locally if your machine
    is <emphasis>not connected</emphasis> to any external network.</para>

</refsect1>
<refsect1>
    <title>Choosing a TTL</title>

    <para>LCM by default uses a time-to-live (TTL) value of 0.  This will
    prevent any LCM packets from being transmitted on the wire.  Only local
    applications will see them.  Choose a value of 1 for the entire subnet to
    see the traffic.  Even larger values will enable the packets to pass
    through routers.  However, these routers must be set up with multicast
    routing tables in order to successfully relay multicast traffic.
    </para>

    <para>There are two ways to set the TTL.  First, LCM constructors in all
    languages provide a way to set the TTL (e.g. see <function><link
    linkend="lcm-create">lcm_create</link></function>), so you can set this in your
    code.  Second, if your program does not explicitly set its provider URL,
    then you can use the <literal>LCM_DEFAULT_URL</literal> environment
    variable.  For exaple, in the bash shell:
    <blockquote>
    <command>export LCM_DEFAULT_URL=udpm://239.255.76.67:7667?ttl=1</command>
    </blockquote>
    </para>

</refsect1>
<refsect1>
    <title>Using IGMP snooping</title>

    <para>When the multicast protocol was designed, it was intended that on
    local subnets it would act like broadcast traffic.  However, for
    high-bandwidth applications, this may not be desirable since it will
    degrade the bandwidth available for hosts not participating in the
    multicast group.</para>

    <para>Switch manufacturers have come up with a solution to this problem
    called "IGMP snooping".  IGMP snooping consists of two parts.  First, the
    switch must monitor network traffic across its ports and decode any IGMP
    packets that deal with multicast group subscriptions.  The switch can then
    send multicast traffic for a particular group only to those ports where a
    subscription request has been observed.  The second part is IGMP querying.
    Hosts will only send out subscription requests upon first subscribing or
    when queried by a router.  In order for the switch to maintain its state,
    these queries must be sent every few minutes.  If there is no router on
    your subnet sending these queries, you must configure the switch to send
    out "fake queries".  Without this, the hosts will eventually stop sending
    IGMP subscription requests, and the switch will "fail open" causing all
    multicast traffic to be broadcast again.</para>

    <title>Mixed-speed networks</title>

    <para>If you have a subnet with devices of different speeds, such as
    10Mbps links mixed with 100Mbps links, be extra careful using multicast.
    LCM traffic will be sent to all devices (unless you enable IGMP snooping,
    above).  If two 100Mbps devices are exchanging more than 10Mbps of
    traffic, this traffic will be relayed to the slow device by the switch,
    even if that device does not run LCM.  This can overwhelm the bandwidth of
    that link, and depending on the switch, may even cause back pressure on
    the 100Mbps devices.  That means, not only will the 10Mbps link be
    swamped, but the 100Mbps devices will be unable to transmit faster than
    10Mbps because their send queues will start filling up and cause
    blocking writes in the LCM API.</para>

    <para>The easiest solution to this problem is to enable IGMP snooping, as
    described above.  If that is not available, sometimes it is sufficient to
    upgrade from consumer-grade switches to managed switches, which often have
    better buffering strategies.  As a last resort, separating the slow
    devices from high-speed devices using two levels of switches can also
    improve the situation.</para>

</refsect1>
<refsect1 id="kernel-receive-buffer-size">
    <title>Kernel UDP receive buffer sizing</title>

    <note><para>Much of this information is taken from <ulink
    url="http://www.29west.com/docs/THPM/udp-buffer-sizing.html">http://www.29west.com/docs/THPM/udp-buffer-sizing.html</ulink></para></note>

    <para>When used on a properly shielded local area network, the most common
    source of dropped and lost packets is not electrical disturbances.
    Instead, it will usually be packet buffers that have reached capacity and
    cannot store any more packets.</para>

    <para>Operating system kernels typically allocate both a
    <emphasis>send buffer</emphasis> and <emphasis>receive buffer</emphasis>
    for a network socket.  The send buffer stores packets that the operating
    system has agreed to transmit (i.e. a call to <function>send()</function>
            was successful) but that it hasn't actually yet transmitted over
    the network.  The receive buffer stores packets that the operating system
    has received from the network, but that the application hasn't yet
    retrieved.  In both cases, the buffers have a maximum capacity, and no new
    packets can be sent or received if that capacity is reached.  In the
    send case, a call to <function>send()</function> blocks until there is
    space in the buffer (or fails with <literal>EAGAIN</literal> for
    non-blocking sockets).  In the receive case, incoming packets are simply
    discarded.</para>
    
    <para>When LCM is used in a high-bandwidth application, it may become
    necessary to increase both the default and the maximum kernel receive
    buffer size to avoid dropped packets.
    </para>
   
    <para>Configuring the kernel can be done without rebooting, but requires
    superuser privileges.  <xref linkend="table:kernel-udp-buffer-resizing"/>
    demonstrates how to do this and set a 2MB maximum buffer size.
    </para>

<table id="table:kernel-udp-buffer-resizing">
    <title>Setting Kernel UDP Receive Buffer Parameters</title>

    <tgroup cols="3">
    <thead>
    <row>
    <entry>OS</entry>
    <entry>Maximum</entry>
    <entry>Default</entry>
    </row>
    </thead>
        <tbody>
            <row>
                <entry><para>Linux</para></entry>
                <entry><literal>sysctl -w net.core.rmem_max=2097152</literal></entry>
                <entry><literal>sysctl -w net.core.rmem_default=2097152</literal></entry>
            </row>
            <row>
                <entry><para>Solaris</para></entry>
                <entry><literal>ndd -set /dev/udp udp_max_buf 2097152</literal></entry>
                <entry>unknown.</entry>
            </row>
            <row>
                <entry><para>FreeBSD, Darwin (OS X)</para></entry>
                <entry><literal>sysctl -w kern.ipc.maxsockbuf=2097152</literal>
                </entry>
                <entry>unknown.</entry>
            </row>
        </tbody>
    </tgroup>
</table>

    <para>Most GNU/Linux distributions also allow setting these permanently
    using the file <filename>/etc/sysctl.conf</filename>.  To do this, add
    the following lines into the file somewhere:
    <screen>
    net.core.rmem_max=2097152
    net.core.rmem_default=2097152
    </screen>
    </para>

    <para>An LCM application can also explicitly request a larger receive
    buffer, which is useful on Solaris and FreeBSD/OS X.  This is done by
    setting the <literal>recv_buf_size</literal> option with the
    <literal>udpm</literal> provider in <function><link
    linkend="lcm-create">lcm_create</link></function>.  In our initial
    projects, which involved typical data rates of 10 MB/s, a 2 MB receive
    buffer was generally sufficient.</para>

</refsect1>

</refentry>
