January 1, 1998

Designing Fast Ethernet switches is easy
with chip sets and reference kits

Stephen Kempainen, Technical Editor

Designing Ethernet switches has never been easier. Off-the-shelf switch chips and reference designs dramatically reduce switch development costs.

Fast Ethernet (FE) switches are the popular way to satisfy bandwidth-hungry applications on millions of computers. Low-cost Ethernet switches with 10- and 100-Mbps ports give each end station dedicated, rather than shared, bandwidth. However, packet switching alone doesn't relieve performance-stunting congestion. Congestion relief comes from switch features such as virtual LANs (VLANs), packet filtering, and priority levels. Building these features into switches has become easier with a variety of available switch-on-chip and chip sets. Chip vendors are even providing designs, including software, that are ready for you to take to production.

Applications with increasing bandwidth requirements create demand for switching performance in networked workgroups and internetwork connections. The Layer 2, or "frame," switches--so named because they forward unaltered frames--bridge switch ports for workgroups. These switches are drop-in replacements for hubs and repeaters. They operate at the data-link layer by analyzing the media-access-control (MAC) header (Reference 1). Layer 2 switches maintain the appearance--a transparent bridge--but increase data throughput of one network. These switches provide performance gains because each port has dedicated bandwidth, and multiple ports can concurrently receive and transmit packets. Switches eliminate packet collisions. Layer 2 switches are simple to install and set up be-cause they need little configuration.

Adding functionality to Layer 2 switches gives you "multilayer" switches. Additional functions in-crease performance beyond switching-speed gains by reducing congestion. The frame switch traditionally learns addresses, forwards packets without altering them, and resolves network loop paths by using spanning-tree routines to shut down paths. The same switches become multilayer by using Simple Network Management Protocol (SNMP) and Remote MONitoring (RMON) to perform management operations and enhance diagnostic capabilities. By adding these management tools, the switch be-comes visible to network operation controllers. Additional features, such as active flow control and buffering, enhance performance by alleviating congestion. Multilayer switches can also incorporate VLANs--configuring LANs by policies rather than physical location--in which a frame's tag header includes information on destination policy and priority provisions. The policy information further reduces segment traffic and congestion and increases security.

Layer 3 switches, multilayer switches with internetwork routing capability, perform routing functions to connect LANs. Much confusion exists about Layer 3 switches because approximately half those switches that manufacturers call "Layer 3'' are just multilayer bridges with a different moniker, and the other half are truly routers operating at the network layer. The standard routing protocols, such as Open Shortest Path First (OSPF), use the Internet Protocol (IP) header to determine destinations. Routing protocols can increase network performance by allowing for loop paths to act as redundant paths. Therefore, the protocols need not shut those paths down, as a spanning-tree routine would.

A Layer 3 switch does everything a Layer 2 switch does plus routing packets between networks, conditionally modifying and forwarding frames, and managing operations. Because these switches concentrate traffic from multiple connections, keeping performance high depends on making packet-forwarding decisions at wire speed--10-Mbps for Ethernet and 100 Mbps for FE. Router forwarding requires parsing the frame from the packet, searching large routing-address tables, and reconstructing a framed packet to transmit. The new frame includes the next destination's MAC address from the routing table, the decremented time-to-live field for the datagram, and the recalculated IP header checksum. Conditional forwarding is synonymous with packet filtering and is useful in controlling congestion and providing security. Installing a Layer 3 switch changes a network's architecture and requires configuration and maintenance to keep it functioning properly.

Most chip vendors offer both store-and-forward and cut-through packet-forwarding mechanisms (Table 1). The store-and-forward, or packet-by-packet, switch takes the whole frame and checks it for errors before forwarding it. This switching mechanism supports Layer 3 routing and forwards only error-free frames. It also supports complex packet-filtering schemes and mixed-speed ports. The store-and-forward mode increases usable bandwidth by filtering all nonregulation packets from a network. However, store-and-forward mode can add latency.

The cut-through switch starts forwarding the frame when the switch determines the destination. Because forwarding can begin before the switch receives the whole frame and error checking occurs only after receiving the whole frame, this mode can forward frames with errors. This switch has consistent and short latencies and applies best in single-speed workgroups and backbones. "Runt-free" cut-through switching prevents forwarding packets with less than 64 bytes, or "runts," by stalling the packet transmission until 64 bytes--the minimum regulation packet--arrive at the ingress port.

Layer 2 and 3 switches make sophisticated packet-handling decisions based on information from headers and tags. The headers and tags encapsulate the packet to make it into a frame. However, these decisions hinder the switch's throughput performance unless they are fast enough to keep up with the incoming data's wire speed. For switches with multiple FE ports, many designers think hardwired decision-making is the only way to keep up with the wire speed. But hardwiring the logic assumes that the networking protocol is stable and standardized, which is only now becoming the case for packet priority and VLANs (see box "Standards for VLANs and priority tagging"). For this reason, some chip vendors, such as Vertex and MMC, assume that optimized protocols will emerge, enabling the vendors to use programmable network processors in their chip sets, easing future upgrades.

Decisions increase performance

A switch's packet-handling decisions also increase performance by alleviating congestion. Congestion impairs performance by blocking packets from reaching destinations. If there is too little buffer space to store blocked packets, the switch drops them, and the source must try to send them again. To increase end-to-end performance in today's networks with their primarily internetworking traffic, the protocols must be the same across all LAN boundaries. Chip manufacturers are adding logic for the new protocols to their switching chips. These protocols address problems across networks.

Switch performance can suffer from packet broadcasts in which every node in the network is a destination. Layer 3 devices define network boundaries and, therefore, limit the broadcast domains. Layer 2 switches propagate "broadcast storms"--congestion resulting from simultaneous broadcast packets arriving at multiple ingress ports. These storms occur unless the Layer 2 switch provides a broadcast forward-and-filter mechanism.

Broadcasts are inherently less problematic for some switch fabrics. For example, switches that use a multidrop, shared bus as the switch fabric broadcast more easily because every node simultaneously receives the broadcast packet from the shared bus (Figure 1). However, shared-memory switch fabrics become congested when every port tries to access the same memory location. On the other hand, crossbar-switch fabrics become congested because they individually resend the same message, or "unicast," to every receiving port. In all cases, broadcast filtering through port-policy descriptors--criteria for each port to accept broadcasts--and VLAN subnets control the aforementioned broadcast storms. Almost all chip sets provide some broadcast-filter mechanism to reduce broadcast storms.

IP "multicasting," in which only selected egress ports are destinations for the packet, can also hurt switch performance by causing congestion. The standard for IP multicasting is the Internet Group Management Protocol (IGMP). Layer 3 switches and routers include IGMP because it uses the IP address that these switches and routers analyze. In the IGMP, the receiving end station requests and maintains membership in a multicast group of interest, such as a videoconference. This approach reduces network traffic through the switch because the multicast source sends only to those stations that are interested rather than broadcasting to all that might be interested. IGMP is essential because of the bandwidth savings it provides.

Congestion in networks also comes from video and voice streams because they require on-time delivery of a constant data flow. In addition, these streams require low latency, ordered delivery, and constant bandwidth. These requirements mean that the network must guarantee levels of service. Because Ethernet provides no means of reserving resources to guarantee levels of service, it needs new protocols to provide some reservation ability. The ReSerVation Protocol (RSVP) is an Internet Engineering Task Force draft standard that could provide some bandwidth guarantees in small networks. But, because RSVP requires each router to maintain descriptor tables--latency and bandwidth requirements--for each packet stream, this protocol may not work across large internetworks. Another protocol with some promise is IEEE 802.3p, which provides eight priority levels. Packets with higher priority move to the head of queues to avoid head-of-line blocking that increases latency and causes jittery bandwidth. 802.1p and 802.1Q work together for priority and bridging VLANs because they use the same header-tag format for priority and VLAN information. Switch-chip sets need to incorporate the queue-management functions necessary to enforce 802.1p priority transmissions.

Management of many VLAN segments is essential to network performance and health but complex in a switched network. Each port in a switch needs to maintain a record of the traffic load, traffic type, delays, and other performance parameters. The port must then communicate the statistics to a management-control unit and processor for analysis. The SNMP and RMON standards define the management interfaces and statistics for including management in the switches. Almost all the chip sets in Table 1 include hardware support for management-statistics gathering.

Chip sets and reference designs

Chip vendors' reference designs--ranging from simple evaluation boards to complete, ready-to-manufacture product designs--significantly reduce your design-cycle time for high-speed switches with congestion control. The evaluation boards provide a platform on which to test signals and develop software. Vendors are also offering more complete designs that provide schematics, Gerber files, and lists of materials to take hardware to manufacturing. The most complete kits available are the reference designs that hand you the hardware, software, and documentation for entering the switch OEM market on a platter.

The evaluation boards are inexpensive ways to run the chips through some exercises. Galileo offers an evaluation-board ensemble with each of its switch chips. For example, the Galileo-7 support board for the GT-48002A is available for $2495. The GT-48002A chip provides switching between two FE ports and a PCI-bus-expansion port. The device also includes RMON Management Information Base (MIB) counters, a CPU routing-intervention mode, and spanning-tree support. The Galileo-7 plugs into the Galileo-4BP PCI passive backplane ($495), which provides five PCI-master slots and a reprogrammable arbiter. By adding a Galileo MIPS or i960 processor-development board, you can assemble a managed, Layer 2 switch prototype. You use the prototype to develop and test code and perform benchmarking and system validation. Third-party software vendors also supply modules for the prototyping.

The next level of reference design is the kit and schematics you need to produce a product. These designs are complete enough to allow a nontraditional switch manufacturer to easily enter the market with a competitive product. For example, Vertex Networks offers a few reference designs with the XpressFlow 2001 chip set. One design is a Layer 2 switch with eight 10/100-Mbps ports. It includes the basic software to get it functioning for $3200 (Figure 2). It also provides software modules for SNMP- and RMON-management functions. The chip set incorporates a high-density-instruction-set processor core (HISC) in the XpressFlow engine. The embedded HISC processes incoming frames for forwarding and filtering. The combination of the HISC and a CPU gives this chip set the ability to offer compliant 802.1p priority and 802.1Q VLAN tagging. The HISC allows for this switch design to adapt to new or changing protocols with changes in firmware.

Texas Instruments offers OEM-ready, Layer 2-switch reference designs for its ThunderSwitch switch-on-chip products. The reference kits for the TNETX3100 Desktop ThunderSwitch 8/2 and the TNETX3150A are available for $2500 each. Both include board and enclosure schematics, Gerber files, software for test diagnostics, and chip documentation for a desktop workgroup switch. The TNETX3100 switch design has eight 10-Mbps ports and two 10/100-Mbps ports. The TNETX3100 supports on-chip, per-port storage of Etherstat for RMON and SNMP. The auxiliary device, TNETX15AE, adds 8000-address expansion to the reference design for the Desktop ThunderSwitch. The TNETX15VE is not on the reference design, but when you add it to a design, it gives VLAN capability to both the TNETX3100 and the TNETX3150A.

Another reference design for a high-performance, Layer 2 switch is available from chip vendor I-Cube. The Raptor reference design is a managed switch with 16 full-duplex 100-Mbps ports. It can concurrently operate every port in full-duplex transmit and receive at wire speed. The Raptor uses the I-Cube crossbar-switch technology to deliver nonblocking datapaths because the Raptor has no shared buses (Figure 3). It also provides store-and-forward, cut-through, and runt-free cut-through forwarding modes on a per-port basis. An address-caching architecture supports 65,536 MAC addresses. The Raptor also supports SNMP management with RMON and MIB statistics. The design also provides 802.1d-compliant spanning-tree support and VLAN capability to help with network management. For $5000, you get a complete system, including an assembled pc board, software source code, schematics, Gerber files, and application notes.

At the top level are the total-package reference designs. For example, the PS1000 reference design from MMC Networks leaves nothing left undone. For $250,000, you get everything you need to build a product and enter the switch business. The PS1000 FE switch-engine chip set is flexible enough to let you build in as little or as much functionality as you want. The modular design centers on MMC's ViX architecture for a shared SRAM buffer switch and the packet-switch processor. You use 10- and 100-Mbps Ethernet and ATM-interface units to add ports. The design also includes devices for address-resolution logic and bridges to a PCI bus and a CPU. The reference design includes all the Gerber files, the bill of materials, and a full suite of bridging and management software.

The Layer 3 AnyFlow 5000 network-processor chip set will be available in March. This chip set includes a programmable network processor that provides the Layer 3 routing functions. Other building-block chips are two modular switch controllers, a per-flow queuing controller, Ethernet and ATM bit-stream processors, a per-flow scheduler, a memory-access buffer, and a control-message interface. The Any-Flow 5000 provides wire-speed switching and routing with class and quality-of-service provisions.

For more seasoned switch designers, building-block components are available to complete Layer 2 and 3 switch designs. For example, Digital Semiconductor offers two chips for flexible switch design. The eight-port, 10/100-Mbps 21440 Ethernet controller supplies a building block for switches by combining flexible port interfaces, packet-header preprocessing, and RMON probes at each port. The 21440 features target-routing functions. For example, it assembles Ethernet frames with a MAC header, an IP header, and data from different sources. The 21340-AB buffered port switch, a building block for Layer 2 switches, lets any of the four ports connect to any of the four segment buses. Each segment bus connects separate collision domains.

The new chips and reference designs lower your design costs. The reference designs shorten the cycle for experienced designers and open the field to beginners. You can use a chip vendor's reference design as your switch or just to get a quick start on a design. You differentiate your switch by enhancing the hardware design or by adding your own software modules. Either way, switch design is getting easier with today's highly integrated switching chips.

References

1. Kempainen, Stephen, "Gigabit Ethernet and ATM go neck and neck in the communications race," EDN, Jan 2, 1997, pg 32.

2. Wright, Maury, "Network-switch ICs simplify design and slash per-port costs," EDN, Nov 23, 1995, pg 53.

| EDN Access | Feedback | Table of Contents |