The Era of Ultra-Low Latency 25G Ethernet

Back in the early 2000s, store and forward networking was used by both market data providers, exchanges and customers executing electronic trading applications where the lowest latency execution can make the difference in a strategy from a profit to a loss. Moving closer to the exchange to reduce link latency, eliminating any unnecessary network hops, placing all feed handler and trading execution servers on the same switch to minimize transit time, and leveraging high-performance 10Gb NICs with embedded FPGAs all contributed to the ongoing effort to squeeze out every last microsecond to execute trades and gain a performance edge.

Around 2008, the networking industry went through a seismic shift with the introduction of the first generation of high-performance 10G switches with cut-through architectures that challenged the fundamentals of data center networking. Before this fundamental shift, switch latency was measured in microseconds, and 10-30 microseconds was considered fast. Cut-through architectures allowed latency as low as hundreds of nanoseconds and a race was on to shift to cut-through networking. These new networks leveraged dedicated switches with integrated traffic filtering and latency monitoring to ensure networks performed with no delays due to inadvertent queuing or drops from congestion. From 2008 to 2012,  high-density systems were released with incremental latency improvements but the electronic trading architecture remained relatively consistent, leveraging 10Gb leaf and spine systems for low latency trading.

This situation remained relatively unchanged until around 2013 when a new class of network devices emerged that dispensed with even the low latency performance of L2/L3 switches. The rise of “Layer 1” switches leveraging cross-points allowed for single device latency from 4-6ns, but it sacrificed networking functions that allow for many-to-many communications. These shortcomings were addressed through the use of in-network FPGAs where only the essential and minimum amount of packet processing occurs via the FPGA for multiplexing, rather than traditional switching, and one to many traffic at the lowest possible latency. The race to zero continued. One of the key companies was Metamako, based in Sydney, which was acquired by Arista Networks in September 2018.

The crosspoint L1 and FPGA based systems were released as the Arista 7130 Series with ongoing investment in the operating models, transitioning from MOS to EOS, the release of higher density systems and an integrated L2/L3 switch application for the embedded FPGA, allowing the consolidation of multiple devices to the L1+ infrastructure.

With higher trading volumes, the frequency of traffic microbursts has also increased. Microbursts can result in variable latency due to queueing and potentially packet drops.

It is now time to move to 25GbE for ULL networks. The rise of high-frequency trading (HFT) drove multiple innovations in networking technologies and the next phase of innovation has arrived. 

Arista 7130 Series 25G Portfolio

The Arista team has developed a portfolio of 7130 Series systems that deliver 25G on L1, L1+, and L2/L3 systems that ensure no part of the trading execution pipeline is held back. Existing Arista applications are 25G-ready, and the FDK used by partners has a 25G MAC.

The new systems deliver end-to-end 25G solutions with 2.5x higher performance and bandwidth, addressing the challenges of increasing market data volumes and the risks caused by queuing and congestion.

So what does that leave to be solved? The small problem that 25G Ethernet “requires” FEC. 

Arista Enhanced Performance Optics

25G as a published IEEE standard uses Forward Error Correction (also known as FEC) to overcome the technical challenges of reliably running higher speed signals over longer links to automatically detect and correct errors. This is not a problem for traditional data center and enterprise networks, but for trading it imposes a per hop latency penalty of as much as 250ns that is a barrier to the use of 25G for ULL networks. 

With the release of a set of unique multimode and single mode Enhanced Performance optics, we enable 25G and 100G links without FEC, saving as much as 250ns for each link in the path. 

The future of high-speed 25G Ethernet for high-frequency trading and market data has arrived!

References: