High-speed serial links, and their associated protocols, are now everywhere: for example, SATA, PCIe, HDMI, DVI-D, Display Port, Firewire, USB, SAS, and Fibre Channel, to name but a few. (See also Max Maxfield's recent blog on Aurora -- a scalable, lightweight, high-speed serial interconnect core and communications protocol.)
The ability to send bits down a pair of wires, rather than down a wide bus, makes the high-speed digital world possible. This is obviously a technology that's worth knowing something about, but how does one set about learning it? Well, as with most things, my feeling is that there is no better way to understand high-speed serial links than to implement one from the ground up, so that is what I've set out to do.
For my experiments, I've been using my Spartan 3E-based Papilio One development board (click here to see a special offer on this board), and I am attempting to transfer data at about 400Mb/s. I guess this is what is termed a "stretch goal" in industry. Here is a picture of my test setup:
I've deliberately chosen to use the Papilio One development board because (a) it's inexpensive enough that you can afford to own two and (b) its Spartan 3E FPGA does not come equipped with any inbuilt SERDES functionality, thereby increasing my "learning opportunities."
In its favor, the Spartan 3E FPGA does support the Low Voltage Differential Signaling (LVDS) standard, which is excellent for maintaining signal integrity at high speeds. Rather than one wire, LVDS employs a pair of wires to carry the signals. The extra wire carries the logical inverse of the desired signal, thereby allowing the receiver to be sensitive to the relative voltage differential between the two wires, rather than to the absolute voltage differential between a single signal wire and ground.
The current and voltage transmitted on a LVDS pair is carefully controlled, and this allows the receiver to absorb all of the energy in the signal without causing reflections in the form of excess energy bouncing back towards the transmitter. Also, since the "signal" and "anti-signal" are transmitted at the same time, there should be no net current imbalance, so minimal current has to flow over the ground connection, enhancing performance.
For my experiments, the transmitting of the data is easy -- even a lowly Spartan 3E can easily generate Low Voltage Differential (LVD) signals at 512Mb/s just by using a DDR output in a 256MHz clock domain, the only trick is to set the Vcco voltage to 2.5V. (The Papilio One has a jumper block for this).
With transmission taken care of, all that is left to do is the hard work in the form of actually receiving the data. The Internet (as always) provides a few hints. At first, this paper looked promising because if featured the same FPGA as the Papilio One achieving 400Mb/s -- but I soon realized it wasn't suited for what I had in mind.
Their design uses the dynamic phase shift feature in a DCM (Digital Clock Manager) inside the FPGA to track where the transitions in the bitstream occur, and this phase tracking has a "wall" of 3ns. If I have a design running at 200MHz (5ns), there is only 2ns of achievable adjustment.
Another issue is that -- in this paper's test setup -- the receiver and the transmitter were implemented on a single FPGA using an external loop-back for transmission, and the signal was referenced to the same clock source. Since I am using two boards, their clocks will be running at slightly different frequencies, which mean my boards will drift in and out phase with each other. So, "close, but no banana," as they say.
Here on All Programmable Planet (APP), we've been chatting about propagation delays and routing delays, so I've decided to take advantage of these effects in my design. In my next column, I'll detail how a receiver for the link was implemented.