MCE fibre protocol
- This document describes the low-level MCE protocol, and should be of interest mainly to firmware developers. Most end-users of the MCE will be more interested in the high-level commanding and data handling provided by MAS as described primarily in the mce_cmd document.
This document describes the data exchange between the MCE and the controlling PC over the fibre-optic link. On the PC, communications are handled by the PCI card firmware. On the MCE, the communications are handled by the Clock Card firmware.
Contents
Physical layer
The communication between the PC and the MCE is through a duplex fibre-optic link. The fibre-optic receive/transmit circuitry contains Agilent parts HFBR1119/2119 operating in the IR regime at a bit rate of 250MHz, derived from Cypress HotLink, CY923/33, parallel-to-serial and serial-to-parallel converters that synthesize a 250MHz clock from an input clock of 25MHz. HOTLink is an 8B/10B physical layer for point-to-point communication over high-speed serial link with special characters defined, but no HOTLink protocol standard exists. These parts require ten clocks to transmit or receive each byte, so the 250MHz bit rate translates into 25MB/s. On the PC side, the PCI card from San Diego State University (SDSU) handles data acquisition from the MCE. On the MCE side, the Clock Card has the fibre interface circuitry.
Data link layer
The communication protocol between the data-acquisition PC and the MCE consist of three types of packets. Packets originating from the PC are referred to as command packets and packets originating from the MCE are referred to either as reply packets or data packets. Each command packet has to be acknowledged by the MCE through a reply packet within a limited time; if no reply is received then the PC issues a timeout warning and stops waiting. As a general rule, the PC ignores unsolicited reply or data packets from the MCE. Consequently, a late reply packet is ignored. Each command has to be acknowledged (or timed out) before another command is issued. The requirement prevents packet collision in the protocol. There are two special command types:
- GO: a reply packet is followed by one or many data packets and the last data packet is marked with the LAST bit set
- ST: one or many data packets with the last data packet marked with the LAST bit set are followed by a reply packet.
A packet consists of 32-bit little-endian words. The asynchronous nature of communication requires that all valid packets (command/reply/data) include two 32-bit preamble words (0xA5A5A5A5 followed by 0x5A5A5A5A) to indicate the start of a valid packet. The transmitter inserts a simple XOR checksum as the last word in the packet. The receiver regenerates the checksum based on the packet data and compares it with the one included in the packet to detect bit errors. The simple checksum offers a naive error detection mechanism, but no error correction mechanism exists in the protocol. The decision was made based on low fibre bit-error rates. A bit-error rate (BER) < 10−19 has been measured.
Packet structure and content
There are three types of packets exchanged between the MCE and the PC. Each is described below in turn.
Command packet
A command packet is sent by the PC to the MCE. It's is fixed at 64 words long and looks like this:
| 32-bit word offset | Name | Value / Description | |||
|---|---|---|---|---|---|
| Byte | 3 | 2 | 1 | 0 | |
| 0 | Preamble | 0xA5A5A5A5 0x5A5A5A5A | |||
| 1 | |||||
| 2 | Command type | 0x2020xxxx where xxxx is one of ASCII "RB", "WB", "GO", "ST", "RS" | |||
| 3 | Card ID | Param ID | Both IDs are 16-bits wide. See Card ID for card IDs and MCE commands for param IDs | ||
| 4 | Data payload size | between 1 and 58 inclusive | |||
| 5 | Data payload | data[00] or 0x00000000 if unused
| |||
| 6 | data[01] or 0x00000000 if unused
| ||||
| ... | ... | ||||
| 62 | data[57] or 0x00000000 if unused
| ||||
| 63 | Checksum | XOR checksum of words 5 through 62 | |||
- Word 2 is the command type:
Command types Type Value Description RB 0x20205242 Read block to read back a parameter from the MCE WB 0x20205742 Write block to set an MCE parameter GO 0x2020474F Start a data-acquisition process on the MCE ST 0x20205354 Stop a data-acquisition process on the MCE RS 0x20205253 Reset an MCE device
- Word 3 contains the MCE Card ID (upper 16 bits) and parameter address (lower 16-bits), which are enumerated in the linked documents.
- Word 4 indicates the size of the data payload (in 32-bit words) associated with the command (for everything but RB, the data are provided in words 5 through 62). For command types RB and WB, this is between 1 and 58 inclusive. For command types GO, ST, RS, this is always equal to one.
- Words 5 to word 62 contain the command data payload. For all command types except RB, the number of data words specified in word 4 (1 to 58 words) are initialised. Unused space at the end of the data payload is filled with zeros up to the end of word 62. For command type RB, word 4 indicates the number of words to return, and the entire data payload is zeroed.
- Word 63 is an XOR checksum of the data payload (words 5 through 62), including any zero padding at the end of the payload.
Reply packet
A reply packet is send from the MCE to the PC in response to the receipt of a command packet. It has a variable length, between 7 and 64 32-bit words, and looks like:
| 32-bit word offset | Name | Value / Description | |||
|---|---|---|---|---|---|
| Byte | 3 | 2 | 1 | 0 | |
| 0 | Preamble | 0xA5A5A5A5 0x5A5A5A5A | |||
| 1 | |||||
| 2 | Packet type | 0x20205250 (ASCII: "␣␣RP") indicating a reply packet | |||
| 3 | Packet size (n+3) | between 4 and 61 inclusive, where n is the data payload size | |||
| 4 | Cmd type | Err/OK | Cmd type is one of ASCII "RB", "WB", "GO", "ST", "RS"; Err/OK is ASCII "OK" or ASCII "ER" indicating success or failure | ||
| 5 | Card ID | Param ID | Both IDs are 16-bits wide. See Card ID for card IDs and MCE commands for param IDs | ||
| 6 | Data payload | data[00]
| |||
| ... | ... | ||||
| 5+n | data[n]
| ||||
| 6+n | Checksum | XOR checksum of words 4 through 5+n | |||
- Word 3 is specifies the reply packet size, specifying the number of words in the reply packet following word 3. This will be three plus the data payload size (n). The data payload size, n, is between 1 and 58 for successful RB replies (i.e. in cases when word 4 is "RBOK"), and 1 otherwise. So, this word will be between 4 and 61.
- Word 4 indicates the command type and success or failure of the command. The upper 16 bits contains the lower 16 bits of word 2 of the originating command packet (i.e. one of 0x5242, "RB"; 0x5742, "WB"; 0x474F, "GO"; 0x5354, "ST"; 0x5253, "RS"). The lower 16-bits are either 0x4F4B (ASCII: "OK") or 0x4552 (ASCII: "ER") indicating, respectively, success or failure of the command.
- Word 5 contains the MCE Card ID (upper 16 bits) and parameter address (lower 16-bits), which are enumerated in the linked documents.
- Word 6 et seq. is the reply data payload. The number of words is indicated by word 3. For successful non-RB commands, this is always a single zero word. For successful RB commands (word 4 is "RBOK"), this is the data returned. For unsuccessful commands (the lower 16-bits of word 4 is "ER") this is the reply error word:
Reply error word Bit offset Origin Description 0 PSUC Wishbone execution error 1 Backplane communcation error 2 Card not present 3 CC Wishbone execution error 4 Backplane communcation error 5 Card not present 6 RC4 Wishbone execution error 7 Backplane communcation error 8 Card not present 9 RC3 Wishbone execution error 10 Backplane communcation error 11 Card not present 12 RC2 Wishbone execution error 13 Backplane communcation error 14 Card not present 15 RC1 Wishbone execution error 16 Backplane communcation error 17 Card not present 18 BC3 Wishbone execution error 19 Backplane communcation error 20 Card not present 21 BC2 Wishbone execution error 22 Backplane communcation error 23 Card not present 24 BC1 Wishbone execution error 25 Backplane communcation error 26 Card not present 27 AC Wishbone execution error 28 Backplane communcation error 29 Card not present 30 — Internal reset occurred 31 Stale data
- Card not present is not, strictly-speaking, an error. In reply packets to the PC, the MCE will assert the bits of all cards that are missing from the MCE, but will not assert the "ER" flag, even if the missing card was commanded.
- Backplane communication error includes CRC errors and timeouts.
- Wishbone execution error includes attempts to write to read-only parameters (i.e. registers), and attempts to access a parameter that is not supported by the firmware.
- Word 6+n is an XOR checksum of the reply type, card and parameter id, and data payload (words 4 through 5+n).
Data packet
Data packets are produced in response to a prior GO command, after the GO is acknowledged by a reply packet. The data packet is of variable length, depending on how many rows are reporting (see cc num_rows_reported) and the number of readout cards reporting (see cc rcs_to_report_data). It looks like this:
| 32-bit word offset | Name | Value / Description |
|---|---|---|
| 0 | Preamble | 0xA5A5A5A5 0x5A5A5A5A |
| 1 | ||
| 2 | Packet type | 0x20204441 (ASCII: "␣␣DA") indicating a data packet |
| 3 | Packet size | (n+1) where n is the data payload size |
| 4 | Data payload | data[00]
|
| ... | ... | |
| 3+n | data[n]
| |
| 4+n | Checksum | XOR checksum of words 4 through 3+n |
- Word 3 indicates the packet size. It is one more than the data payload size (n).
- Words 4 et seq. is the data payload. It is simply the MCE data frame, which is described in detail in MCE flat-file format. It consists of:
- a 43 word header (
data[00]throughdata[42];data[00]contains the Frame Status Bits) - 8 × rcs_to_report_data × num_rows_reported data values (
data[43]throughdata[n])
- a 43 word header (
- Word 4+n is the XOR checksum of words 4 through 3+n (This value is also part of the MCE flat-file format).
Legacy documents
The following document is the original MCE communications protocol description from which this document is descended. It does not discuss any changes made after 2012 (including the version 6 firmware). It also contains information on the MCE backplane protocol (as well as the Clock Card–PSUC communications protocol, which is now obsolete).
