MCE flat-file format

This document describes one of MAS/MCE's output file formats. The "DAS flat-file" format is closely related to the form that frame data have when they arrive at the PC from the MCE. This format has many weaknesses as far as end users are concerned. Only flat-files with "header version 6 and higher" are described here.

The data consist of any number of "frames". In a given flat-file, all frames should have the same size. The frame size can be determined from the first 28 bytes of the file. No MCE flat-file will be shorter than 28 bytes.

The natural data unit of MCE data is the 32-bit unsigned integer. We refer to 32-bit integers as "words". All offsets are expressed in terms of words, and should be multiplied by 4 to get the byte offset.

An MCE/DAS frame consists of the following:

• 43 words of header information
• 8 × rcs_to_report_data × num_rows_reported words of data
• 1 word of checksum information

Frame Header

The Frame Header is present at the start of every data frame. The data contained within are values that could change over the course of a data acquisition. In some cases, the data acquisition software should monitor the values of certain words or bits to ensure that the acquisition is healthy.

Header version History:

• Up to September 2007, the Frame Header had been changing sporadically and in an un-recorded fashion with no frame Header version number reported. (cc 4.0.1).
• In September 2007, the Frame Header structure was formalized as described here. (header version 6 or cc 4.0.2)
• Since September 2007, a few bits have been added to word '0' or "Frame Status" word without updating the header version (still at 6). To help determine which bits are present, firmware version numbers are listed in the table for Frame Status Bits below. (cc v04.0.2+)
• Header version 7 adds bit 9 to the frame-status word (word 0) to report the level of the non-manchester encoded fibre input of the clock card. (cc 5.0.b+)

Frame Header Words

The frame header is 43 32-bit words long. All words below have remained unchanged since the inception of version '6' (= cc_04000002+) of the header, except for word '0': Frame Status Bits. In practice, only the first 12 words are important. Words 34-40 (the PSUC block) are not meaningful unless a PSUC (obsolete) is installed.

Word	Byte				Category	Detail
	3	2	1	0
0	Frame Status Bits				Status/Structure	described below
1	CC Frame Counter				Status	internal counter, starts at the value of "cc ret_dat_s" and increments by 1 for each output frame.
2	row_len				Timing	Dwell time of the multiplexer on each row, in units of 1/50 MHz
3	num_rows_reported				Structure	Number of rows in each column that are reported in each output frame.
4	data_rate				Timing	Number of internal frames that MCE processes before outputting one to the user.
5	CC ARZ Counter				Timing	internal counter, counts number of address-return-to-zero (ARZ) or number of internal frames.
6	Header Version				Structure	starting header version 6, this word encapsulates the version number, i.e., 6, 7, etc.
7	Ramp Value				Status	Current value of the internally ramped parameter
8	ramp_card_addr		ramp_param_id		Status	Card and parameter address of the internally ramped parameter
9	num_rows				Timing	Number of rows that the MCE is servoing
10	Sync Box Number				Status	Synchronization number, originating at the sync box, that is used to synchronize data from multiple MCEs and the housekeeping system
11	run_id				Status	Contents of "cc run_id", typically the ctime (unix time) associated with the acquisition.
12	user_word				Status	Contents of "cc user_word", which the user may use to store other significant information
13	errno				Status	A bitfield of error bits
14	FPGA Temperature, AC				Housekeeping	Only updated in internal_cmd_mode 1
15	FPGA Temperature, BC1
16	FPGA Temperature, BC2
17	FPGA Temperature, BC3
18	FPGA Temperature, RC1
19	FPGA Temperature, RC2
20	FPGA Temperature, RC3
21	FPGA Temperature, RC4
22	FPGA Temperature, CC
23	errno				Status	The difference between this and the previous errno in word 13 indicates errors in the FPGA temperature block
24	Card Temperature, AC				Housekeeping	Only updated in internal_cmd_mode 1
25	Card Temperature, BC1
26	Card Temperature, BC2
27	Card Temperature, BC3
28	Card Temperature, RC1
29	Card Temperature, RC2
30	Card Temperature, RC3
31	Card Temperature, RC4
32	Card Temperature, CC
33	errno				Status	The difference between this and the previous errno in word 23 indicates errors in the Card temperature block
34	—				Reserved	Used for PSUC read-out, if one is installed. See Legacy File-Formats for details
35
36
37
38
39
40
41	errno				Status	The difference between this and the previous errno in word 33 indicates errors in the PSUC block
42	box_temp				Housekeeping	Subrack temperature readout; only updated in internal_cmd_mode 1

Frame Status Bits

Some bits have been added without upping the the Frame Header Revision from 6 (CC 4.0.2+). See the CC Firmware Revision column below for more information. Header Version 7 adds bit 9.

Bits 10-13 and 16-19 are especially relevant for determining the frame size and data structure.

Bit	Category	Detail	CC Firmware Revision
0	Status	Last frame bit, indicates that current acquisition cycle has completed. This does not necessarily indicate the last frame in a flat-file.	All Revisions
1	Status	Stop bit, indicates that MCE was commanded to stop acquisition and this is the last frame.	All Revisions
2	Status	Sync box free run mode bit, indicates that sync box is generating the data valid information (rather than accepting an external trigger)	3.0.4, 4.0.0+
3	Status	Sync box error bit, asserted when a data-valid pulse was received while transmitting the previous data frame and therefore not acted upon (skipped)	4.0.2+
4	Status	Active Clock, indicates whether the clock source is the on-board 25MHz crystal-based clock (0) or Sync Box (1)	3.0.4, 4.0.0+
5	Status	TES bias square wave level	3.0.4 only
6	Reserved
7
8
9	Status	level of U23 fibre input (non-manchester-encoded and not populated on RevD cards) at the time of the time the frame is output	5.0.b+ (header version 7+)
10	Structure	Readout card 1 data are present	4.0.2+
11	Structure	Readout card 2 data are present
12	Structure	Readout card 3 data are present
13	Structure	Readout card 4 data are present
14	Reserved
15
16	Structure	Number of columns returning data, per readout card.	4.0.a+
17
18
19
20	Status	Data timing error (CC timed-out waiting for an external DV during an acquisition). Once the DTE bit is set, it remains set until the acqusition is terminated.	5.0.1+
21	Reserved
22
23
24
25
26
27
28
29
30
31

Timing parameters

The frame rate of the data can be determined from data in the header:

• row_len (word two)
• data_rate (word four)
• num_rows (word nine)

The internal frame rate of the MCE (f_ARZ, the rate at which a given row is servoed) and the output frame rate of the MCE (f_DV, the rate at which frames are returned to the PC and recorded to disk) can be calculated as follows:

fARZ = 50 MHz / (num_rows × row_len)
fDV   = fARZ / data_rate

The address-return-to-zero (ARZ) counter in the frame header increments by data_rate between consecutive output frames. This can be used to detect dropped frames in the MAS output stream. Being only 32 bits wide, the time it takes for the ARZ counter to overflow and wrap around to zero is:

pARZ overflow = 232 / fARZ
             ≈ 86 seconds × num_rows × row_len

For a num_rows of 41 and row_len of 64, the ARZ counter overflows every 2.6 days. The ARZ counter increments whenever the MCE is servoing, even if data are not being output.

Structure determination

The size and structure of the readout data block can be determined from data in the header:

• num_rows_reported (word three)
• bits 10-13 of the Frame Status Bits (word zero), which we will call RC1_on, RC2_on, RC3_on, RC4_on (each taking a value of 1 or 0)

In particular we can determine the number of readout cards reporting, as

nRC = (RC1_on + RC2_on + RC3_on + RC4_on)

and this gives the size of the readout data block, in words, as

ndata = 8 × num_rows_reported * nRC

with 8 representing the number of columns managed by each readout card.

The organization of the frame data is described in the next section.

Frame data

Normal acquisitions (row / column)

By "normal acquisitions", we mean everything except data mode 12 (or 3).

Each word of MCE data is associated with a unique row and column in the multiplexing scheme. Each readout card is responsible for 8 columns and up to 41 rows. Columns 0-7 are associated with readout card 1, columns 8-15 are associated with readout card 2, and so on. MCE can support up to 32 columns.

Schematically, the data is organized like this (with rNNcMM indicating the data assoicated with row NN and column MM):

r00c00 r00c01 r00c02 ... r00c31
r01c00 r01c00 r01c02 ... r01c31
...
r40c00 r40c01 r40c02 ... r40c31

When num_rows_reported is less than 41, then the data for rows >= num_rows_reported is omitted. When a readout card is not reporting data, the data for its associated columns are omitted. For example, if num_rows_reported is 20 and only RC2 is reporting data then the frame structure is

r00c08 r00c09 r00c10 ... r00c15
r01c08 r01c09 r01c10 ... r01c15
r02c08 r02c09 r02c10 ... r02c15
...
r19c08 r19c09 r19c10 ... r19c15

The data for row r of card n can be found starting at index

 ir,n = r × ncard × 8

The adjustment to this index to get the data associated with column C depends on which cards are reporting data.

Note that currently MCE only supports single-card and all-card modes; so the slightly more complicated cases of, for example, reporting from readout cards 2 and 4 are not available. The format described here does uniquely specify a way of handling those cases, however.

Checksum

The checksum word is the result of a cumulative XOR of all other words in the frame, including the header. Since the XOR of any number with itself is zero, the cumulative XOR of all words in the frame (including the header, data, and checksum) should be zero.

Legacy File-Format Documents

• MCE File Formats [Rev. 2] [Rev. 3.6]