MCE flat-file format
This document describes one of MAS/MCE's output file formats. The "DAS flat-file" format is closely related to the form that frame data have when they arrive at the PC from the MCE. This format has many weaknesses as far as end users are concerned. Only flat-files with "header version 6" are described here.
The data consist of any number of "frames". In a given flat-file, all frames should have the same size. The frame size can be determined from the first 28 bytes of the file. No MCE flat-file will be shorter than 28 bytes.
The natural data unit of MCE data is the 32-bit unsigned integer. We refer to 32-bit integers as "words". All offsets are expressed in terms of words, and should be multiplied by 4 to get the byte offset.
An MCE/DAS frame, up to and including header version 6, consists of the following:
- 43 words of header information
- 8*R*N words of data, where N is the number of readout cards reporting and R is the number of rows reporting.
- 1 word of checksum information
Contents
Frame Header
The Frame Header is present at the start of every data frame. The data contained within are values that could change over the course of a data acquisition. In some cases, the data acquisition software should monitor the values of certain words or bits to ensure that the acquisition is healthy.
History:
- Up to September 2007, the Frame Header had been changing sporadically and in an un-recorded fashion, and the Frame Header version number was not included in the Header. This pertains to Clock Card firmware versions cc_v04000001-.
- In September 2007, Joe Fowler asked UBC to make certain changes to the Frame Header, and provide a way of differentiating between it and previous headers. It was determined from CVS that there had been 5 previous versions of the header, so the new one was called version 6, and a new constant called DATA_PACKET_HEADER_REVISION was introduced (reply_queue.vhd). This pertains to cc_v04000002.
- Since September 2007, a few bits have been added to word '0' of the Frame Header, called "Frame Status Bits". However, the Frame Header version number has not been upped -- it is still at '6'. This is not super awesome, so to help determine which bits are present, firmware version numbers are listed in the second table below. This pertains to Clock Card revisions cc_v04000002+.
Frame Header Words
- Up-to-date as of 21 April 2010
- The frame header is 43 words long. The table below describes the most useful fields in the header. Esoteric fields are omitted; absent offsets are not necessarily unused.
- All words below have remained unchanged since the inception of version '6' (= cc_04000002+) of the header, except for word '0': Frame Status Bits.
| Offset | Name | Category | Detail |
|---|---|---|---|
| 0 | Frame Status Bits | Status/Structure | described below |
| 1 | CC Frame Counter | Status | internal counter, starts at the value of "cc ret_dat_s" and increments by 1 for each output frame. |
| 2 | row_len | Timing | Dwell time of the multiplexer on each row, in units of 1/50 MHz |
| 3 | num_rows_reported | Structure | Number of rows in each column that are reported in each output frame. |
| 4 | data_rate | Timing | Number of internal frames that MCE processes before outputting one to the user. |
| 5 | CC ARZ Counter | Timing | internal counter, counts number of address-return-to-zero (ARZ) or number of internal frames. |
| 6 | Header Version | Structure | 6, the number "six". Subject to change in future header versions. |
| 7 | Ramp Value | Status | Current value of the internal ramping parameter |
| 8 | Ramp Address | Status | Card and parameter address of the internal ramping parameter (bits [31-16] are card_addr, bits [15-0] are param_id) |
| 9 | num_rows | Timing | Number of rows that the MCE is servoing |
| 10 | Sync Box Number | Status | Synchronization number, originating at the sync box, that is used to synchronize data from multiple MCEs and the housekeeping system |
| 11 | run_id | Status | Contents of "cc run_id", typically the ctime (unix time) associated with the acquisition. |
| 12 | user_word | Status | Contents of "cc user_word", which the user may use to store other significant information. |
A more complete description of the version 6 frame header can be found in the document "MCE File Formats", version 3.3:
http://www.phas.ubc.ca/%7Emce/mcedocs/index.html
This document is out of date in some respects, but the descriptions of the frame headers up to version 6 should be accurate.
Frame Status Bits (Header Word #0)
- Up-to-date as of 21 April 2010
- The table below describes the most useful fields in the header. Esoteric fields are omitted; absent offsets are not necessarily unused.
- Since this word's inception, some bits have been added without upping the the Frame Header Revision from '6' (= cc_04000002+). See the CC Firmware Revision column below for more information.
| Bit | Category | Detail | CC Firmware Revision |
|---|---|---|---|
| 0 | Status | Last frame bit, indicates that current acquisition cycle has completed. This does not necessarily indicate the last frame in a flat-file. | All Revisions |
| 1 | Status | Stop bit, indicates that MCE was commanded to stop acquisition and this is the last frame. | All Revisions |
| 2 | Status | Sync box free run mode bit, indicates that sync box is generating the data valid information (rather than accepting an external trigger) | (cc_v03000004, and cc_v04000000+) |
| 3 | Status | Sync box error bit | (cc_v04000002+) |
| 4 | Status | Active Clock | (cc_v03000004 and cc_v04000000+) |
| 5 | Status | TES bias square wave level | (reported ONLY in cc_v03000004) |
| 10 | Structure | Readout card 1 data are present | (cc_v04000002+) |
| 11 | Structure | Readout card 2 data are present | (cc_v04000002+) |
| 12 | Structure | Readout card 3 data are present | (cc_v04000002+) |
| 13 | Structure | Readout card 4 data are present | (cc_v04000002+) |
| 19:16 | Structure | Number of columns returning data, per readout card. | (cc_v0400000a+). |
| 20 | Status | Data timing error. Will be followed shortly by a stop frame! | (cc_v05000001+) |
Bits 10-13 and 16-19 are especially relevant for determining the frame size and data structure.
Timing parameters
The internal frame rate of the MCE (the rate at which a given row is servoed) and the output frame rate of the MCE (the rate at which frames are returned to the PC and recorded to disk) can be calculated as follows:
f_internal = 50 MHz / (num_rows * row_len) f_output = f_internal / data_rate
Structure determination
The size and structure of the readout data block can be determined from
- num_rows_reported
- bits 10-13 of the frame status bits, which we will call RC1_on, RC2_on, RC3_on, RC4_on (each taking a value of 1 or 0)
In particular we can determine the number of readout cards reporting, as
N_card = (RC1_on + RC2_on + RC3_on + RC4_on)
and this gives the size of the readout data block, in words, as
N_data = 8 * num_rows_reported * N_card
with 8 representing the number of columns managed by each readout card.
The organization of the frame data is described in the next section.
Frame data
Normal acquisitions (row / column)
By "normal acquisitions", we mean everything except data mode 3.
Each word of MCE data is associated with a unique row and column in the multiplexing scheme. Each readout card is responsible for 8 columns and up to 41 rows. Columns 0-7 are associated with readout card 1, columns 8-15 are associated with readout card 2, and so on. MCE can support up to 32 columns.
Schematically, the data is organized like this (with rNNcMM indicating the data assoicated with row NN and column MM):
r00c00 r00c01 r00c02 ... r00c31 r01c00 r01c00 r01c02 ... r01c31 ... r40c00 r40c01 r40c02 ... r40c31
When num_rows_reported is less than 41, then the data for rows >= num_rows_reported is omitted. When a readout card is not reporting data, the data for its associated columns are omitted. For example, if num_rows_reported is 20 and only RC2 is reporting data then the frame structure is
r00c08 r00c09 r00c10 ... r00c15 r01c08 r01c09 r01c10 ... r01c15 r02c08 r02c09 r02c10 ... r02c15 ... r19c08 r19c09 r19c10 ... r19c15
Mathematically, the data for row R can be found starting at index
I_R = R * N_card * 8
The adjustment to this index to get the data associated with column C depends on which cards are reporting data.
Note that currently MCE only supports single-card and all-card modes; so the slightly more complicated cases of, for example, reporting from readout cards 2 and 4 are not available. The format described here does uniquely specify a way of handling those cases, however.
Checksum
The checksum word is the result of a cumulative XOR of all other words in the frame, including the header. Since the XOR of any number with itself is zero, the cumulative XOR of all words in the frame (including the header, data, and checksum) should be zero.
