Runfile format v2

From MCEWiki

"Design" considerations

The following considerations have shaped the format described below:

  • the runfile should require some minimal set of information pertinent to almost all MCE acquisition contexts (such as the internal state of the MCE at the time of acquisition)
  • the format should be flexible enough that additional information can be included, or not, depending on the acquisition context (e.g. IV curves, parameter ramps, or normal acquisition)
  • changes or additions to runfile content should not break other applications that need to access the runfile

Some constraints on the runfile format are:

  • MCE status information (the <HEADER> stuff) is generated by mce_status in the DAS format, which we don't like changing
  • several existing applications (KST, some IDL scripts) already read or write runfiles in the XMLish format that DAS started

Other very good ideas that have NOT (to date) been addressed in this format:

  • the format should follow some established standard, such as XML or JSON
  • entries in the file should describe their data type (e.g. integer, string)


Outline

Data is added to the runfile in blocks. Blocks are designed so that applications can ignore whole blocks of information if they suspect that they won't understand the content (i.e. they would choke on data types or get confused by new/missing entries). Reading the runfile should involve first finding your block and then trying to parse the information from the tags it contains.

Basic formatting

The .run file contains ascii data only.

Whitespace is used to separate words, and can consist of any number of tabs and spaces. Words are case-sensitive. Whitespace at the beginning of a line should be ignored. After any leading whitespace, each line must either

  • begin with left angle-bracket "<"
  • begin with hash "#"
  • be blank

Lines that are blank or begin with "#" are comments and can be ignored.

Lines that begin with "<" are either part of a block delimiter set or a data tag. We'll only talk about these lines from here on.

Data structure

The data will contain some number of "tag-blocks," each of which contains some number of "tags." A tag has the format 

 <tag_name [specifier [...] ]> [DATA ...]

and a block has the format 

 <block_name>
   <tag ...> ...
   ...
   <tag ...> ...
 </block_name>

A .run file consists of a series of blocks; there can be no tags outside blocks. Blocks may not be nested.

Block names should not be repeated in the same file. Within a block, tag names need not be unique, but the ensemble of tag_name and its specifiers must be.

Mandatory run-file data

FRAMEACQ Block

The mandatory data will be contained in a block called "FRAMEACQ". It should contain at least the tags called "RUNFILE_VERSION", "DAS_VERSION", "RC", "DATA_FILENAME". The block will look like this: 

<FRAMEACQ>
  <RUNFILE_VERSION> 2
  <MAS_VERSION> MAS/trunk/439
  <RC> 2
  <DATA_FILENAME> act_123
  <DATA_FRAMECOUNT> 1000
</FRAMEACQ>

The data of RUNFILE_VERSION should consist of a single integer. For the v2 runfile format, it should be the number 2. The data of MAS_VERSION should consist of a single-word string. The data of RC is a list of non-negative integers.

Here <RC> lists the read-out cards that have been queried for data. For multiple cards, the indexes of the cards are listed. For example, reading 32 columns from a 5MDM subrack will result in 

 <RC> 1 2 3 4

If the RC argument is a single 0, this identifies the data as not originating from read-out cards.

For normal RC arguments, DATA_COLUMNS and DATA_ROWS may be used to specify the number of columns and rows per read-out card: 

 <RC> 1 2 3 4
 <DATA_COLUMNS> 8
 <DATA_ROWS> 33

If DATA_ROWS is not present, the number of data rows must be determined from information in the HEADER block (see below).

If DATA_COLUMNS is not present, the number of data columns per read-out card must be assumed to be 8.

The "<RC> 0" case should be treated as a single read-out card, i.e. the total number of data words in the frame will simply be DATA_COLUMNS*DATA_ROWS.

To distinguish between TEXT and BINARY output frame data, the tag "DATA_FORMAT" can be used. 

 <DATA_FORMAT> TEXT

If DATA_FORMAT is not present, the frame data should be assumed to be in BINARY format.

The DATA_FILENAME tag gives the name of the frame data file associated with this .run file, or, if file-sequencing is enabled, the base name. If file-sequencing is enabled then the FRAMEACQ block must contain the tags "FILESEQ_FORMAT" and "FILESEQ_INTERVAL", e.g. 

 <FILESEQ_FORMAT> %s.%03i
 <FILESEQ_INTERVAL> 240000

which indicate how to construct the data filenames from the basename (DATA_FILENAME) and the number of frames to expect in each file.

The DATA_FRAMECOUNT tag indicates the number of frames that were requested in the acquisition.


HEADER block

The HEADER block is produced by MAS' mce_status application (formerly by DAS' mcestatus function) and indicates the configuration of the MCE. The tags in this block have the form 

 data...

where 'card' and 'param' describe the MCE address of the data. The data consist of at least 1 unsigned, 32 bit integer, expressed in decimal format, possibly with leading zeros. For example, 

<RB rc1 sa_bias> 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

Determining frame data size and sampling rate from the runfile

This is no longer necessary, the MCE frame header has enough information to extract the frame rate and frame size. For reference, though:

The output frame rate is a function of the values of the tags "RB cc num_rows", "RB cc row_len", and "RB cc data_rate". The frame rate is 

fsample = 50 MHz / ( num_rows * data_rate * row_len )

The frame size depends on the number of readout cards queried for data (nRC), and the tag "RB cc num_rows_reported". The frame size, in bytes is 

nframe = 4 * (44 + 8 * nRC * num_rows_reported)

Determining data mode from the runfile

It is necessary to determine the data mode for each readout card using the runfile. The tags to examine are: 

RB rc1 data_mode
RB rc2 data_mode
RB rc3 data_mode
RB rc4 data_mode

These tags (some of which may not be present on your system) each contain a single integer describing the data mode. The meaning of the data mode codes is described in the Data mode page.

Optional run-file data

The following block is adopted by some MCE scripts to specify one or more parameters that change per frame in a ramp manner: 

<par_ramp>
 <loop_list> loop1 loop2
   <par_list loop1> par1 par2
      <par_title loop1 par1> sa_bias
      #<par_step loop par> start step_size num_steps
      <par_step loop1 par1> 15000 1500 25
      <par_title loop1 par2> offset
      <par_step loop1 par2> 15000 1500 25
   <par_list loop2> par1
      <par_title loop2 par1> sa_fb
      <par_step loop2 par1> 0 160 400
</par_ramp>

Note that loop1 and loop2 are nested loops. Therefore, in this example sa_bias, offset, and sa_fb are specified for 25x400 frames contained in the corresponding data file.

Backwards compatibility

There are already many run-files that do not have this format. They can be identified by the lack of FRAMEACQ block. The files may still contain what look like <DAS_VERSION> and <RC> tags, but outside of any blocks. If these tags are not present, the run is most likely for single RC card and from das_versions prior to 011220070826.

Examples

# auto_setup_squids.pro output
<SQUID>
  <SQ_tuning_completed> 0
  <SQ_tuning_date> 20070831
  <SQ_tuning_dir> 1188543226
</SQUID>

# mcestatus output
<HEADER>
  <RB sys row_len> 00000064 00000064 00000064 00000064 00000064 00000064 00000064 00000064 00000064 
  ...
  <RB cc fw_rev> 50331652
  ...
  <RB sys num_rows> 00000033 00000033 00000033 00000033 00000033 00000033 00000033 00000033 00000033 00000033
  ...
</HEADER>

# runfile_stamp output
<FRAMEACQ>
  <RUNFILE_VERSION> 2
  <MAS_VERSION> MAS/trunk/439
  <RC> 2
  <DATA_FILENAME> 118800005
  <DATA_FRAMECOUNT> 480000
</FRAMEACQ>
Retrieved from "https://e-mode.phas.ubc.ca/mcewiki/index.php?title=Runfile_format_v2&oldid=6488"