Python data and runfile modules

Python. Way of the future.

The python modules mce_data.py and mce_runfile.py provide roughly the same functionality that mas_data.pro and mas_runfile.pro provide for IDL.

mce_data.py

This provides a way to load SMALL frame data files into python. SMALL means less than 50000 frames.

>>> from mce_data import *
>>> f = SmallMCEFile('/data/cryo/current_data/data002')
>>> d = f.Read()
18600000 items requested but only 37200 read
>>> d.data
array([[0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       ..., 
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0],
       [0, 0, 0, ..., 0, 0, 0]], dtype=int32)

The data member of d is a 2d array, [n_detectors, n_frames]:

>>> d.data.shape
(328, 100)

The list of rows and columns associated with the 328 detectors is available through col_list and row_list:

>>> d.col_list
[8, 9, 10, 11, 12, 13, 14, 15, 8, 9, 10, 11, 12, 13, 14, 15,
 8, 9, 10, 11, 12, 13, 14, 15, 8, 9, 10, 11, 12, 13, 14, 15,
 ... 
 8, 9, 10, 11, 12, 13, 14, 15]

The data from the first frame header is available in the header member:

>>> d.header
{'status': 2052, 'data_rate': 47, 'userfield': 0, 'num_rows_reported': 41,
 'runfile_id': 1231969044, 'row_len': 64, 'header_version': 6, 'rc_present':
 [False, True, False, False], 'address0_ctr': 23142826, 'num_rows': 41,
 'ramp_value': 0, 'frame_counter': 0, 'sync_box_num': 0, 'ramp_addr': 0}

If you want to extract raw data, with no conversion based on data mode, force the data mode to 0 in the Read call:

>>> d = f.Read(force_data_mode=0)

mce_runfile.py

Load a runfile:

>>> from mce_runfile import *
>>> runfile_name = '/data/cryo/current_data/1220531790_dat.run'
>>> rf = MCERunfile(runfile_name)

Recall that the structure of runfiles is such that a line of data has an address defined by its 'block' and 'key' (where the key is the tag + specifiers...). The contents of rf include a dictionary of dictionaries of all the block / key pairs. For example:

>>> print rf.data['HEADER']['RB rc1 data_mode']
 00000010
>>> print rf.data['SQUID']['SQ_tuning_dir']
 1220510497

However, the member function "Item" allows us to repackage the runfile data by specifying a data type ('string', 'int', 'float') and whether or not we expect an array or a single value. For example:

>>> print rf.Item('HEADER', 'RB rc1 data_mode')
['00000010']
>>> print rf.Item('HEADER', 'RB rc1 data_mode', type='int')
[10]
>>> print rf.Item('HEADER', 'RB rc1 data_mode', type='int', array=False)
10
>>> print rf.Item('HEADER', 'RB rc1 data_mode', type='float')
[10.0]

Some runfile entries are really 2d arrays, entered row by row. For example, in the 'IV' block there are per-column entries for responsivity:

<IV>
...
<Responsivity(W/DACfb)_C0> 1.92290e-16 1.92119e-16 0.00000 1.93100e-16 1.92978e-16 1.89769e-16 1.91119e-16 ...
<Responsivity(W/DACfb)_C1> 0.00000 1.82838e-16 0.00000 1.84197e-16 1.84822e-16 1.84447e-16 1.83693e-16 ...
<Responsivity(W/DACfb)_C2> 1.89962e-16 1.88649e-16 0.00000 1.85339e-16 1.84462e-16 1.82965e-16 1.84045e-16 ...
...
</IV>

These can be extracted column by column:

>>> print rf.Item('IV', 'Responsivity(W/DACfb)_C24', type='float')
[1.46858e-16, 1.4528800000000001e-16, 0.0, 1.4240899999999999e-16, 1.4162699999999999e-16, 1.4100399999999999e-16, 1.4088899999999999e-16, 1.39608e-16, 1.3787599999999999e-16, 1.37429e-16, 1.3614499999999999e-16, 1.34423e-16, 1.3543599999999999e-16, 1.3587200000000001e-16, 1.3711399999999999e-16, 0.0, 0.0, 1.39424e-16, 1.40202e-16, 1.4450300000000001e-16, 1.47099e-16, 1.4857100000000001e-16, 1.49762e-16, 1.5139199999999999e-16, 1.5468600000000001e-16, 1.5743600000000001e-16, 1.5934699999999999e-16, 1.62431e-16, 1.6465e-16, 1.65256e-16, 1.6683999999999999e-16, 1.6823900000000001e-16, 0.0]

But they can all be extracted at once if you pass a printf-style format string to the member function Item2d:

>>> a = rf.Item2d('IV', 'Responsivity(W/DACfb)_C%i', type='float')
>>> print len(a)
32
>>> print len(a[0])
33 
>>> print a[2][1]
1.88649e-16

(i.e. a[2][1] is the responsivity for column 2, row 1.)