Difference between revisions of "Raw-mode readout"

From MCEWiki
Line 1: Line 1:
 +
 
= General Information =
 
= General Information =
* Raw data refers to 50MHz ADC samples. The MCE has a dedicated buffer for each column to store 8192 samples per column. Depending on the firmware revision of the readout-card, the buffer holds the 8 upper bits or the full 14-bit of the ADC samples. The resulting signed 8- or 14-bit number is reported as usual signed 32b.
+
 
 +
* "Raw data" and "Raw mode" refer to the acquisition of ADC samples at 50 MHz.
 +
* Raw data acquisition occurs in the 'background', i.e. it can occur while multiplexing and servoing are going on.
 +
* Special readout card firmware is required to support raw mode.  This firmware supports most normal readout card features, but does not support the low-pass filter nor the reading back of some parameters.  It should be loaded temporarily (i.e. as a .sof).  Here is the file:
 +
http://www.phas.ubc.ca/~mce/mcedocs/firmware/rc_v04030007_26may2008_raw.sof
 +
* The MCE has a dedicated buffer for each column to store 8192 samples per column. The ADC readings are 14 bits, signed, and are reported as 32-bit signed integers.
 +
* The ADC_offset '''is''' included in raw mode readings.  So the values reported are "ADC_reading - ADC_offset".  
 
* 8192 samples corresponds to two full frames if num_rows x row_len  = 4096.  Therefore, if you want to collect two full frames alter one or the other of these parameters.  For example, 41 rows x 95 samples per row is adequate.
 
* 8192 samples corresponds to two full frames if num_rows x row_len  = 4096.  Therefore, if you want to collect two full frames alter one or the other of these parameters.  For example, 41 rows x 95 samples per row is adequate.
* Raw data acquisition is only available in certain build of readout-card firmware. The latest firmware to support raw mode is: 0x04030007.  It can be downloaded from:  http://www.phas.ubc.ca/~mce/mcedocs/firmware/rc_v04030007_26may2008_raw.sof
 
* Raw data acquisition and 4-pole low-pass filter are both memory-intensive features of the MCE readout-card firmware. As a general rule, a given firmware has only one of these features available, e.g., the firmware that has raw mode enabled will not include the filter and vice versa. In some versions that support raw mode, read back of gaini0/gainp0/gaind0 is also not supported.
 
  
 +
== Raw mode firmware ==
 +
 +
Raw data acquisition and 4-pole low-pass filter are both memory-intensive features of the MCE readout-card firmware. As a general rule, a given firmware has only one of these features available, e.g., the firmware that has raw mode enabled will not include the filter and vice versa. In some versions that support raw mode, read back of gaini0/gainp0/gaind0 is also not supported.
  
 
{| border=1
 
{| border=1
 
! Firmware Revision !! reported data (signed)
 
! Firmware Revision !! reported data (signed)
 
|-
 
|-
| 4.3.7  || 14-bit ADC samples  
+
| 4.3.7  || 14-bit ADC samples (recommended)
 
|-
 
|-
 
| 4.2.7 || upper 8-bit of the 14-bit ADC samples  
 
| 4.2.7 || upper 8-bit of the 14-bit ADC samples  
Line 16: Line 24:
 
|}
 
|}
  
 +
== High(er)-level support ==
 +
 +
The script "mce_raw_acq" handles the details of disabling the sync-box, computing the number of frames the user probably wants, issuing the capture command, and reading out the data.  Use it like this:
 +
 +
* Tune and configure your array.
 +
* Run, for some RC:
 +
mce_raw_acq rc1
 +
 +
This will create files <ctime>_raw, <ctime>_raw.run.
 +
 +
== Low-level support ==
 +
 +
Raw mode readout is accomplished by first triggering the MCE to store 50 MHz data, and then reading out that data using a special data mode.
  
= Setup =
+
* There are two RC parameters that have to be set for raw data acquisition:
* There are two MCE parameters that have to be set for raw data acquisition:
+
** '''rc# captr_raw''' stores a snapshot of ''row_len'' samples of ''num_rows'' rows for '''2''' consecutive frames up to 8192 entries for each channel and stores them in MCE memory buffer for readout in raw mode. The user must delay here before proceeding to read out the data.  The delay should be a few return-to-zeros.  (Computers are fast, and RTZ can take a long time if row_len is increased for noise measurements... so be generous when running from within scripts.)
** '''captr_raw''' stores a snapshot of ''row_len'' samples of ''num_rows'' rows for '''2''' consecutive frames up to 8192 entries for each channel and stores them in MCE memory buffer for readout in raw mode.  
+
** '''rc# data_mode''' determines the pixel-data readout mode. data mode has to be set to 3 for raw mode.
** '''data_mode''' determines the pixel-data readout mode. data mode has to be set to 3 for raw mode.
 
 
* A typical sequence of commands for data acquisition is:
 
* A typical sequence of commands for data acquisition is:
 
   wb cc ret_dat_s 1 1
 
   wb cc ret_dat_s 1 1
Line 27: Line 47:
 
   acq_config ''filename'' rc1
 
   acq_config ''filename'' rc1
 
   acq_go ''row_len*2''
 
   acq_go ''row_len*2''
or
 
  mce_raw_acq
 
  
= Side effects =
+
== Data handling ==
* Low-pass Filter disabled.  
+
 
* Readback from gainp/gaini/gaind parameters not available.
+
Raw mode data files are MCE flatfiles and can be loaded using the usual mas_data.pro or mce_data.py programs.  However, the data should be time-ordered for meaningfullness.  This is done by re-ordering the dimensions and collapsing the row and time dimensions.
   
+
 
 +
In python:
 +
d = SmallMCEFile('1234000000_raw').Read(row_col=True)
 +
_, nc, _ = d.data.shape
 +
d.data = d.data.transpose((1,2,0)).reshape((nc,-1))
 +
 
 +
In IDL (it's practically a one-liner!):
 +
data = mas_data('1234000000_raw')
 +
ds = size(d)
 +
nc = ds[1]
 +
nt = ds[2]*ds[3]
 +
  data_raw = reform(data, nc, nt)
 +
 
 +
= Details =
 +
 
 +
== Data storage and packaging ==
  
= Considerations =
 
 
* Once the raw-buffer of 8192 samples is full, no more samples are stored till a new captr_raw is issued.
 
* Once the raw-buffer of 8192 samples is full, no more samples are stored till a new captr_raw is issued.
 
* The raw-buffer is shipped out of MCE in a regular frame format (header + num_rows_reported*num_columns + checksum).  As compared with normal MCE flatfile data organization, in raw mode the time index takes the place of row.  At each time, all columns are reported.
 
* The raw-buffer is shipped out of MCE in a regular frame format (header + num_rows_reported*num_columns + checksum).  As compared with normal MCE flatfile data organization, in raw mode the time index takes the place of row.  At each time, all columns are reported.
Line 77: Line 109:
  
  
= Firmware Details =
+
== Firmware ==
 
* To toggle between raw data firmware and filtering firmware, one must un-comment one or the other (not both) of the following data banks:
 
* To toggle between raw data firmware and filtering firmware, one must un-comment one or the other (not both) of the following data banks:
 
** i_raw_dat_bank in adc_sample_coadd.vhd, for enabling/disabling raw data.
 
** i_raw_dat_bank in adc_sample_coadd.vhd, for enabling/disabling raw data.
 
** i_fsfb_wn11_Q, i_fsfb_wn12_Q, i_fsfb_wn21_Q, i_fsfb_wn22_Q in fsfb_fltr_regs.vhd, for enabling/disabling filtered data.
 
** i_fsfb_wn11_Q, i_fsfb_wn12_Q, i_fsfb_wn21_Q, i_fsfb_wn22_Q in fsfb_fltr_regs.vhd, for enabling/disabling filtered data.

Revision as of 14:47, 3 April 2009

General Information

  • "Raw data" and "Raw mode" refer to the acquisition of ADC samples at 50 MHz.
  • Raw data acquisition occurs in the 'background', i.e. it can occur while multiplexing and servoing are going on.
  • Special readout card firmware is required to support raw mode. This firmware supports most normal readout card features, but does not support the low-pass filter nor the reading back of some parameters. It should be loaded temporarily (i.e. as a .sof). Here is the file:
http://www.phas.ubc.ca/~mce/mcedocs/firmware/rc_v04030007_26may2008_raw.sof
  • The MCE has a dedicated buffer for each column to store 8192 samples per column. The ADC readings are 14 bits, signed, and are reported as 32-bit signed integers.
  • The ADC_offset is included in raw mode readings. So the values reported are "ADC_reading - ADC_offset".
  • 8192 samples corresponds to two full frames if num_rows x row_len = 4096. Therefore, if you want to collect two full frames alter one or the other of these parameters. For example, 41 rows x 95 samples per row is adequate.

Raw mode firmware

Raw data acquisition and 4-pole low-pass filter are both memory-intensive features of the MCE readout-card firmware. As a general rule, a given firmware has only one of these features available, e.g., the firmware that has raw mode enabled will not include the filter and vice versa. In some versions that support raw mode, read back of gaini0/gainp0/gaind0 is also not supported.

Firmware Revision reported data (signed)
4.3.7 14-bit ADC samples (recommended)
4.2.7 upper 8-bit of the 14-bit ADC samples
4.1.7 upper 8-bit of the 14-bit ADC sample

High(er)-level support

The script "mce_raw_acq" handles the details of disabling the sync-box, computing the number of frames the user probably wants, issuing the capture command, and reading out the data. Use it like this:

  • Tune and configure your array.
  • Run, for some RC:
mce_raw_acq rc1

This will create files <ctime>_raw, <ctime>_raw.run.

Low-level support

Raw mode readout is accomplished by first triggering the MCE to store 50 MHz data, and then reading out that data using a special data mode.

  • There are two RC parameters that have to be set for raw data acquisition:
    • rc# captr_raw stores a snapshot of row_len samples of num_rows rows for 2 consecutive frames up to 8192 entries for each channel and stores them in MCE memory buffer for readout in raw mode. The user must delay here before proceeding to read out the data. The delay should be a few return-to-zeros. (Computers are fast, and RTZ can take a long time if row_len is increased for noise measurements... so be generous when running from within scripts.)
    • rc# data_mode determines the pixel-data readout mode. data mode has to be set to 3 for raw mode.
  • A typical sequence of commands for data acquisition is:
 wb cc ret_dat_s 1 1
 wb rc1 data_mode 3
 wb rc1 captr_raw 1
 acq_config filename rc1
 acq_go row_len*2

Data handling

Raw mode data files are MCE flatfiles and can be loaded using the usual mas_data.pro or mce_data.py programs. However, the data should be time-ordered for meaningfullness. This is done by re-ordering the dimensions and collapsing the row and time dimensions.

In python:

d = SmallMCEFile('1234000000_raw').Read(row_col=True)
_, nc, _ = d.data.shape
d.data = d.data.transpose((1,2,0)).reshape((nc,-1))

In IDL (it's practically a one-liner!):

data = mas_data('1234000000_raw')
ds = size(d)
nc = ds[1]
nt = ds[2]*ds[3]
data_raw = reform(data, nc, nt)

Details

Data storage and packaging

  • Once the raw-buffer of 8192 samples is full, no more samples are stored till a new captr_raw is issued.
  • The raw-buffer is shipped out of MCE in a regular frame format (header + num_rows_reported*num_columns + checksum). As compared with normal MCE flatfile data organization, in raw mode the time index takes the place of row. At each time, all columns are reported.
  • For rc1 only, for example, time indices t = [0, num_rows_reported-1] are reported in the first frame, with the offset of column c and time t being
index = t*8 + c
  • Subsequent frames contain subsequent data samples. Since the number of time samples, per column, in a full multiplexing cycle of the MCE is row_len*num_rows, the number of readout frames required to record an entire MCE internal frame's-worth of raw data is equal to row_len (assuming that num_rows = num_rows_reported).
  • The readout frame and index of a given time t, column c, is thus
frame = floor[ (t*8 + c) / num_rows_reported ]
index = (t*8 + c) mod num_rows_reported
  • Since the number of clock ticks spent on each row is row_len, the row associated with time t is
row = floor [ t / row_len ]
  • It is possible to get raw mode data from multiple readout cards; in that case the data will be packaged in the usual way, with all columns reporting at each time.
  • Schematically, the organization of raw data for num_rows = num_rows_reported = 33 and row_len = 100 can be represented as follows (here sNNNNNcXX represents a word containing the data for time sample NNNNN, column XX):
(header of frame 0, 43 words)
s0000c00 s0000c01 s0000c02 s0000c03 s0000c04 s0000c05 s0000c06 s0000c07
s0001c00 s0001c01 s0001c02 s0001c03 s0001c04 s0001c05 s0001c06 s0001c07
s0002c00 s0002c01 s0002c02 s0002c03 s0002c04 s0002c05 s0002c06 s0002c07
...
s0032c00 s0032c01 s0032c02 s0032c03 s0032c04 s0032c05 s0032c06 s0032c07
(checksum of frame 0, 1 word)

(header of frame 1, 43 words)
s0033c00 s0033c01 s0033c02 s0033c03 s0033c04 s0033c05 s0033c06 s0033c07
...
s0065c00 s0065c01 s0065c02 s0065c03 s0065c04 s0065c05 s0065c06 s0065c07
(checksum of frame 1, 1 word)

...

(header of frame 199, 43 words)
s6567c67 s6567c01 s6567c02 s6567c03 s6567c04 s6567c05 s6567c06 s6567c07
...
s6599c00 s6599c01 s6599c02 s6599c03 s6599c04 s6599c05 s6599c06 s6599c07
(checksum of frame 199, 1 word)
  • Unfortunately, collecting data for up to 2 internal MCE frames is hard-coded in firmware. Even if you reduce number of rows, you will be restricted by 2 frames. If you really want to look at more samples, then you may want to increase row_len instead and using row_order parameter of the address card, move your particular row of interest to the beginning of the frame.
  • readout_row_index parameter is ignored when data_mode is set to 3 or raw mode.
  • if num_rows_reported < num_rows, it takes more frames (to be exact: n = (num_rows*row_len*2/num_rows_reported) frames) to read all the data out.


Firmware

  • To toggle between raw data firmware and filtering firmware, one must un-comment one or the other (not both) of the following data banks:
    • i_raw_dat_bank in adc_sample_coadd.vhd, for enabling/disabling raw data.
    • i_fsfb_wn11_Q, i_fsfb_wn12_Q, i_fsfb_wn21_Q, i_fsfb_wn22_Q in fsfb_fltr_regs.vhd, for enabling/disabling filtered data.