MCE config template system

An important component of the current MCE configuration system is the "mce_config_template" file. This file is a bash script that puts the MCE in some desired state, which usually means that it sets up the biases and locking points of the squids and series arrays to the values determined in the most recent auto-tuning.

General structure

In previous versions of MCE software, there was a template file called config_mce_auto_setup_<date>, with date of the form YYYYMMDD, that lived in the /data/cryo/<date> folder (which was linked to /data/cryo/current_data and was the destination for all MCE data files). When the auto-tune was run, certain lines of the config script were updated to reflect the biases and locking points discovered by the auto-tune code. The auto-tune code would then run the updated config script to load the values into the MCE in preparation for the next stage.

In order to create a more flexible and robust system, the auto-tune programs no longer edit the config script directly. Instead, there is configuration file, called "experiment.cfg", where the auto-tune programs can read and write values. A config script is then generated by running "mce_make_config", which creates a config script based on the values in experiment.cfg.

The auto-tune read and write is accomplished via IDL functions called "load_exp_params" and "save_exp_params". These functions are in "load_exp_params.pro", which is automatically generated by the program mas_param based on a template for experiment.cfg. They allow IDL to import settings from experiment.cfg as an IDL structure, manipulate them using normal IDL vector operations, and then save the structure data back to the file.

The mce_make_config script constructs a config script by combining various pre-defined blocks of bash code with a block of bash variable declarations that is generated by mas_param from experiment.cfg.

experiment.cfg

The experiment.cfg file structure is that of libconfig (http://www.hyperrealm.com/libconfig/). Here are some typical lines:

# output level and time interval for driving detectors normal.

tes_bias_idle = [0, 0, 0];
tes_bias_normal = [65000, 65000, 65000];
tes_bias_normal_time = 0.1;

# Note that each entry of flux_quanta_rc# is repeated 41 times and
# written to 'rc# flux_quanta%'

flux_quanta = [ 0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0 ];

Although libconfig permits complex nested data structures, experiment.cfg is restricted to root-level simple data types (strings, floats, integers, arrays of floats, and arrays of integers).

Unfortunately, experiment.cfg contains values associated with

• very stable properties that define the hardware configuration (e.g. number of rows of squids and sync box presence/absence)
• properties that must be manipulated during auto-tuning but then should return to some experiment default (e.g. rc data_mode)
• settings that a user can manipulate to select different auto-tuning options (e.g. the list of rows for which to generate sq1 bias ramp plots)
• properties that are discovered or revised at various stages of the auto-tuning (e.g. the sq2 feedback lock points)

It is necessary to distinguish between values that will be used to generate the config script and values that should be used to generate the /final/ config script. For example "default_data_mode = 2;" tells auto-tune what data mode to set in its final config script generation; in the mean time it will change the line "data_mode = 2;" to all sorts of other values for doing ramps and servos.

There are 2 copies of experiment.cfg in use at any time. The first is the working copy, kept in $MAS_DATA, which is edited by auto-tune and used to generate the config script. The second is kept in $MAS_TEMPLATE, and is a clean copy where experiment parameters are recorded. The $MAS_TEMPLATE copy is the one that should be commented carefully, and kept under version control.

The set_directory script performs a number of tasks, including copying experiment.cfg from $MAS_TEMPLATE into $MAS_DATA if there is no copy there already. In particular, if set_directory decides that it is necessary to create a new daily folder (and move the current_data link to point there), then a fresh copy of experiment.cfg is copied from $MAS_TEMPLATE to the new $MAS_DATA folder.

Another way to replace $MAS_DATA/experiment.cfg with the $MAS_TEMPLATE copy is to remove it and then run set_directory.

Ok, I'll level with you

This is broken, because partial tunings (short=2) will trash experiment.cfg with experiment_template.cfg. I think it can be fixed; we just have to separate "final settings" from "set this to this now" settings.