MCE Control Computer

Mechanical and electrical

Physical dimensions

2U chassis:

Weight^†: ~4 kg
Dimensions^‡: 19" rack-mount (17" wide excluding front plate), 13.5" deep (including connector room), 3.5" high

Small chassis:

Weight^†: ~2 kg
Dimensions^‡: 34.75cm wide (31.5cm without lower mounting plate), 28.5cm deep (including connector room), 5.25cm high

†: weight varies with installed components; ‡: approximate; see mechanical drawings.

I/O

A labelled schematic of main panel of the MCC is presented below. Differences between the two configurations are indicated.

The two form-factors have the same basic external connections with the following exceptions:

the reset switch, hard drive and power LEDs, and DC-in power connector on the panel are populated on in small MCC but not in the 2U MCE (a blank is installed in their place). The 2U MCC has the reset switch, LEDs and as well as a power switch on the front panel (not shown in diagram).
the 2U MCC has a HP-style AC-in power connector and fuse on the back panel.
The lower four of the six USB ports are optional. They are typically not installed on a small MCC (simply for weight saving).

In MCCs with only one fibre card, typically Card #1 is missing, and the panel replaced with a blank. Note also that the two fibre card front plates are not the same shape (as can be seen in the diagram), meaning the two fibre cards can't be swapped without also swapping front plates.

Power

Power draw: 25-30 W with one fibre card (5 W more with two fibre cards); at power on, current draw can spike to ~1.5 amps at 24V momentarily.
Input power:
- The 2U MCC takes 85-264V, 50-60Hz AC mains power over a 3-pin HP-style power connector.
- The small MCC requires 24-Volt nominal (18-36 V allowed) DC power.
Power switch: The 2U MCC has a front panel power switch. The small MCC has no power switch: DC-in must be switched externally.
Power connector: The small MCE power connector is an Amphenol PTC-2A-14-5P. Positive voltage is on pins B and C. Return is on pins D and E.

Hardware: Block Diagram and Wiring

Block Diagram - small MCC [PDF]
Wiring Diagram - small MCC [PDF]

Block Diagram - rack-mount 2U MCC [PDF]

Wiring Diagram - rack-mount 2U MCC [PDF]

Operating system

The MCC comes pre-installed with all the software needed to run MCEs. It runs a stripped-down Ubuntu 12.04. The main operating system is installed on a compact-flash card on the computer and the Ubuntu root filesystem is, by default, mounted read-only. Data are written to one or two hard-drives also installed in the system (typically one hard drive is installed per fibre card). Both /home and the writable parts of /var are placed one of these hard drives; this should be kept in mind if swapping out the hardware. (See the mcc_init discussion below for ways to simplify this.)

Bookkeeping numbers and hostname

The MCC operating system is replicated from a master image of the system kept at UBC. The serial number of the particular image used for an install of the MCC operating system is available in /etc/mcc_serial. (The serial number corresponds only roughly to the date of the image's creation.)

When mastered, each MCC is assigned a installation number as well (for bookkeeping purposes). This number is available in /etc/mcc_id. The default hostname of an MCC is mcc## where ## is it's installation number. (For a convenient way to change the computer's hostname, see the mcc_init program below.)

Networking and security considerations

The normal interface to the MCC by other computers is over the 1000-baseT ethernet. By default it is set up to obtain its IP address via DHCP. Because the operating system is not typically kept up to date, the MCC network interface should be considered insecure: do not expose it to the wider Internet. By default the MCC exports its data hard drive(s) to the world via read-only NFS.

User accounts

There is only one user on the MCC, called 'mce', with full sudo rights. Password information is provided along with the MCC when shipped, or may be requested from UBC. It is rarely necessary to create more accounts.

Filesystem architecture

In addition to the standard Linux filesystem hierarchy, the following points should be noted.

Data drives are mounted on /data0 and /data1. Data collected from PCI fibre card #0 are written to /data0/mce. Data collected from PCI fibre card #1 are written to /data1/mce. These are also aliased as /data/mce0 and /data/mce1 respectively.
The parts of /var which must be writable are located under /data0/var.
/tmp is a symlink to /data0/var/tmp.
/home is a symlink to /data0/home.
MAS and MCE script are installed in /data0/mas. This is symlinked as /usr/mce for backwards compatibility.
The MAS configuration files are in /data0/mas/etc. This is symlinked as /etc/mce for backwards compatibility.
The 'mce' user has checked-out and configured/built copies of MAS/trunk and MCE_script in ~/src/. A "make install" of either of these can be done as 'mce' without needed to use sudo.
Because most of the root filesystem is mounted read-only, normal install of the MAS kernel driver and the MAS init and udev scripts aren't possible (even as root). The configuration of the default check-out copy of MAS disables the install of all of these. It is rarely necessary to update them, remount the drive if it does become necessary.

Configuration

For simplicity, we generally recommend using DHCP, perhaps keyed to your MCC's MAC address, for network configuration. A static IP can be assigned. The mcc_init script can help with network configuration.

MAS configuration

Although the default installation does install MAS and MCE script, the configuration of MAS hasn't been matched to any particular MCE configuration. Before using the MCC to acquire data it is generally necessary to reinstall the mce0.cfg and mce1.cfg with ones appropriate to your MCEs. To do this:

log into the MCC as 'mce'
change directory to ~/src/mas
edit config2/mce0.cin and config2/mce1.cin as appropriate for your systems.
run make from ~/src/mas to build the mce*.cfg files.
delete the existing /data0/mas/etc/mce[01].cfg and run make install to install the new ones.

MCE script configuration

MCE script expects to find configuration data in /data0/mas/config (aliased as /usr/mce/config). Either create this directory from the MCE script template directory or else check-out your experiment's files from the mce_config repository. You will also have to create the /data0/mce/array_id and /data1/mce/array_id files.

Using the MCC

The MCC has a very stripped down installation of Ubuntu. It is unsuited for data processing or analysis, so the data should be routinely copied off it. By default, the MCC computer provides anonymous read-only access to /data0 and /data1 by NFS exporting these partitions to the world. (Another good reason not to run the MCC without a firewall between it and the Internet.) Other data export schemes may be available.

If you've never used MAS on a computer with multiple PCI fibre cards, you may want to read Using Multicard MAS.

`mcc_init`

A utility script, mcc_init is provided to simplify some common tasks associated with MCC configuration. This script is located at /root/mcc_init. The script was written to automate initial installation and configuration of the MCC, but can be useful in certain situations on a running system as well.

Caveat utor: Given the right options, this script will do such potentially harmful things as mount and unmount volumes, repartition and format hard drives, all without user intervention. As a result, the script should be treated cautiously: cavalier use of mcc_init may result in data loss and/or render the MCC unbootable or the operating system unusable. (This is why we keep it out of the way in /root.)

The following restrictions are placed on running mcc_init:

It must be run as root (usually via sudo)
It must be run from /.

As a result, the script is usually invoked with something like:

$ cd / && sudo /root/mcc_init [options]

Running the script without any options will produce a usage summary. To further discourage inadvertant use of the script, unless installing a new system, the first option to the script must be reconfig. Other options indicating the desired operation may be given after this.

Available operations include:

duplicate: make /data1 a copy of /data0. (It's a partial copy: acquired data in /data0/mce is not copied.)
hostname=HOSTNAME: set the hostname to HOSTNAME;
installmas: install MAS and MCE script from the sources in ~/src. If they aren't found, they will be checked out, configured, and built, as necessary, beforehand;
net=dhcp: set up the network to use DHCP; this is the factory default, so this is only really necessary if a static IP address has been set up in the past;
net=IP_ADDR,NETMASK,GATEWAY,DNS_SERVER: set up a static IP network. IP_ADDR, NETMASK, GATEWAY, and DNS_SERVER are IPv4 dotted quads, separated by commas, with no intervening whitespace;
newkey: generate a new SSH server key;
remaster=0: recreate the default /data0. This requires network access to e-mode.phas.ubc.ca. This will delete everything on /data0 (Including mce's home directory, replacing it with the default home directory)!
remaster=1: recreate the default /data1. This will delete everything on /data1!
remaster=01: recreate both hard drives
noupdate: normally the script checks whether an updated version of itself is available before running. This operation with skip this step.

See the maintenance section below for examples of use.

Maintenance

Reconfiguring MAS

By default, MAS comes pre-configured and manually re-running the ./configure script shouldn't be required since the Makefile contains appropriate rules to do it automatically. In case you do want to explicitly re-run ./configure, the default MAS ./configure options are:

$ ./configure --prefix=/data/mas --enable-multicard --disable-driver --disable-sys-links --disable-init-script --with-etc-dir=/data/mas/etc \
              --with-data-root=/data0/mce,/data1/mce

Reinstalling the system (returning to the "factory default")

The MCC can be reinitialised to it's original configuration in the field with the following procedure:

Record your installation number (available in /etc/mcc_id).
Remove the compact flash card from the single board computer.
On another computer, with the CF card in a card reader, download a mcc_image file from http://e-mode.phas.ubc.ca/mce/mcc/ in a convenient location (not on the CF card). In general it's best to use the latest image. The image on the server may have been compressed (uncompressed it's ~8 GB); if so, decompress it before putting it on the CF card:

$ xz -d mcc_image_########.xz

Re-image the compact flash card:

# dd if=mcc_image_######## of=/dev/sdX

where ######## is the image serial number and sdX is the device node of the CF card. This will repartition the device, destroying any existing data on it, so be very certain you have the right node name.

reinstall the CF card and boot the MCC.

The first time you boot the MCC after doing this, you may see a number of warning or error messages; they shouldn't be fatal.

Log in as mce using the default password and use mcc_init to finish the installation:

$ cd / && sudo /root/mcc_init config id=N

where N is the installation number you recorded at the beginning. Configuration requires network access to UBC; if the MCC requires a static IP address instead of the default DHCP, you must set that up as well at this time by appending the net=... option to the script. The hostname=... can also be used here if you don't want the default mce## hostname.

Optionally record the numbers mcc_init spits out at the end. (Notably: the SSH fingerprint will have been regenerated).
Reboot the computer after mcc_init completes

You should now be back to the default installation.

Replacing /data0 without preserving data

Because parts of the operating system, including /home are on /data0, replacing it may semi-break the system. If you don't care about the contents of the old hard drive, simply replace the hardware and boot the MCC.

The first time you boot the MCC after doing this, you may see a number of warning or error messages; they shouldn't be fatal.

Log in as mce and use mcc_init to remaster the new hard drives:

$ cd / && sudo /root/mcc_init reconfig remaster=0

Reboot the computer after mcc_init completes.
Reconfigure MAS and MCE script as described above.

Replacing /data0 with partial data preservation

It is possible to replace the hard drives in the MCC while preserving the current contents of /home and /usr/mce. It's a two step process:

make a copy of /data0 on a new hard drive
install the new hard drives in their correct locations

NB: because there may be bugs, it's a good idea to back up your data regardless. This procedure doesn't preserve anything in /data0/mce.

Remove the old /data1 and install the hard drive you want to use as the new /data0 in it's place and boot the computer.
Log in as mce and use mcc_init to duplicate /data0 on /data1:

$ cd / && sudo /root/mcc_init reconfig duplicate

Turn off the MCC again; remove the old /data0; move the new /data0 into the right place and reinstall /data1. Boot the computer again.
Recreate /data0/mce/array_id.

Things should be back to normal.

Replacing /data1

Because /data1 contains no special files, installing a new hard drive for data one should be straightforward. After installing the new hard drive, just use mcc_init to recreate the filesystem skeleton:

$ cd / && sudo /root/mcc_init reconfig remaster=1

and then recreate /data1/mce/array_id.