Multicard MAS

Most of MAS is agnostic about the number of fibre cards in the system. By default, MAS only supports one fibre card. Support for multiple cards ("Multicard MAS") can be turned on, however, when building MAS. This page outlines specific procedures and caveats when using Multicard MAS.

Care has been taken to make Multicard MAS backwards compatible with the old, single card system, to permit use of legacy scripts and applications (albeit, perhaps restricted to one of the fibre cards in the system).

Building MAS

For generic build instructions, see: MAS OS setup

To enable Multicard MAS, pass --enable-multicard to configure before building MAS:

 ./configure --enable-multicard[=N]

where N is the maximum number of cards you want MAS to support. If omitted, N defaults to 2. Specifiying N<=1, is the same as not specifying this option at all (i.e.: multicard support is turned off). This results in both a multicard capable driver and MAS library/applications.

Because the subracks attached to each fibre card may be different, instead of a single mce.cfg file, Multicard MAS requires one mce.cfg file for each fibre card supported, called /etc/mce/mce0.cfg, /etc/mce/mce1.cfg, /etc/mce/mce2.cfg, &c. As a result, instead of making a mce.cin template file, you must make mce0.cin, mce1.cin, mce2.cin, &c.

Running make and make install should proceed as usual. (See MAS OS setup.)

Card numbering

The kernel driver assigns sequential physical numbers to cards in the order in which they're passed in by the kernel at boot time. Because there is no guarantee that this procedure results in the same physical card enumeration each time, MAS abstracts physical card numbers to logical card numbers, which are fixed to a given physical card. Although the cards are indistinguishable themselves, MAS uses the PCI slot address to break the degeneracy.

After the kernel boots, and the kernel driver has assigned physical cards, udev runs mas_mknodes for each card it finds, passing this script the PCI slot address of the card. mas_mknodes then consults /proc/mce_dsp to determine the card's physical number and a the file /etc/mce/mce_card_id (if present) to determine it logical card number. It then makes nodes /dev/mce_cmd<l>, /dev/mce_cmd<l>, /dev/mce_cmd<l> pointing to the appropriate physical card, where l is the logical card number. (See the udev ruleset in scripts/91-mas.rules and "mas_mknodes --help" for more details.)

Generating /etc/mce/mce_card_id

The /etc/mce/mce_card_id file is a simple text file with two columns, and one row for each fibre card supported. For a given card, the first column contains its PCI slot address, and the second column it's logical card number. The file may also contain comment lines whose first character is a hash mark (#). A typical file might look like:

# PCI_SLOT_ID   LOGICAL_CARD_NUM
0000:02:0c.0        0
0000:02:0d.0        1

The file can be created by hand, but it is typically made by running the mas_make_card_id script. This script examines the system as it is currently configured and creates a /etc/mce/mce_card_id which will result in the same configuration on subsequent boots. This script obtains logical card numbers by searching for /dev/mce_cmd# devices, and physical card numbers from /proc/mce_dsp. It ignores /dev/mce_cmd# devices which do not point to a valid physical card. It will also ignore physical cards which do not have a corresponding /dev/mce_cmd# (ie. no logical card number assigned), unless the '-a' option is passed to the script, in which case, it will automatically assign logical card numbers to unenumerated physical cards.

As a result, a /etc/mce/mce_card_id file can be created for a brand new system with no cards configured by running:

mas_make_card_id -a

If /etc/mce/mce_card_id doesn't exist, mas_mknodes simply uses physical card numbers for logical card numbers. If the file does exist, but the specified PCI slot address isn't in the file, mas_mknodes will fail.