2. 900260 Automated Visual Event Detection and Classification : AVEDac Beowulf Cluster Installation Notes This page last changed on Jun 09, 2010 by dcline. # Configuration Your Beowulf cluster should have a NFS shared /home directory which is typically the way your cluster is configured it already configured. You will need at least 7 nodes to run the parallel version of the detection and tracking software called pmbarivision. There are also specific firewall and network settings required to run the pmbarivision through MBARI scripts noted in the instructions below. ## Install Cluster Command Tool (optional) If your cluster is configured with the Using the Open Source Cluster Application Resources (OSCAR), you already have this tool and you may skip this step. As the root user, download and install Cluster Command Tool version 3.1 http://www.csm.ornl.gov/torc/C3/ (this also must be installed on all nodes in the cluster. see above site for more information). If nodes are removed or added to the cluster, you must update this list and restart the mpd service. This utility must be installed first to give you the ability to push files and execute commands easily on your worker nodes. Download and install to /opt/c3-3. Add a file /etc/c3.conf, e.g. for an 8-node cluster with master node named beowulffish /etc/c3.conf
       cluster oscar_cluster {
               beowulffish
               dead remove_line_for_0-indexing
               node1.private.net
               node2.private.net
               node3.private.net
               node4.private.net
               node5.private.net
               node6.private.net
               node7.private.net
               node8.private.net
       }
1. ## Install MPI libraries The Message Passing MPI libraries are required to build pmbarivision. If you already have the MPI development libraries installed, then all that is required is to configure your mpd.hosts file (step 2). As root user, download and install MPI libraries to /opt/mpich-2.1.1p1
       tar -zxvf mpich2-1.1.1p1.tar.gz
       cd mpich2-1.1.1p1
       ./configure --prefix=/opt/mpich-2.1.1p1
       make
       cd src/mpe2; make
       cd ..
       make install
2. Create and add the following to /opt/mpich2-1.1p1/etc/mpd.hosts
       touch /opt/mpich2-1.1p1/etc/mpd.hosts
       chmod a+r /opt/mpich2-1.1p1/etc/mpd.hosts
   /opt/mpich2-1.1p1/etc/mpd.hosts
       master.private.net
       node1.private.net
       node2.private.net
       node3.private.net
       node4.private.net
       node5.private.net
       node6.private.net
       node7.private.net
       node8.private.net
3. Push the libraries to the other nodes:
       cpush /opt/mpich-2.1.1p1
## Add user aved It is recommended to install and run the pmbarivision binary as a user aved that is visible across all nodes. To support this, create a user aved for installing and running pmbarivision. As the root user:
    groupadd -g 300 aved
    useradd -c "AVED" -g aved -u 300 aved
The aved user and group id must be synchronized across all nodes and machines in the Beowulf cluster. Once the aved user is created, syncing up the user is typically done automatically by the OSCAR Password Installer and User Management (OPIUM) software, but you can also do this manually with:
    /opt/opium/bin/sync_users -f
## Setup the user profile Add to aved.sh/csh in the /etc/profile.d/ your MPDIR and SCRATCH_DIR settings. MPDIR points to the root of your mpich installation. SCRATCH_DIR is an NFS mounted directory that is shared between nodes where video will be processed. SCRATCH_DIR can be a separate disk, a separate partition, or a NFS shared directory - it just represents a tempporary "scratch" location that is shared across nodes where video will be temporarily placed for processing. SCRATCH_DIR must have read and write access for all users. /etc/profile.d/aved.sh
    export MPDIR=/opt/mpich2-1.1.1p1
    export PATH=$MPDIR/bin:$PATH
    export SCRATCH_DIR=/mnt/scratch
/etc/profile.d/aved.csh
    set MPDIR = /opt/mpich2-1.1p1
    set PATH = ($MPDIR/bin $PATH )
    set SCRATCH_DIR = /mnt/scratch
When done push these to the other nodes:
    cpush /etc/profile.d/aved.* /etc/profile.d/
## Test password-less SSH across all nodes Password-less ssh login is required to run the AVED MPI jobs. Your machine may already be configured for this. To test this, switch to the user aved and try ssh into the first node1. You should not be prompted for a password
    root@beowulffish ~]# su aved
    [aved@beowulffish ~]$ ssh n01
    Last login: Thu Apr 15 10:54:27 2010 from master.private.net
    [aved@node1 ~]$
If your machine is not configured for password-less login, there is a wealth of information on how to set this up on the web. However, this is typically configured by default in the OSCAR tool suite in /etc/profile.d/ssh-oscar.ssh/csh. ## Copy ffmpeg library dependencies to all nodes These dependencies are required in the pmbarivision code. This is the only dependency needed outside those installed in the /home/aved NFS shared directory. Push dependencies to the client nodes and refresh the dynamic linker:
    cpush /usr/lib/libavcodec.so.51
    cpush /usr/lib/libavcodec.so.52
    cpush /usr/lib/libavcodec.so.51.40.4
    cpush /usr/lib/libavformat.so.51.12.1
    cpush /usr/lib/libavformat.so.51
    cpush /usr/lib/libavformat.so.51.12.1
    cpush /usr/lib/libavutil.so
    cpush /usr/lib/libavutil.so.49
    cpush /usr/lib/libavutil.so.49.4.0
    cpush /usr/lib/libSDL-1.2.so.0
    cpush /usr/lib/libSDL-1.2.so.0.7.0
    cexec 'ldconfig'

Document generated by Confluence on Feb 03, 2026 14:12