900731 Data Handling for Eye-in-the-Sea on MARS : EITS workflow scripts and cron job installation This page last changed on May 10, 2010 by dcline. ## Important facts The EITS workflow must be executed by a user with a local account that mirrors a valid user account in the MBARI network. E.g. if your user name is erika and password is foo, use these same user/password combinations for your local account on EITS. The workflow that processes the EITS data is created by a series of bash and perl scripts. This workflow is then executed by the Condor software on the Condor pool and setup for the EITS experiment. The following describes how to install the workflow scripts as the user erika on the computer EITS. ## Store Condor credentials for the submitting user This presumes the user erika will be processing jobs and has a valid local account on EITS. A small Condor utility must be run to store the user credentials for the submitting user. This is used to authenticate who is submitting the job. Do this with the following in a cygwin shell as user Administrator:
    condor_store_cred add
You will be prompted for a user password. Use the same password you use for login. ## Problem: condor_ store_cred is failing, and I'm sure I'm typing my password correctly. Frequently, the error is a result of PERMISSION DENIED errors, so make sure you are running this as user Administrator:
    condor_store_cred add -u erika@EITS
3. If that doesn't work, first, make sure the condor_ schedd is running. You can also restart the condor service by navigating to the services in the control panel. Next, check the SchedLog. It will contain more detailed information about the failure. ## Check-out the workflow scripts Work flow scripts consist of both AVED-specific scripts, and EITS workflow scripts. AVED scripts are checked out from the CVS respository Moonjelly and EITS workflow scripts are checked out from the Subversion repository Kahuna. These scripts are used to process the EITS data through AVED. Login as the user who will be submitting jobs for processing. This presumes the user erika will be processing jobs and this user must have valid accounts on the servers EITS, Moonjelly and Kahuna. Launch cygwin by double-clicking on the desktop icon Check-out aved scripts from moonjelly CVS repository.
       export CVSROOT=:pserver:erika@moonjelly:/home/cvs
       cvs login
       cvs co aved/scripts aved/beoscripts
4. Check-out EITS workflow scripts from subversion
       export SVN_SSH='ssh -l erika'
       svn co svn+ssh://erika@kahuna.shore.mbari.org/svn/repos/eits/trunk/workflow
## Setup the cron job Now that the service is started, switch to the user who will be submitting jobs for processing, and add the following to their crontab file. This presumes the user erika will be processing jobs. First, launch the crontab editor.
    crontab -e
Next, add the following to the crontab to setup the environment variables and runEITS script to run every 5 minutes
    MAILTO="erika@mbari.org"
    USER=erika
    PATH=/cygdrive/c/condor/bin:/home/erika/workflow:/home/erika/aved/scripts:
    /home/erika/beoscripts:/condor/bin:/usr/bin:/usr/sbin

    # This sets up the command options to pass to the AVED processing binary called mbarivision
    # This is a global environment variable that will be applied to processing ALL clips
    MBARIVISION_OPTIONS='--mbari-save-non-interesting-events --mbari-tracking-mode=BoundingBox
    --mbari-cache-size=15 --mbari-min-event-area=50 --mbari-max-event-area=10000
    --mbari-mask-path=videomask.jpg'

    Every 5 minutes run the EITS script runEITS <directory to place processed results>
    <directory to copy raw data to>

    0-59/5 * * * * /home/erika/workflow/runEITS /cygdrive/e/EITSprocessed /cygdrive/e/EITSraw > /
    var/log/runEITS
Attachments: cygwin.png (image/png)

Document generated by Confluence on Feb 03, 2026 14:51