McLane Moored Profiler Data Reduction and Processing
Procedures
John Toole Woods Hole Oceanographic Institution November 2001, updated
February 2002
Introduction
These pages describe the processing system designed at
the Woods Hole Oceanographic Institution to operate on data from McLane
Moored Profilers. These Profiler data files are processed (together
with calibration information that characterize the raw sensor data) into
uniformly binned pressure series of ocean temperature, salinity, and velocity
using a series of Matlab scripts. The software for unpacking the
data is provided by McLean. This documentation pertains to the non-gui
version of the software which performs conversions and calibrations to
obtain a pressure gridded profile of the data from the CTD and ACM.
The documentation of the GUI interface to the processing is available here.
-
The Instrument:
As currently configured, the McLane Moored Profiler (MMP) acquires
three types of time-series data during each vertical profile:
-
Engineering - time, motor current, battery voltage, pressure
-
CTD - conductivity, temperature, pressure
-
ACM - acoustic current meter - four channels of raw velocity
data, three channels of compass readings and two axes of tilt
-
Raw data:
For each vertical profile, one binary file of each type is written
to the flash memory card. The binary data on the MMP's flash card are accessed
on a PC via the MRL software routine unpk2_07. Choose the option
for producing "machine readable" output; the product will be a set of three
ascii data files (C#######.TXT A#######.TXT, E#######.TXT, where #######==profile
number, padded with leading 0's) for each profile.
Quick Look at raw data
For looking quickly at any
of these ascii data files within the Matlab environment, use the routines
MMP_read_eng.m, MMP_read_ctd.m, MMP_read_ACM.m. These are not included
in the GUI tool, so must be run from the command line.
MMP_read_eng.m returns
the following variables as Matlab vectors:
esnum: index of the engineering scans
epres: engineering pressure in decibars
ecurr: motor current in milli-Amps
evolt: battery Voltage
edpdt: profile speed dbars/s from the engineering data
engtime: time of each engineering scan coded with matlab function datenum.m
MMP_read_ctd.m returns
the following variables as Matlab vectors:
csnum: index of the CTD scans
cpres: pressure in decibars
ctemp: temperature in nominal degrees C
ccond: conductivity in nominal mmho
MMP_read_acm.m
returns the following variables as Matlab vectors:
asnum: index of the ACM scans
Vab Vcd Vef Vgh: raw path velocities in nominal cm/s
aHx aHy aHz: horizontal (x,y) and vertical (z) normalized compass readings
aTx aTy: tilt in fore-aft and athwartships degrees
Data processing:
Once the data is unpacked, three steps are typically
completed. More information on each is provided in the sections below.
-
Merge the three raw data files and stores all
the raw variables in one .mat file
-
Create and examine the summary
-
Average the ACM and CTD data into a user specified
pressure bins
Merge profile files
To facilitate processing and archiving the raw MMP data, we merge the Engineering,
CTD and ACM data from each profile into one, Matlab-format data file (e.g.
one file per profile) using MMP_asc2mat.m.
Required input includes the
directory names for the source data, the name of the directory where you
want the combined output files to go, and the range of profiles you wish
to process.
The output data files are called raw####.mat.
Each file contains the following variables:
psdate: profile start date
pedate: profile end date
pstart: profile start time
pstop: profile end time
esnum: index of the engineering scans
epres: engineering pressure in decibars
ecurr: motor current in milli-Amps
evolt: battery Voltage
edpdt: profile speed dbars/s from the engineering data
engtime: time of each engineering scan coded with matlab function
datenum.m
csnum: index of the CTD scans
ccond: conductivity in nominal mmho
ctemp: temperature in nominal degrees C
cpres: pressure in decibars
asnum: index of the ACM scans
Vab: raw path velocities in nominal cm/s
Vcd: raw path velocities in nominal cm/s
Vef: raw path velocities in nominal cm/s
Vgh: raw path velocities in nominal cm/s
aTx: tilt in fore-aft and athwartships degrees
aTy: tilt in fore-aft and athwartships degrees
aHx: horizontal (x) normalized compass readings
aHy: horizontal (y) normalized compass readings
aHz: vertical normalized compass readings
Overview of a deployment
The Matlab function MMP_raw_summary.m gives the user a
way to look at data from an entire deployment and quantify such things
as minimum and maximum pressure sampled on each profile and the profile-averaged
speed, motor current and battery voltage. The routine works with
the combined matlab-format raw data files created by MMP_asc2mat
(raw####.mat). User is asked to input the name of the directory
where these data files are located, and the range of profiles to process.
The user must also enter a minimum and maximum pressure within which various
vertically-averaged parameters will be evaluated.
Hint:
picking these limits
somewhat smaller than the actual range sampled by the Profiler will exclude
the start up and ending transients from the averages.
The following vectors are written to a user-specified
summary output file:
files:
vector of profile numbers
starttime:
profile start time encoded with datenum.m
endtime:
profile ending time encoded with datenum.m
neng:
number of engineering scans in the profile
nctd:
number of CTD scans in each profile
nacm:
number of ACM scans in each profile
cpmin:
minimum Pressure recorded by the CTD on each profile
cpmax:
maximum Pressure recorded by the CTD on each profile
epmin:
minimum engineering pressure recorded on each profile
epmax:
maximum engineering pressure recorded on each profile
espeed:
average travel speed for each profile dbars/second
avevolt:
average battery voltage during each profile
avecur:
average motor current during each profile
Pressure gridding
The Matlab routine MMP_pgrid.m and its associated subroutines are
designed to reduce the profiles of raw temperature, conductivity and pressure
data from the FSI EMCTD and velocity path data and compass measurements
from the FSI acoustic current meter and create pressure-bin-averaged
data in scientific units. This step in the processing assumes that
the previous step of loading the ascii engineering, CTD and ACM data files
into matlab format (one file per profile) has been completed. (These
must be called raw####.dat in an arbitrary, user-specified
directory.) Input to the pressure gridding routine include paths to the
input and output data directories, various sensor calibration files and
edit parameters, and the user specified pressure bounds and grid increment
on which the data will be averaged. The user also specifies the profiles
to be processed on each run of the program. Gridded output
files (one per profile) are called
grd####.mat.
In addition, a documentation file is created (one doc file per run
of MMP_pgrid) called mmp_pgrid_M_N.doc where
M is the first profile processed in the run and N is the last.
A shell script, MMP_pgrid_script.sh was created to simplify entering
the inputs for MMP_pgrid_mfile.m. These modifications to John's
pgrid code were made by Maggie Cook (also of WHOI) to simplify the
processingbefore the GUI tool was developed, and to include more information
in the mmp_pgrid_M_N.doc file. The script will not
run on PC's but it can be used to illustrate the steps it executes.
The computations are the same as performed by MMP_pgrid, as described
below.
Conceptually, the data from the CTD and ACM are checked to insure
that all variables are with sensible range (specified by the user)
and pass a first difference check. Bad raw data are interpolated.
Time is logged in the engineering file along with a pressure reading.
These data are used to interpolate a time estimate for each CTD scan.
Apart from the logged times when the MMP starts and stops the external
sensors, there are no cross-referenced time marks between the CTD and ACM
apart from a manufacturer-stated common sample rate. To align
the CTD and ACM data streams, the program automatically locates the
starting and ending transients at the top and bottom of the profile
interval and assumes constant (but not necessarily the same) sample rate
for each sensor. This is sufficient to define the time value for
each ACM scan, allowing common binning of all data against pressure.
Once the alignment is set and the raw data have been edited and calibrated,
the bin-averaging against pressure is carried out and the resultant data
are written out to the user specified directory as grd####.mat.
Input:
The input data for this step of the processing are in the raw####.mat
files. Both the CTD and the ACM require input of some edit parameters,
specifically the minimum and maximum allowable values for each parameter.
Data outside these intervals will be assumed to be bad and interpolated
over. You are also required to have a matlab format file holding
sensor calibration information.
-
Calibration files:
CTD: the calibration file requires the following vectors:
p_coefs, t_coefs and c_coefs, where each are the coefficients
of polynomials that convert raw sensor values to calibrated values with
the matlab utility polyval:
cpres=polyval(p_coefs,cpres); ctemp=polyval(t_coefs,cpres); ccond=polyval(c_coefs,cpres);
In addition, the file must hold the variable "tlag" which is the
lag in scans that the thermometer lags the conductivity. The CTD
data are lag corrected by recursively filtering the C and P data to match
the response characteristics of the T data.
[ccond]=MMP_recur_filt(ccon,tlag);
[cpres]=MMP_recur_filt(cpres,tlag);
-
ACM: the calibration file requires the following variables:
Hx_bias, Hx_range, Hy_bias, Hy_range, dirsign, compass_bias, Vab_bias,
Vcd_bias, Vef_bias, Vgh_bias, velocity_scale
The magnetometer data are scaled to the unit circle as follows:
[(aHx-Hx_bias)/Hx_range]^2
+ [(aHy-Hy_bias)/Hy_range]^2 = 1
In situ data are used to work out the magnetometer bias and range terms
that will scale the compass data to the unit circle. The true compass
direction towards which the ACM sensor sting points is derived as
follows:
mvpdir=dir_sign*atan2(aHy,aHx)+compass_bias+mag_dev
where mag_dev is the earth's magnetic deviation . Laboratory spin
test data are used to derive dirsign and compass_bias.
You may create the calibration files manually or use the scripts
create_ctd_calfile.m
and
create_acm_calfile.m
to guide you through the data entry.
Each gridded file (grd####.mat)
(one per profile) contains the following variables:
startdaytime:
start day and time of the profile (encoded with datenum.m)
stopdaytime:
stop day and time of the profile (encoded with datenum.m)
pgrid:
the center values of the pressure grid used in the bin-average
ctimave:
day and time of values averaged in each pgrid bin (datenum.m)
pave:
average of the pressure values in each pgrid bin dbars
tave:
average of the temperature values in each pgrid bin degC
s_ave:
salinity computed pave, tave and cave pss
thetave:
potential temperature computed from pave, tave, and s_ave degC
sigthave:
potential density computed from pave, tave and s_ave kg/m^3
cave:
average scaled conductivity in each pgrid bin mmho
uave:
average east velocity in each pgrid bin cm/s
vave:
average north velocity in each pgrid bin cm/s
wave:
average relative vertical velocity in each pgrid bin cm/s
dpdtave:
average time rate of change of pressure dbars/s
cscan1
cscan2: indices of the CTD data averaged in each pgrid bin
ascan1
ascan2: indices of the ACM data averaged in each pgrid bin