Author: Ellyn Montgomery

### McLane Moored Profiler Data Reduction and Processing Procedures

##### John Toole Woods Hole Oceanographic Institution November 2001, updated February 2002

### Introduction

These pages describe the processing system designed at the Woods Hole Oceanographic Institution to operate on data from McLane Moored Profilers. These Profiler data files are processed (together with calibration information that characterize the raw sensor data) into uniformly binned pressure series of ocean temperature, salinity, and velocity using a series of Matlab scripts. The software for unpacking the data is provided by McLean. This documentation pertains to the non-gui version of the software which performs conversions and calibrations to obtain a pressure gridded profile of the data from the CTD and ACM. The documentation of the GUI interface to the processing is available here.

- The Instrument:
As currently configured, the McLane Moored Profiler (MMP) acquires three types of time-series data during each vertical profile:

- Engineering - time, motor current, battery voltage, pressure
- CTD - conductivity, temperature, pressure
- ACM - acoustic current meter - four channels of raw velocity data, three channels of compass readings and two axes of tilt

- Raw data:
For each vertical profile, one binary file of each type is written to the flash memory card. The binary data on the MMP's flash card are accessed on a PC via the MRL software routine unpk2_07. Choose the option for producing "machine readable" output; the product will be a set of three ascii data files (C#######.TXT A#######.TXT, E#######.TXT, where #######==profile number, padded with leading 0's) for each profile.

Quick Look at raw data
For looking quickly at any of these ascii data files within the Matlab environment, use the routines MMP_read_eng.m, MMP_read_ctd.m, MMP_read_ACM.m. These are not included in the GUI tool, so must be run from the command line.

MMP_read_eng.m returns the following variables as Matlab vectors:
esnum: index of the engineering scans
epres: engineering pressure in decibars
ecurr: motor current in milli-Amps
evolt: battery Voltage
edpdt: profile speed dbars/s from the engineering data
engtime: time of each engineering scan coded with matlab function datenum.m

MMP_read_ctd.m returns the following variables as Matlab vectors:
csnum: index of the CTD scans
cpres: pressure in decibars
ctemp: temperature in nominal degrees C
ccond: conductivity in nominal mmho

MMP_read_acm.m returns the following variables as Matlab vectors:
asnum: index of the ACM scans
Vab Vcd Vef Vgh: raw path velocities in nominal cm/s
aHx aHy aHz: horizontal (x,y) and vertical (z) normalized compass readings
aTx aTy: tilt in fore-aft and athwartships degrees

### Data processing:

Once the data is unpacked, three steps are typically completed. More information on each is provided in the sections below.

- Merge the three raw data files and stores all the raw variables in one .mat file
- Create and examine the summary
- Average the ACM and CTD data into a user specified pressure bins

###

Merge profile files

To facilitate processing and archiving the raw MMP data, we merge the Engineering, CTD and ACM data from each profile into one, Matlab-format data file (e.g. one file per profile) using MMP_asc2mat.m.

Required input includes the directory names for the source data, the name of the directory where you want the combined output files to go, and the range of profiles you wish to process.

The output data files are called raw####.mat. Each file contains the following variables:

> psdate: profile start date
> pedate: profile end date
> pstart: profile start time
> pstop: profile end time
> esnum: index of the engineering scans
> epres: engineering pressure in decibars
> ecurr: motor current in milli-Amps
> evolt: battery Voltage
> edpdt: profile speed dbars/s from the engineering data
> engtime: time of each engineering scan coded with matlab function datenum.m
> csnum: index of the CTD scans
> ccond: conductivity in nominal mmho
> ctemp: temperature in nominal degrees C
> cpres: pressure in decibars
> asnum: index of the ACM scans
> Vab: raw path velocities in nominal cm/s
> Vcd: raw path velocities in nominal cm/s
> Vef: raw path velocities in nominal cm/s
> Vgh: raw path velocities in nominal cm/s
> aTx: tilt in fore-aft and athwartships degrees
> aTy: tilt in fore-aft and athwartships degrees
> aHx: horizontal (x) normalized compass readings
> aHy: horizontal (y) normalized compass readings
> aHz: vertical normalized compass readings

Overview of a deployment

The Matlab function MMP_raw_summary.m gives the user a way to look at data from an entire deployment and quantify such things as minimum and maximum pressure sampled on each profile and the profile-averaged speed, motor current and battery voltage. The routine works with the combined matlab-format raw data files created by MMP_asc2mat (raw####.mat). User is asked to input the name of the directory where these data files are located, and the range of profiles to process. The user must also enter a minimum and maximum pressure within which various vertically-averaged parameters will be evaluated.

Hint:
picking these limits somewhat smaller than the actual range sampled by the Profiler will exclude the start up and ending transients from the averages.

The following vectors are written to a user-specified summary output file:
files: vector of profile numbers
starttime: profile start time encoded with datenum.m
endtime: profile ending time encoded with datenum.m
neng: number of engineering scans in the profile
nctd: number of CTD scans in each profile
nacm: number of ACM scans in each profile
cpmin: minimum Pressure recorded by the CTD on each profile
cpmax: maximum Pressure recorded by the CTD on each profile
epmin: minimum engineering pressure recorded on each profile
epmax: maximum engineering pressure recorded on each profile
espeed: average travel speed for each profile dbars/second
avevolt: average battery voltage during each profile
avecur: average motor current during each profile

###

Pressure gridding

The Matlab routine MMP_pgrid.m and its associated subroutines are designed to reduce the profiles of raw temperature, conductivity and pressure data from the FSI EMCTD and velocity path data and compass measurements from the FSI acoustic current meter and create pressure-bin-averaged data in scientific units. This step in the processing assumes that the previous step of loading the ascii engineering, CTD and ACM data files into matlab format (one file per profile) has been completed. (These must be called raw####.dat in an arbitrary, user-specified directory.) Input to the pressure gridding routine include paths to the input and output data directories, various sensor calibration files and edit parameters, and the user specified pressure bounds and grid increment on which the data will be averaged. The user also specifies the profiles to be processed on each run of the program. Gridded output files (one per profile) are called grd####.mat. In addition, a documentation file is created (one doc file per run of MMP_pgrid) called mmp_pgrid_M_N.doc where M is the first profile processed in the run and N is the last.

A shell script, MMP_pgrid_script.sh was created to simplify entering the inputs for MMP_pgrid_mfile.m. These modifications to John's pgrid code were made by Maggie Cook (also of WHOI) to simplify the processingbefore the GUI tool was developed, and to include more information in the mmp_pgrid_M_N.doc file. The script will not run on PC's but it can be used to illustrate the steps it executes. The computations are the same as performed by MMP_pgrid, as described below.

Conceptually, the data from the CTD and ACM are checked to insure that all variables are with sensible range (specified by the user) and pass a first difference check. Bad raw data are interpolated. Time is logged in the engineering file along with a pressure reading. These data are used to interpolate a time estimate for each CTD scan. Apart from the logged times when the MMP starts and stops the external sensors, there are no cross-referenced time marks between the CTD and ACM apart from a manufacturer-stated common sample rate. To align the CTD and ACM data streams, the program automatically locates the starting and ending transients at the top and bottom of the profile interval and assumes constant (but not necessarily the same) sample rate for each sensor. This is sufficient to define the time value for each ACM scan, allowing common binning of all data against pressure. Once the alignment is set and the raw data have been edited and calibrated, the bin-averaging against pressure is carried out and the resultant data are written out to the user specified directory as grd####.mat.

Input:
The input data for this step of the processing are in the raw####.mat files. Both the CTD and the ACM require input of some edit parameters, specifically the minimum and maximum allowable values for each parameter. Data outside these intervals will be assumed to be bad and interpolated over. You are also required to have a matlab format file holding sensor calibration information.
- Calibration files:

CTD: the calibration file requires the following vectors:
p_coefs, t_coefs and c_coefs, where each are the coefficients of polynomials that convert raw sensor values to calibrated values with the matlab utility polyval:
cpres=polyval(p_coefs,cpres); ctemp=polyval(t_coefs,cpres); ccond=polyval(c_coefs,cpres);

In addition, the file must hold the variable "tlag" which is the lag in scans that the thermometer lags the conductivity. The CTD data are lag corrected by recursively filtering the C and P data to match the response characteristics of the T data.
[ccond]=MMP_recur_filt(ccon,tlag);
[cpres]=MMP_recur_filt(cpres,tlag);

- ACM: the calibration file requires the following variables:
Hx_bias, Hx_range, Hy_bias, Hy_range, dirsign, compass_bias, Vab_bias, Vcd_bias, Vef_bias, Vgh_bias, velocity_scale

The magnetometer data are scaled to the unit circle as follows:
[(aHx-Hx_bias)/Hx_range]^2 + [(aHy-Hy_bias)/Hy_range]^2 = 1

In situ data are used to work out the magnetometer bias and range terms that will scale the compass data to the unit circle. The true compass direction towards which the ACM sensor sting points is derived as follows:
mvpdir=dir_sign*atan2(aHy,aHx)+compass_bias+mag_dev

where mag_dev is the earth's magnetic deviation . Laboratory spin test data are used to derive dirsign and compass_bias.

You may create the calibration files manually or use the scripts create_ctd_calfile.m and create_acm_calfile.m to guide you through the data entry.

Output
Each gridded file (grd####.mat) (one per profile) contains the following variables:

startdaytime: start day and time of the profile (encoded with datenum.m)
stopdaytime: stop day and time of the profile (encoded with datenum.m)
pgrid: the center values of the pressure grid used in the bin-average
ctimave: day and time of values averaged in each pgrid bin (datenum.m)
pave: average of the pressure values in each pgrid bin dbars
tave: average of the temperature values in each pgrid bin degC
s_ave: salinity computed pave, tave and cave pss
thetave: potential temperature computed from pave, tave, and s_ave degC
sigthave: potential density computed from pave, tave and s_ave kg/m^3
cave: average scaled conductivity in each pgrid bin mmho
uave: average east velocity in each pgrid bin cm/s
vave: average north velocity in each pgrid bin cm/s
wave: average relative vertical velocity in each pgrid bin cm/s
dpdtave: average time rate of change of pressure dbars/s
cscan1 cscan2: indices of the CTD data averaged in each pgrid bin
ascan1 ascan2: indices of the ACM data averaged in each pgrid bin