August 3, 1999

MBARI Core Data Streams

Introduction

A large number of data streams are collected at MBARI in the course of the ongoing research and development activities. Some of these data streams result from divisional activities (e.g. for safety or ship operations issues) and are neither formally approved nor intended for archiving or dissemination. Some other data streams are acquired, processed, analyzed, and archived as part of projects that require yearly approval through the internal proposal process. A few data streams may be designated as "core", which means that support for the collection, processing, and archiving of the data is provided on an institutional basis rather than through individual projects or divisional budgets. In a general fashion, one may view core data streams as being so important to so many avenues of MBARI research that an institutional commitment of support makes sense and benefits MBARI as a whole.

In this document, the MBARI Data Users Committee (M-DUC) provides specific criteria for the identification and handling of core data streams.

Necessary Conditions for a Data Stream to be Considered "Core"

This table lists some necessary (but not sufficient) characteristics of "core" data streams.

1. The sensor(s) and data acquisition hardware and software must be mature. Mature sensors have the following characteristics:

· Reliable.

· Well calibrated.

· Stable configuration.

· Stable format.

· Stable and efficient data processing.

· Established archiving and dissemination scheme.

1. The data must be of sufficient interest to the general MBARI community. Any of the MBARI divisions may, through its director, designate a data stream as being of sufficient interest to the MBARI community.
2. The data stream must be:

· vital to ongoing research, OR

· judged important for the future (e.g. unique data streams)

1. It must be possible to identify someone among the end users of the data stream who will commit to regular data validation (a data stream "champion"). This task may be shared or rotating.
2. The data cannot be embargoed within the MBARI community.

Process by Which Data Streams are identified as "Core"

Management, science, and relevant support groups must all be involved in any effort to designate a particular data stream as "core". Taking the review process through the steps outlined below will ensure that the requisite broad involvement occurs.

1. MDUC will consider any request for a data stream to be included as core.
2. MDUC will consult with relevant users, technicians, and platform committees (e.g. ROVUC and MUC) regarding the candidate data stream.
3. One or more division directors must concur that the data stream satisfies criterion (2).
4. Following consultations, MDUC will review the data stream and decide whether to recommend core status.
5. If core status is to be recommended, MDUC will develop a data stream management plan in conjunction with the relevant support personnel, end users, and affected resource supervisors. This plan will contain the following elements:

· Data quality specifications.

· Sensor maintenance requirements.

· Data processing scheme.

· Data archiving and dissemination scheme.

· Quality assurance plan.

· Resource requirements.

1. RSC will review the data stream management plan.
2. Management will make the final decision.

Long Term Management of "Core" Data Streams

Core data streams must have high priority and must be reviewed on a regular basis in order to insure consistent data quality. The following elements should allow MBARI to maintain core data quality and to identify and respond to problems in a timely fashion.

1. Core data streams are understood to have a high priority for resources.
2. None of the hardware, software, or personnel components of the acquisition, processing, archiving, and dissemination of core data streams may be changed without prior approval of MDUC and management.
3. Each designated data champion will frequently check the core data stream to assure quality and identify problems. When problems are found, the "champion" will alert the relevant personnel so that the problems are resolved in a timely fashion.
4. MDUC will check the status of each core data stream regularly (at least quarterly).
5. MDUC will update the core data stream management plans on a yearly basis at proposal time.

Preliminary Candidates for "Core" Data Streams

MDUC has identified six data streams that meet the criteria for core status. These are:

1. ROV video annotation (outline mode)
2. Ship navigation
3. ROV navigation
4. Ship CTD
5. ROV CTD
6. Sample archive and database

MDUC will now begin working on developing data stream management plans for these data streams. These plans may, if approved by RSC and management, be implemented for year 2000.