(last updated 2024-12-04)

Design Goals

1. Preserve original "full"-resolution data from the instruments - especially primary navigation.

2. Organize files in a fleet-standard directory structure that can be routinely "broken out" shoreside by regular expression matching against vessel instrument profile.

   • Segregate the "routine underway" data (resident digital instrument systems) from everything else (science party files, visiting national facility data, etc).
   • Segregate data from documentation.
   • Segregate the original instrument data from secondary products and/or processed data.
   • Segregate the standard, underway data and documentation collected by the vessel operator from that generated by the science party.

3. Preserve all of the standard, underway data and documentation collected by the vessel operator organized as it was provided to the science party.

Standardize the Directory Structure

• R2R uses regexes to map files to devices.
• Consolidating and standardizing data directories and files within distros makes it is easier for R2R to maintain up-to-date mapping and ensure that it identifies all relevant data files.
• Acquisition systems are an effective means of standardizing the directory structure for serial devices while performing other useful bookkeeping procedures like enforcing valid filenames.
• A standardized directory structure also helps scientists who work across multiple vessels.

Recommended Directory Tree

• CRUISEID /
  • docs / - general cruise docs eg. personnel list, operations report, vessel diagrams
  • SYSTEM / - where SYSTEM is either the vessel's native serial acquisition system ( OpenRVDAS, OpenVDM, scs, lds, suds, vids, etc ) or a standalone system ( adcp, ctd, echo, multibeam, xbt, etc)
    • docs / - manuals, photos, calibration sheets, etc. specific to this instrument system
    • raw /
      • DEVICETYPE [_[ MAKE ][ MODEL ]][_LOCATION ][_MNEMONIC ] - DATE T TIME Z

        where MNEMONIC allows finer-grained separation of devce interfaces

        (could be serial number, datatype ("centerbeam"), or just a number 1, 2, 3)

    • proc / - if processing was done at sea
    • r2r/
      • elog / - produced by (or formatted for) the R2R Event Logger
    • science / - "dropbox" files created by the science party including photos, blogs, etc. This directory is provided to separate science party information from that of the vessel operator and should not be included in the cruise distribution provided to R2R.

Recommended Practices

• Cruise IDs are unique within R2R. The exact naming style is left to each operator, but IDs should begin with the vessel's unique prefix.
• Directory and file names use standard vocabulary of device type, make, and model. Names are lowercase, with no spaces or non-alphanumeric characters except nonconsecutive underscores and periods. R2R will never modify any original file received from an operator.
• Some operators support simultaneous acquisition of both original/full-resolution and subsampled/multiplexed data. These should be clearly separated in the directory structure.
• Write all data from the same "talker" (i.e. multiple NMEA sentences from the same GNSS receiver port) to the same file. Break data files daily where possible.
• Some vessels log both original/full-resolution and multiplexed/subsampled data files. These should be clearly separated in the directory structure.
• Individual files are not compressed. The entire cruise distro may be packaged with GNU tar/gzip.
• Operator delivers a checksum manifest with each distro so that R2R can validate shoreside, using the md5deep program: md5deep -c -r -l -o f -t -z cruise > cruise.md5deep
• Cruise distros are posted for R2R to download via network transfer (rsync). These distros should include only standard, underway data and documentation collected by the vessel operator. Science party data should not be included.