WaveGlider ADCP Processing

Python notebook for processing raw Teledyne RDI WorkHorse ADCP data collected from a Liquid Robotics Wave Glider into analysis-ready NetCDF files.

---

Background

The Teledyne RDI WorkHorse ADCP records data in a proprietary binary .ENS format. The standard Python library for reading this format (pycurrents) misaligns on .ENS files produced by Wave Gliders, resulting in garbage timestamps, incorrect array dimensions, and corrupted scalar fields.

This notebook bypasses pycurrents entirely and reads the binary file at the byte level. Real ensemble boundaries are located using the 0x7f7f header marker and filtered by consistent byte spacing. All fields are extracted at their known byte offsets within the RDI binary structure.

---

What it does

• Reads a raw .ENS binary file into memory
• Locates real ensemble boundaries (filtering 0x7f7f false positives by spacing)
• Reads instrument configuration from the FixedLeader block (NCells, cell size, bin geometry)
• Extracts and validates timestamps from the VariableLeader block
• Identifies deployment segments separated by large time gaps
• Detects mid-deployment coordinate system changes (beam ↔ ENU) from the EX field
• Extracts per-ensemble data: velocity, correlation, echo intensity, percent good
• Decodes GPS position from the Wave Glider Navigation block (BAM4 format)
• Extracts platform orientation: heading, pitch, roll
• Extracts hydrographic scalars: temperature, pressure, speed of sound
• Saves one NetCDF file per coordinate system segment with full CF-1.8 metadata

---

Input / Output

Input:

• Raw .ENS binary file from the ADCP recorder (tested on Teledyne RDI WorkHorse 300 kHz)
• ADCP command file (.txt) — optional, for reference only (see docs/config_file_guide.md)

Output:

• waveglider_adcp_beam.nc — beam coordinate segment (raw radial velocities per beam)
• waveglider_adcp_ENU.nc — ENU coordinate segment (East, North, Up), if present

Dataset variables

Variable	Dimensions	Units	Description
velocity	time, cell, beam	mm/s	Beam or ENU velocities
correlation	time, cell, beam	counts	Signal correlation (0–255, <64 = suspect)
echo_intensity	time, cell, beam	counts	Acoustic backscatter (0.45 dB/count)
percent_good	time, cell, pg_type	%	Data quality flags (use pg_type=4)
latitude	time	degrees_north	GPS latitude (BAM4 decoded)
longitude	time	degrees_east	GPS longitude (BAM4 decoded)
heading	time	degrees	Magnetic heading
pitch	time	degrees	Platform pitch
roll	time	degrees	Platform roll
temperature	time	°C	Transducer face temperature (proxy for SST)
pressure	time	dbar	Instrument pressure
speed_of_sound	time	m/s	Speed of sound used by ADCP

---

Quick start

1. Clone the repository

git clone https://github.com/jedholm/waveglider-adcp.git
cd waveglider-adcp

2. Create the conda environment

conda env create -f environment.yml
conda activate waveglider-adcp

3. Open the notebook

jupyter lab notebooks/waveglider_adcp_processing.ipynb

4. Set your file paths

Edit the USER CONFIG cell at the top of the notebook:

ENS_FILE            = "YOUR-DATA-FILE.ens"  # path to input .ens file
OUTPUT_NC           = "YOUR-OUTPUT-FILE.nc"  # path to output .nc file
DEPLOYMENT_AREA     = "YOUR-DEPLOYMENT-AREA"   # for metadata only
SAMPLING_INTERVAL_S = 1      # seconds between ensembles (1 for 1 Hz, 4 for 0.25 Hz)
MIN_GAP_SECONDS     = 3600   # gaps longer than this define separate segments
MAIN_SEGMENT        = -1     # -1 = last segment, 0 = first, 1 = second, etc.
SAVE_BEAM           = True   # extract and save beam coordinate segment
SAVE_ENU            = True   # extract and save ENU coordinate segment (if present)

Then run all cells top to bottom.

---

Known limitations and caveats

Coordinate system
If the ADCP was configured in ENU mode (EX11111), the onboard coordinate transform is applied before recording. The raw beam measurements are discarded — only the transformed East/North/Up values are stored. The notebook detects this automatically and saves separate files for each coordinate system segment.

Magnetic declination
Heading is recorded as magnetic heading. Magnetic declination is not applied in this notebook and must be corrected before using heading for coordinate transforms or directional analysis. Check NOAA NCEI or BGS for the declination at your deployment location and date.

GPS Navigation block
The Wave Glider Navigation block (RDI ID 0x2000) uses BAM4 (Binary Angle Measurement) encoding for lat/lon, not the standard 1/10,000,000 degree format. The decoding formula is value × 180 / 2³¹. Byte offsets were verified by inspection against known deployment coordinates.

Pressure sensor
The WorkHorse pressure sensor is designed for moored instruments at fixed depth. On a surface Wave Glider it reads ~0–1 dbar and tracks wave surface elevation rather than instrument depth.

Memory
The entire .ENS file is loaded into RAM. A 3.8 GB file requires approximately 4 GB of available RAM.

Salinity
Salinity is a fixed instrument setting (ES command), not a measured variable. It is stored as a global attribute in the NetCDF file.

---

Next steps towards L1 processing

For the beam coordinate segment:

• Apply magnetic declination correction to heading
• Transform beam → instrument → Earth coordinates using heading, pitch, and roll
• Apply correlation threshold (> 64 counts) to mask bad velocity data
• Apply percent good threshold (pg_type=4 > 50%) as quality flag
• Correct for platform motion using GPS velocity or IMU data (if available)
• Ensemble average to reduce noise (e.g. 5-minute bins)
• Nice paper describing steps towards science grade data: Hodges et al., 2023

For the ENU coordinate segment:

• Coordinate transform already applied by instrument
• Magnetic declination correction still required
• Platform motion correction still applies
• Quality masking with correlation and percent good still recommended

---

Repository structure

waveglider-adcp/
├── README.md
├── LICENSE
├── environment.yml
├── .gitignore
├── notebooks/
│   └── waveglider_adcp_processing.ipynb
├── data/
│   └── .gitkeep
└── docs/
    └── config_file_guide.md
    └── ADCP_300KHZ_RAW_1HZ_WAM.txt

---

References

• Teledyne RDI WorkHorse Commands and Output Data Format — binary format specification (search "WorkHorse ADCP Commands and Output Data Format")
• RDI data type IDs and field offsets — pycurrents documentation
• CF Conventions 1.8 — NetCDF metadata standard
• BODC SeaDataNet P01 vocabulary — standard variable names

---

Citation

If you use this code in your work, please cite it using the information in CITATION.cff.

---

Author

Johan Edholm, University of Gothenburg (UGOT)
Contact: [johan.edholm @ gu.se]