# Radar PCAP Data Processing Tool

## Git LFS 

This repository leverages [Git Large File Storage](https://git-lfs.github.com) for creating lightweight repositories that have access to large datasets. Please visit the link for details on how to install and then enable this in your system.

After installing `git lfs`, you can download large files with a `git lfs pull`

## Building Instructions

This toolset is supported on *nix platforms at present. 

## Prerequisites

### Package manager libraries

    sudo apt install build-essential libpcap-dev hdf5-tools hdf5-dev cmake git-lfs

### LCM

You can install LCM in a number of ways, I have yet to test the pkg manager 1.3 against this build. 

Try this snippet from the Dockerfile: 

    wget https://github.com/lcm-proj/lcm/archive/refs/tags/v1.4.0.tar.gz \
    && tar xf v1.4.0.tar.gz \
    && cd lcm-1.4.0 \
    && mkdir -p build && cd build \
    && cmake .. \
    && make && make install \
    && ldconfig /usr/local/lib

## Building/Install

Once you have installed the prerequiresites, you can build and install the utility using CMake:

    mkdir build
    cd build
    cmake ..
    make
    sudo make install

## Use

Running the tool with no inputs provides the options: 

    wec-pcap-utils/data [master] » pcaptoradar
    radar packet parser, usage pcaptoradar <pcap filename>
      flags: -a | -b      : print report data for channels A or B
             -h           : add csv header line for reports, use with -a/-b
             -A | -B      : print scanline data for channels A or B
             -H           : add csv header line for scan data, use with -A/-B
             -i           : print imu data from LCM packets in record
             -I           : add csv header line for imu data
             -o [OUTFILE] : convert all data to HDF5 format with filename [OUTFILE]
             -s           : print statistics for information decodable in the file
             -q           : quiet mode, suppress output


### File Statistics (-s)
The tool can be used to see statistical information about a file: 

    » pcaptoradar -s mbariAtSeaWecRadar_00315_20210928171518.pcapng

    Opening file: mbariAtSeaWecRadar_00315_20210928171518.pcapng
    ----------------
    REPORTS_A: 300
    REPORTS_B: 300
    FRAMES_A:  11469
      SCANS:     367008
      REVS_A:    89
    AHRS:      3005
    ----------------
    
### AHRS/IMU CSV data

You can print data from LCM AHRS packets to a csv file:

    » pcaptoradar -iI mbariAtSeaWecRadar_00315_20210928171518.pcapng  

    Opening file: mbariAtSeaWecRadar_00315_20210928171518.pcapng
    pcap_time,lcm_time,latitude,longitude,yaw_true,pitch,roll,x_accel,y_accel,z_accel,n_velocity,e_velocity,d_velocity,altitude,x_rate,y_rate,z_rate, x_rate_temp,
    1632874518.862951,1632874518.862882,36.743080139160156,-121.881141662597656,140.372314,-1.356812,-0.747070,0.074768,0.059814,-0.993958,0.171875,-0.132812,-0.507812,-33.750000,6.517639,-0.134583,1.038208,24.157715,
    1632874518.962927,1632874518.962861,36.743080139160156,-121.881141662597656,140.460205,-1.444702,-0.126343,0.092773,0.031433,-1.018982,0.062500,-0.093750,-0.515625,-33.750000,5.787048,-1.538086,1.018982,24.157715,
    1632874519.062824,1632874519.062752,36.743083953857422,-121.881141662597656,140.537109,-1.713867,0.302124,0.097656,0.003357,-1.034851,0.031250,0.007812,-0.539062,-33.750000,3.441467,-3.595276,0.826721,24.154663,

### Reports Data on Radar Channels

The radar outputs its configuration at 1hz, these packets can be decoded as well to CSV: 

    » pcaptoradar -ah mbariAtSeaWecRadar_00315_20210928171518.pcapng

    Opening file: mbariAtSeaWecRadar_00315_20210928171518.pcapng
    time,channel,range,gain,sea_auto,sea,rain,interference_rejection,target_expansion,target_boost
    1632874519.070718,3,5000,60,0,25,0,0,1,1
    1632874520.070762,3,5000,60,0,25,0,0,1,1
    1632874521.070754,3,5000,60,0,25,0,0,1,1
    1632874522.072089,3,5000,60,0,25,0,0,1,1
    1632874523.070753,3,5000,60,0,25,0,0,1,1
    1632874524.070784,3,5000,60,0,25,0,0,1,1
    1632874525.070753,3,5000,60,0,25,0,0,1,1
    1632874526.070526,3,5000,60,0,25,0,0,1,1
    1632874527.070512,3,5000,60,0,25,0,0,1,1

### Scan Data

Scans are also available to CSV, though the output is quite large:

    » pcaptoradar -ah mbariAtSeaWecRadar_00315_20210928171518.pcapng

#### HDF5 Ouput

If enabled with the `-o [outname]` option, the program will save the ahrs, reports and scan data to an HDF5 file (`*.h5`). This heirarchal data file can be opened using a number of tools:

    » pcaptoradar -o out.h5 mbariAtSeaWecRadar_00315_20210928171518.pcapng

    Opening file: mbariAtSeaWecRadar_00315_20210928171518.pcapng
    Saving Data to: out.h5

## Radar Data Definitions

The radar data arrive on large network packets referred to as `frames`, each containing 32 `scans`. `scans` are a single set of return intensities, referred to in source as `scans` or `spokes`. For the simrad 4G radar that this code supports, here are some constants:

 * Spokes or Scans per rotations: 2048 (at least in how we're currently decoding data)
 * Number of returns in 1 spoke/scan: 1024
 * Number of spoke/scans per frame: 32