Copy

This will copy data specified between two dates from a local directory to a remote S3 bucket.

Usage

python opendata/run.py copy [options] [bucket name] 

Options can be seen with the --help option, e.g. python opendata/run.py opendata copy --help

  Copy data to a bucket, e.g. python opendata/run.py copy pacific-sound-2khz -d
  /Volumes/PAM_Archive/decimated_2khz

Options:
  -d, --wav_path TEXT  Path to the directory with wav file to upload
  -p, --profile TEXT    AWS profile to use. Default is 'soudscapeopendata'
  -s, --start TEXT     Starting date in UTC
  -e, --end TEXT       Ending date in UTC
  -h, --help           Show this message and exit.

Example output

Run as user pam on cetacean.shore.mbari.org

export HOME=/opt/pam/deployed
python opendata/run.py copy pacific-sound-2khz -d /PAM_Analysis/decimated_2kHz/ -s 20180101T000000Z -e 20180101T000000Z

Setting maximum bandwidth to 62 MB/sec
Fetching files to copy :  52% (557 of 1069) |#####################################                                   | Elapsed Time: 0:00:16 ETA:   0:00:13
...
Copying : 100% (2 of 2) |############################################################################################| Elapsed Time: 0:00:28 Time:  0:00:28