\n",
"\n",
" * Distributed under the terms of the GPL License\n",
" * Maintainer: dcline@mbari.org\n",
" * Authors: Danelle Cline dcline@mbari.org, John Ryan ryjo@mbari.org\n",
"\n",
"## Kernel Selection\n",
"\n",
"If running in SageMaker, the Python 3 (Data Science) kernel is sufficient.\n",
"The Python 3 (TensorFlow 2.3 Python 3.7 GPU Optimized) will run the inference code faster for a higher cost. For more advanced users,\n",
"[SageMaker Batch Transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html)\n",
"is recommended to process data in bulk."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lFyO865Me0p8"
},
"source": [
"## Applying Machine Learning to classify blue whale A calls\n",
"\n",
"---\n",
"Essential to detection and classification of marine mammal vocalizations are the distinct acoustic attributes of those vocalizations. Machine learning (ML) is an effective way to recognize acoustic attributes and reliably classify such vocalizations.\n",
"\n",
"In this brief tutorial, we will:\n",
"* tap into an extensive (6+ years and growing) archive of sound recordings from a deep-sea location [along the eastern margin of the North Pacific Ocean](https://www.mbari.org/at-sea/cabled-observatory/),\n",
"* illustrate the beautiful songs produced by baleen whales, and\n",
"* demonstrate the application of ML to classify one of the three types of calls produced by blue whales in their songs.\n",
"\n",
"If you use this data set or tutorial, please [cite our project](https://ieeexplore.ieee.org/document/7761363)[1]."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yrwRNVqwe0p8"
},
"source": [
"## Install required dependencies\n",
"\n",
"First, let's install the required software dependencies. \n",
"\n",
"If you are using this notebook in a cloud environment, select a Python3 compatible kernel and run this next section. This only needs to be done once for the duration of this notebook.\n",
"\n",
"If you are working on local computer, you can skip this next cell. Change your kernel to *pacific-sound-notebooks*, which you installed according to the instructions in the [README](https://github.com/mbari-org/pacific-sound-notebooks/) - this has all the dependencies that are needed. "
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "8PVdtRsEe0p9",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "13d2f78c-0157-42ae-c8b0-c12bdddfb043"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\r0% [Working]\r \rHit:1 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease\n",
"\r0% [Connecting to archive.ubuntu.com] [Connecting to security.ubuntu.com (91.18\r0% [1 InRelease gpgv 3,626 B] [Connecting to archive.ubuntu.com] [Waiting for h\r \rIgn:2 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease\n",
"\r0% [1 InRelease gpgv 3,626 B] [Connecting to archive.ubuntu.com] [Waiting for h\r \rHit:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease\n",
"\r0% [1 InRelease gpgv 3,626 B] [Connecting to archive.ubuntu.com] [Waiting for h\r \rHit:4 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release\n",
"Hit:5 http://security.ubuntu.com/ubuntu bionic-security InRelease\n",
"Hit:6 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease\n",
"Hit:7 http://archive.ubuntu.com/ubuntu bionic InRelease\n",
"Hit:9 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease\n",
"Hit:10 http://archive.ubuntu.com/ubuntu bionic-updates InRelease\n",
"Hit:11 http://archive.ubuntu.com/ubuntu bionic-backports InRelease\n",
"Hit:12 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease\n",
"Hit:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease\n",
"Reading package lists... Done\n",
"Reading package lists... Done\n",
"Building dependency tree \n",
"Reading state information... Done\n",
"libsndfile1 is already the newest version (1.0.28-4ubuntu0.18.04.2).\n",
"The following package was automatically installed and is no longer required:\n",
" libnvidia-common-460\n",
"Use 'apt autoremove' to remove it.\n",
"0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded.\n",
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (22.2.2)\n",
"\u001B[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001B[0m\u001B[33m\n",
"\u001B[0m\u001B[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001B[0m\u001B[33m\n",
"\u001B[0m\u001B[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001B[0m\u001B[33m\n",
"\u001B[0m\u001B[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001B[0m\u001B[33m\n",
"\u001B[0m"
]
}
],
"source": [
"!apt-get update -y && apt-get install -y libsndfile1\n",
"!python -m pip install --upgrade pip\n",
"!pip install tensorflow==2.4.1 --quiet\n",
"!pip install boto3 --quiet\n",
"!pip install oceansoundscape --quiet"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QGi0FNA1e0p9"
},
"source": [
"### Import all packages"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"id": "vHWVyGMEe0p9"
},
"outputs": [],
"source": [
"import boto3\n",
"from botocore import UNSIGNED\n",
"from botocore.client import Config\n",
"import cv2\n",
"from oceansoundscape.spectrogram.signal import psd_1sec\n",
"from oceansoundscape.raven import BLEDParser\n",
"from oceansoundscape.spectrogram import conf, denoise, utils\n",
"import os\n",
"from pathlib import Path\n",
"import soundfile as sf\n",
"import json\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from matplotlib.patches import Rectangle"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "f4DqqeYje0p-"
},
"source": [
"## Why use 2 kHz data?\n",
"\n",
"Because we are studying the low-frequency calls of blue whales, we don't need the original recordings sampled at 256 kHz. Instead, we will use the 2 kHz decimated data in [WAV](https://en.wikipedia.org/wiki/WAV) format; these are stored in an s3 bucket named pacific-sound-2khz. For more information on the storage and organization of the 2kHz data, please see the [2kHz example](https://docs.mbari.org/pacific-sound/notebooks/PacificSound_2kHz/).\n",
"\n",
"## List the contents of a monthly directory\n",
"\n",
"Between August 2015 and July 2021 (a 6-year period), the highest levels of blue whale song activity off central California were detected during November 2017. Let's start by listing the daily 2 kHz files for that month."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "pPz9GAK-e0p-",
"outputId": "b69311e5-65d1-469c-d7a2-2aad5b4f44f0",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2017/11/MARS-20171101T000000Z-2kHz.wav\n",
"2017/11/MARS-20171102T000000Z-2kHz.wav\n",
"2017/11/MARS-20171103T000000Z-2kHz.wav\n",
"2017/11/MARS-20171104T000000Z-2kHz.wav\n",
"2017/11/MARS-20171105T000000Z-2kHz.wav\n",
"2017/11/MARS-20171106T000000Z-2kHz.wav\n",
"2017/11/MARS-20171107T000000Z-2kHz.wav\n",
"2017/11/MARS-20171108T000000Z-2kHz.wav\n",
"2017/11/MARS-20171109T000000Z-2kHz.wav\n",
"2017/11/MARS-20171110T000000Z-2kHz.wav\n",
"2017/11/MARS-20171111T000000Z-2kHz.wav\n",
"2017/11/MARS-20171112T000000Z-2kHz.wav\n",
"2017/11/MARS-20171113T000000Z-2kHz.wav\n",
"2017/11/MARS-20171114T000000Z-2kHz.wav\n",
"2017/11/MARS-20171115T000000Z-2kHz.wav\n",
"2017/11/MARS-20171116T000000Z-2kHz.wav\n",
"2017/11/MARS-20171117T000000Z-2kHz.wav\n",
"2017/11/MARS-20171118T000000Z-2kHz.wav\n",
"2017/11/MARS-20171119T000000Z-2kHz.wav\n",
"2017/11/MARS-20171120T000000Z-2kHz.wav\n",
"2017/11/MARS-20171121T000000Z-2kHz.wav\n",
"2017/11/MARS-20171122T000000Z-2kHz.wav\n",
"2017/11/MARS-20171123T000000Z-2kHz.wav\n",
"2017/11/MARS-20171124T000000Z-2kHz.wav\n",
"2017/11/MARS-20171125T000000Z-2kHz.wav\n",
"2017/11/MARS-20171126T000000Z-2kHz.wav\n",
"2017/11/MARS-20171127T000000Z-2kHz.wav\n",
"2017/11/MARS-20171128T000000Z-2kHz.wav\n",
"2017/11/MARS-20171129T000000Z-2kHz.wav\n",
"2017/11/MARS-20171130T000000Z-2kHz.wav\n",
"2017/11/copy.sh\n"
]
}
],
"source": [
"s3 = boto3.client('s3',\n",
" aws_access_key_id='',\n",
" aws_secret_access_key='',\n",
" config=Config(signature_version=UNSIGNED))\n",
"\n",
"year = 2017\n",
"month = 11\n",
"bucket = 'pacific-sound-2khz'\n",
"\n",
"for obj in s3.list_objects_v2(Bucket=bucket, Prefix=f'{year:04d}/{month:02d}')['Contents']:\n",
" print(obj['Key'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T5vm59Zce0p_"
},
"source": [
"## A view of baleen whale song\n",
"\n",
"Let's produce a spectrogram with sufficient resolution in time and frequency to see the blue whale song with enough resolution to visually identify. We'll limit the exercise to a single hour from a day with calls of variable received intensity (signal strength)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YrPahQDbe0p_"
},
"source": [
"### Download a single 2 kHz file"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "xNH2L5RJe0qA"
},
"outputs": [],
"source": [
"year = 2017\n",
"month = 11\n",
"wav_filename = 'MARS-20171101T000000Z-2kHz.wav'\n",
"bucket = 'pacific-sound-2khz'\n",
"key = f'{year:04d}/{month:02d}/{wav_filename}'\n",
"\n",
"s3 = boto3.resource('s3',\n",
" aws_access_key_id='',\n",
" aws_secret_access_key='',\n",
" config=Config(signature_version=UNSIGNED))\n",
" \n",
"# only download if needed\n",
"if not Path(wav_filename).exists():\n",
"\n",
" # Alternatively, it can be downloaded directly in SageMaker with\n",
" # !aws s3 cp s3://{bucket}/{key} . \n",
"\n",
" print('Downloading') \n",
" s3.Bucket(bucket).download_file(key, wav_filename)\n",
" print('Done') "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mWfJb-Fxe0qA"
},
"source": [
"### Subset to the 5th hour of the day"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "6WtzWTbte0qA"
},
"outputs": [],
"source": [
"sample_rate = int(2e3)\n",
"start_hour = 5\n",
"start_frame = int(sample_rate * start_hour * 3600)\n",
"duration_frames = int(sample_rate* 3600)\n",
"\n",
"pacsound_file = sf.SoundFile(wav_filename)\n",
"pacsound_file.seek(start_frame)\n",
"x = pacsound_file.read(duration_frames, dtype='float32')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lX8gl1ize0qA"
},
"source": [
"### Plot the full 2 kHz spectrogram\n",
"\n",
"Lots of biophony (sounds of ocean life) are represented in this spectrogram. Humpback whale songs are dominant above ~ 100 Hz, while blue and fin whale songs are dominant below ~ 100 Hz. The energy of blue whale A calls is largely between ~70 and 90 Hz (between the white dashed lines). \n",
"\n",
"For more details about creating a calibrated spectrogram, see the [2 kHz tutorial](https://docs.mbari.org/pacific-sound/notebooks/PacificSound_2kHz/)."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"id": "nhVpu3xDe0qA",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "3af1d343-74b5-4dd0-8e8c-042e7cfd3ab8"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"Text(0.5, 1.0, 'Calibrated spectrum levels')"
]
},
"metadata": {},
"execution_count": 20
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"