Dataset Card for KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos

Dataset Summary

We present a novel high-quality dataset for animal behavior recognition from drone videos. The dataset is focused on Kenyan wildlife and contains behaviors of giraffes, plains zebras, and Grevy's zebras. The dataset consists of more than 10 hours of annotated videos, and it includes eight different classes, encompassing seven types of animal behavior and an additional category for occluded instances. In the annotation process for this dataset, a team of 10 people was involved, with an expert zoologist overseeing the process. Each behavior was labeled based on its distinctive features, using a standardized set of criteria to ensure consistency and accuracy across the annotations. The dataset was collected using drones that flew over the animals in the Mpala Research Centre in Kenya, providing high-quality video footage of the animal's natural behaviors. The drone footage is captured at a resolution of 5472 x 3078 pixels, and the videos were recorded at a frame rate of 29.97 frames per second.

Supported Tasks and Leaderboards

The results of our evaluation using I3D, SlowFast, and X3D architectures are given in the table below. For each one, the model was trained for 120 epochs with batch size of 5. For more information on these results, see our [paper](coming soon).

Method	All	Giraffes	Plains Zebras	Grevy’s Zebras
I3D (16x5)	53.41	61.82	58.75	46.73
SlowFast (16x5, 4x5)	52.92	61.15	60.60	47.42
X3D (16x5)	61.9	65.1	63.11	51.16

Languages

English

Dataset Structure

Under KABR/dataset/image/, the data has been archived into .zip files, which are split into 2GB files. These must be recombined and extracted. After cloning and navigating into the repository, you can use the following commands to do the reconstruction:

cd KABR/dataset/image/
cat giraffes_part_* > giraffes.zip
md5sum giraffes.zip # Compare this to what's shown with `cat giraffes_md5.txt`
unzip giraffes.zip
rm -rf giraffes_part_*

# Similarly for `zebras_grevys_part_*` and `zebras_plains_part_*`

Alternatively, there is a download script, download.py, which allows a download of the entire dataset in its established format without requiring one to clone the repository (cloning requires at least double the size of the dataset to store). To proceed with this approach, download download.py to the system where you want to access the data. Then, in the same directory as the script, run the following to begin the download:

pip install requests
python download.py

This script then downloads all the files present in the repository (without making a clone of the .git directory, etc.), concatenates the part files to their ZIP archives, verifies the MD5 checksums, extracts, and cleans up so that the folder structure, as described below, is present.

Note that it will require approximately 116GB of free space to complete this process, though the final dataset will only take about 61GB of disk space (the script removes the extra files after checking the download was successful).

The KABR dataset follows the Charades format:

KABR
    /dataset
        /image
            /video_1
                /image_1.jpg
                /image_2.jpg
                ...
                /image_n.jpg
            /video_2
                /image_1.jpg
                /image_2.jpg
                ...
                /image_n.jpg
            ...
            /video_n
                /image_1.jpg
                /image_2.jpg
                /image_3.jpg
                ...
                /image_n.jpg
    /annotation
        /classes.json
        /train.csv
        /val.csv

The dataset can be directly loaded and processed by the SlowFast framework.

Informational Files

• KABR/configs: examples of SlowFast framework configs.
• KABR/annotation/distribution.xlsx: distribution of classes for all videos.

Scripts:

• image2video.py: Encode image sequences into the original video.
  • For example, [image/G0067.1, image/G0067.2, ..., image/G0067.24] will be encoded into video/G0067.mp4.
• image2visual.py: Encode image sequences into the original video with corresponding annotations.
  • For example, [image/G0067.1, image/G0067.2, ..., image/G0067.24] will be encoded into visual/G0067.mp4.

Data Instances

Naming: Within the image folder, the video_n folders are named as follows (X indicates a number):

• G0XXX.X - Giraffes
• ZP0XXX.X - Plains Zebras
• ZG0XXX.X - Grevy's Zebras
• Within each of these folders the images are simply X.jpg.

Note: The dataset consists of a total of 1,139,893 frames captured from drone videos. There are 488,638 frames of Grevy's zebras, 492,507 frames of plains zebras, and 158,748 frames of giraffes.

Data Fields

There are 14,764 unique behavioral sequences in the dataset. These consist of eight distinct behaviors:

• Walk
• Trot
• Run: animal is moving at a cantor or gallop
• Graze: animal is eating grass or other vegetation
• Browse: animal is eating trees or bushes
• Head Up: animal is looking around or observe surroundings
• Auto-Groom: animal is grooming itself (licking, scratching, or rubbing)
• Occluded: animal is not fully visible

Data Splits

Training and validation sets are indicated by their respective CSV files (train.csv and val.csv), located within the annotation folder.

Dataset Creation

Curation Rationale

We present a novel high-quality dataset for animal behavior recognition from drone videos. The dataset is focused on Kenyan wildlife and contains behaviors of giraffes, plains zebras, and Grevy's zebras. The dataset consists of more than 10 hours of annotated videos, and it includes eight different classes, encompassing seven types of animal behavior and an additional category for occluded instances. In the annotation process for this dataset, a team of 10 people was involved, with an expert zoologist overseeing the process. Each behavior was labeled based on its distinctive features, using a standardized set of criteria to ensure consistency and accuracy across the annotations. The dataset was collected using drones that flew over the animals in the Mpala Research Centre in Kenya, providing high-quality video footage of the animal's natural behaviors. We believe that this dataset will be a valuable resource for researchers working on animal behavior recognition, as it provides a diverse and high-quality set of annotated videos that can be used for evaluating deep learning models. Additionally, the dataset can be used to study the behavior patterns of Kenyan animals and can help to inform conservation efforts and wildlife management strategies.

We provide a detailed description of the dataset and its annotation process, along with some initial experiments on the dataset using conventional deep learning models. The results demonstrate the effectiveness of the dataset for animal behavior recognition and highlight the potential for further research in this area.

Source Data

Initial Data Collection and Normalization

Data was collected from 6 January 2023 through 21 January 2023 at the Mpala Research Centre in Kenya under a Nacosti research license. We used DJI Mavic 2S drones equipped with cameras to record 5.4K resolution videos (5472 x 3078 pixels) from varying altitudes and distances of 10 to 50 meters from the animals (distance was determined by circumstances and safety regulations).

Mini-scenes were extracted from these videos to reduce the impact of drone movement and facilitate human annotation. Animals were detected in frame using YOLOv8, then the SORT tracking algorithm was applied to follow their movement. A 400 by 300 pixel window, centered on the animal, was then extracted; this is the mini-scene.

Annotations

Annotation process

In the annotation process for this dataset, a team of 10 people was involved, with an expert zoologist overseeing the process. Each behavior was labeled based on its distinctive features, using a standardized set of criteria to ensure consistency and accuracy across the annotations.

Personal and Sensitive Information

Though there are endangered species included in this data, exact locations are not provided and their safety is assured by their location within the preserve.

Considerations for Using the Data

Other Known Limitations

This data exhibits a long-tailed distribution due to the natural variation in frequency of the observed behaviors.

Additional Information

Authors

• Maksim Kholiavchenko (Rensselaer Polytechnic Institute) - ORCID: 0000-0001-6757-1957
• Jenna Kline (The Ohio State University)
• Michelle Ramirez (The Ohio State University)
• Sam Stevens (The Ohio State University)
• Alec Sheets (The Ohio State University) - ORCID: 0000-0002-3737-1484
• Reshma Ramesh Babu (The Ohio State University) - ORCID: 0000-0002-2517-5347
• Namrata Banerji (The Ohio State University) - ORCID: 0000-0001-6813-0010
• Elizabeth Campolongo (Imageomics Institute) - ORCID: 0000-0003-0846-2413
• Nina Van Tiel (Eidgenössische Technische Hochschule Zürich) - ORCID: 0000-0001-6393-5629
• Jackson Miliko (Mpala Research Centre)
• Eduardo Bessa (Universidade de Brasília) - ORCID: 0000-0003-0606-5860
• Tanya Berger-Wolf (The Ohio State University) - ORCID: 0000-0001-7610-1412
• Daniel Rubenstein (Princeton University) - ORCID: 0000-0001-9049-5219
• Charles Stewart (Rensselaer Polytechnic Institute)

Licensing Information

This dataset is dedicated to the public domain for the benefit of scientific pursuits. We ask that you cite the dataset using the below citation if you make use of it in your research.

Citation Information

@misc{KABR_Data,
  author = {Maksim Kholiavchenko and et. al},
  title = {KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos},
  year = {2023},
  url = {https://huggingface.co/datasets/imageomics/KABR},
  doi = {10.57967/hf/1010},
  publisher = {Model Database}
}

Contributions

The Imageomics Institute is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) Institute program under Award #2118240 (Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning).