{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "TfUZYrKbqlF7"
},
"source": [
"# Annotate Video with Detections\n",
"\n",
"---\n",
"\n",
"[](https://colab.research.google.com/github/roboflow/supervision/blob/develop/docs/notebooks/annotate-video-with-detections.ipynb)\n",
"\n",
"One of the most common requirements of computer vision applications is detecting objects in images and displaying bounding boxes around those objects. In this cookbook we'll walk through the steps on how to utilize the open source Roboflow ecosystem to accomplish this task on a video. Let's dive in!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rUo1u17yqlF9"
},
"source": [
"## Before you start\n",
"\n",
"Let's make sure that we have access to GPU. We can use `nvidia-smi` command to do that. In case of any problems navigate to `Edit` -> `Notebook settings` -> `Hardware accelerator`, set it to `GPU`, and then click `Save`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0Er370WTqlF9",
"outputId": "57b00a1e-87ea-4912-b4f9-5b94f0e6bf48",
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fri Feb 23 03:15:00 2024 \n",
"+---------------------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 |\n",
"|-----------------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|=========================================+======================+======================|\n",
"| 0 Tesla V100-SXM2-16GB Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 33C P0 24W / 300W | 0MiB / 16384MiB | 0% Default |\n",
"| | | N/A |\n",
"+-----------------------------------------+----------------------+----------------------+\n",
" \n",
"+---------------------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=======================================================================================|\n",
"| No running processes found |\n",
"+---------------------------------------------------------------------------------------+\n"
]
}
],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WuSnN7ejqlF-"
},
"source": [
"## Installing Dependencies\n",
"\n",
"In this cookbook we'll be utilizing the open source packages [Inference](https://inference.roboflow.com/) and [Supervision](https://supervision.roboflow.com/latest/) to accomplish our goals. Let's get those installed in our notebook with pip."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1YgQGum4qlF-",
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"!pip install -q inference-gpu \"supervision\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PZRnzh7EqlF-"
},
"source": [
"## Download a Video Asset\n",
"\n",
"First, let's download a video that we can detect objects in. Supervision comes with a great utility called Assets to help us hit the ground running. When we run this script, the video is saved in our local directory and can be accessed with the variable `path_to_video`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NMg4KOFiqlF-"
},
"outputs": [],
"source": [
"from supervision.assets import download_assets, VideoAssets\n",
"\n",
"# Download a supervision video asset\n",
"path_to_video = download_assets(VideoAssets.PEOPLE_WALKING)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "x93oxF0zrQOU"
},
"source": [
"As a result, we've downloaded a video. Let's take a look at the video below. Keep in mind that the video preview below works only in the web version of the cookbooks and not in Google Colab.\n",
"\n",
"\n",
" \n",
"
\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T8bU7fQ6qlF-"
},
"source": [
"## Detecting Objects\n",
"\n",
"For this example, the objects in the video that we'd like to detect are people. In order to display bounding boxes around the people in the video, we first need a way to detect them. We'll be using the open source [Inference](https://github.com/roboflow/inference) package for this task. Inference allows us to quickly use thousands of models, including fine tuned models from [Roboflow Universe](https://universe.roboflow.com/), with a few lines of code. We'll also utilize a few utilities for working with our video data from the [Supervision](https://github.com/roboflow/supervision) package."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "hlONWHfQqlF-",
"outputId": "81b277fb-ea7c-4242-f49d-5af0c665ea58"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Detections(xyxy=array([[1140., 951., 1245., 1079.],\n",
" [ 666., 648., 745., 854.],\n",
" [ 34., 794., 142., 990.],\n",
" [1140., 505., 1211., 657.],\n",
" [ 260., 438., 332., 612.],\n",
" [1413., 702., 1523., 887.],\n",
" [1462., 472., 1543., 643.],\n",
" [1446., 318., 1516., 483.],\n",
" [ 753., 451., 821., 623.],\n",
" [ 924., 172., 983., 307.],\n",
" [1791., 144., 1852., 275.],\n",
" [ 93., 132., 146., 251.],\n",
" [ 708., 240., 765., 388.],\n",
" [ 200., 44., 267., 161.],\n",
" [1204., 131., 1255., 266.],\n",
" [ 569., 267., 628., 408.],\n",
" [1163., 150., 1210., 280.],\n",
" [ 799., 78., 847., 204.],\n",
" [1690., 152., 1751., 283.],\n",
" [ 344., 495., 396., 641.],\n",
" [1722., 77., 1782., 178.]]),\n",
" mask=None,\n",
" confidence=array([0.83215541, 0.80572134, 0.7919845 , 0.7912274 , 0.77121079,\n",
" 0.7599591 , 0.75711554, 0.75494027, 0.73076195, 0.71452248,\n",
" 0.69572842, 0.65269446, 0.63952065, 0.62914598, 0.61361706,\n",
" 0.5968492 , 0.55311316, 0.5470854 , 0.54070991, 0.52209878,\n",
" 0.41217673]),\n",
" class_id=array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),\n",
" tracker_id=None,\n",
" data={'class_name': array(['person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person', 'person', 'person', 'person',\n",
" 'person', 'person', 'person'], dtype='"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create a bounding box annotator object.\n",
"box_annotator = sv.BoxAnnotator()\n",
"\n",
"# Annotate our frame with detections.\n",
"annotated_frame = box_annotator.annotate(scene=frame.copy(), detections=detections)\n",
"\n",
"# Display the frame.\n",
"sv.plot_image(annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o8SsyCid6YV3"
},
"source": [
"Notice that we create a `box_annotator` variable by initalizing a [BoxAnnotator](https://supervision.roboflow.com/latest/detection/annotators/#boxannotator). We can change the color and thickness, but for simplicity we keep the defaults. There are a ton of easy to use [annotators](https://supervision.roboflow.com/latest/detection/annotators/) available in the Supervision package other than a bounding box that are fun to play with."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gKHSYO14vGC7"
},
"source": [
"## Saving Bounding Boxes to a Video\n",
"\n",
"Let's wrap up our code by utilizing a `VideoSink` to draw bounding boxes and save the resulting video. Take a peak at the final code example below. This can take a couple minutes deppending on your runtime and since since we're processing a full video. Feel free to skip ahead to see the resulting video."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Es4SUdPGqlF-"
},
"outputs": [],
"source": [
"from tqdm import tqdm\n",
"\n",
"# Create a video_info object for use in the VideoSink.\n",
"video_info = sv.VideoInfo.from_video_path(video_path=path_to_video)\n",
"\n",
"# Create a VideoSink context manager to save our frames.\n",
"with sv.VideoSink(target_path=\"output.mp4\", video_info=video_info) as sink:\n",
"\n",
" # Iterate through frames yielded from the frame_generator.\n",
" for frame in tqdm(frame_generator, total=video_info.total_frames):\n",
"\n",
" # Run inference on our frame.\n",
" result = model.infer(frame)[0]\n",
"\n",
" # Parse the result into the detections data model.\n",
" detections = sv.Detections.from_inference(result)\n",
"\n",
" # Apply bounding box to detections on a copy of the frame.\n",
" annotated_frame = box_annotator.annotate(\n",
" scene=frame.copy(),\n",
" detections=detections\n",
" )\n",
"\n",
" # Write the annotated frame to the video sink.\n",
" sink.write_frame(frame=annotated_frame)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EXTE3zWxqlF-"
},
"source": [
"\n",
" \n",
"
\n",
"
\n",
"\n",
"\n",
"In the code above we've created a`video_info` variable to pass information about the video to our `VideoSink`. The `VideoSink` is a cool little context manager that allows us to `write_frames()` to a video output file. We're also optionally using `tqdm` to display a progress bar with a % complete. We only scratched the surface of all of the customizable Annotators and additional features that Supervision and Inference have to offer. Stay tuned for more cookbooks on how to take advantage of them in your computer vision applications. Happy building! \ud83d\ude80"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "V100",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 0
}