xiaoxue_admin
/
supervision
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

initial changes

This commit is contained in:
SkalskiP 2024-03-25 13:11:47 +01:00
parent 781a064d8a
commit 57c6182325
1 changed files with 170 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

211
docs/how_to/detect_and_annotate.md
View File

@ -4,17 +4,29 @@ comments: true
# Detect and Annotate
Supervision offers a streamlined solution to effortlessly annotate predictions from a
range of object detection and segmentation models. This guide demonstrates how to
execute inference using the YOLOv8 model with either the
[Inference](https://github.com/roboflow/inference) or
[Ultralytics](https://github.com/ultralytics/ultralytics) packages. Following this,
you'll learn how to import these predictions into Supervision for image annotation
purposes.
Supervision provides a seamless process for annotating predictions generated by various
object detection and segmentation models. This guide shows how to perform inference
with the [Inference](https://github.com/roboflow/inference),
[Ultralytics](https://github.com/ultralytics/ultralytics) or
[Transformers](https://github.com/huggingface/transformers) packages. Following this,
you'll learn how to import these predictions into Supervision and use them to annotate
source image.
## Run Inference
First, you'll need to obtain predictions from your object detection or segmentation model.
First, you'll need to obtain predictions from your object detection or segmentation
model.
=== "Inference"
```python
import cv2
from inference import get_model
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
```
=== "Ultralytics"
@ -27,26 +39,52 @@ First, you'll need to obtain predictions from your object detection or segmentat
results = model(image)[0]
```
=== "Inference"
=== "Transformers"
```python
import cv2
from inference.models.utils import get_roboflow_model
import torch
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>)
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<PATH TO IMAGE>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
```
## Load Predictions into Supervision
Now that we have predictions from a model, we can load them into Supervision.
=== "Inference"
We can do so using the [`sv.Detections.from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) method, which accepts model results from both detection and segmentation models.
```{ .py hl_lines="2 8" }
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
```
=== "Ultralytics"
We can do so using the [`sv.Detections.from_ultralytics`](detection/core/#supervision.detection.core.Detections.from_ultralytics) method, which accepts model results from both detection and segmentation models.
```python
```{ .py hl_lines="2 8" }
import cv2
import supervision as sv
from ultralytics import YOLO
@ -57,34 +95,123 @@ Now that we have predictions from a model, we can load them into Supervision.
detections = sv.Detections.from_ultralytics(results)
```
=== "Inference"
=== "Transformers"
We can do so using the [`sv.Detections.from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) method, which accepts model results from both detection and segmentation models.
We can do so using the [`sv.Detections.from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) method, which accepts model results from both detection and segmentation models.
```python
import cv2
```{ .py hl_lines="2 19" }
import torch
import supervision as sv
from inference.models.utils import get_roboflow_model
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<PATH TO IMAGE>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
```
You can conveniently load predictions from other computer vision frameworks and libraries using:
You can load predictions from other computer vision frameworks and libraries using:
- [`from_deepsparse`](detection/core/#supervision.detection.core.Detections.from_deepsparse) ([Deepsparse](https://github.com/neuralmagic/deepsparse))
- [`from_detectron2`](detection/core/#supervision.detection.core.Detections.from_detectron2) ([Detectron2](https://github.com/facebookresearch/detectron2))
- [`from_mmdetection`](detection/core/#supervision.detection.core.Detections.from_mmdetection) ([MMDetection](https://github.com/open-mmlab/mmdetection))
- [`from_inference`](detection/core/#supervision.detection.core.Detections.from_inference) ([Roboflow Inference](https://github.com/roboflow/inference))
- [`from_sam`](detection/core/#supervision.detection.core.Detections.from_sam) ([Segment Anything Model](https://github.com/facebookresearch/segment-anything))
- [`from_transformers`](detection/core/#supervision.detection.core.Detections.from_transformers) ([HuggingFace Transformers](https://github.com/huggingface/transformers))
- [`from_yolo_nas`](detection/core/#supervision.detection.core.Detections.from_yolo_nas) ([YOLO-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md))
## Annotate Image
## Annotate Image with Detections
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](annotators/#supervision.annotators.core.LabelAnnotator) classes. If you are running the segmentation model [`sv.MaskAnnotator`](annotators/#supervision.annotators.core.MaskAnnotator) is a drop-in replacement for [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) that will allow you to draw masks instead of boxes.
Finally, we can annotate the image with the predictions. Since we are working with an object detection model, we will use the [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) and [`sv.LabelAnnotator`](annotators/#supervision.annotators.core.LabelAnnotator) classes.
=== "Inference"
```{ .py hl_lines="10-16" }
import cv2
import supervision as sv
from inference import get_model
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
bounding_box_annotator = sv.BoundingBoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = bounding_box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Ultralytics"
```{ .py hl_lines="10-16" }
import cv2
import supervision as sv
from ultralytics import YOLO
model = YOLO("yolov8n.pt")
image = cv2.imread(<PATH TO IMAGE>)
results = model(image)[0]
detections = sv.Detections.from_ultralytics(results)
bounding_box_annotator = sv.BoundingBoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = bounding_box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
=== "Transformers"
```{ .py hl_lines="21-27" }
import torch
import supervision as sv
from PIL import Image
from transformers import DetrImageProcessor, DetrForObjectDetection
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
image = Image.open(<PATH TO IMAGE>)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
width, height = image.size
target_size = torch.tensor([[height, width]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_size)[0]
detections = sv.Detections.from_transformers(results)
bounding_box_annotator = sv.BoundingBoxAnnotator()
label_annotator = sv.LabelAnnotator()
annotated_image = bounding_box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![basic-annotation](https://media.roboflow.com/supervision_annotate_example.png)
## Display Custom Labels
<TODO>
=== "Ultralytics"
@ -102,9 +229,9 @@ Finally, we can annotate the image with the predictions. Since we are working wi
label_annotator = sv.LabelAnnotator()
labels = [
model.model.names[class_id]
for class_id
in detections.class_id
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = bounding_box_annotator.annotate(
@ -118,9 +245,9 @@ Finally, we can annotate the image with the predictions. Since we are working wi
```python
import cv2
import supervision as sv
from inference.models.utils import get_roboflow_model
from inference import get_model
model = get_roboflow_model(model_id="yolov8n-640", api_key=<ROBOFLOW API KEY>
model = get_model(model_id="yolov8n-640")
image = cv2.imread(<PATH TO IMAGE>)
results = model.infer(image)[0]
detections = sv.Detections.from_inference(results)
@ -128,18 +255,20 @@ Finally, we can annotate the image with the predictions. Since we are working wi
bounding_box_annotator = sv.BoundingBoxAnnotator()
label_annotator = sv.LabelAnnotator()
labels = [
f"{class_name} {confidence:.2f}"
for class_name, confidence
in zip(detections['class_name'], detections.confidence)
]
annotated_image = bounding_box_annotator.annotate(
scene=image, detections=detections)
annotated_image = label_annotator.annotate(
scene=annotated_image, detections=detections)
```
![Predictions plotted on an image](https://media.roboflow.com/supervision_annotate_example.png)
![custom-label-annotation](https://media.roboflow.com/supervision-annotator-examples/label-annotator-example-purple.png)
## Display Annotated Image
## Annotate Image with Segmentations
To display the annotated image in Jupyter Notebook or Google Colab, use the [`sv.plot_image`](utils/notebook/#supervision.utils.notebook.plot_image) function.
```python
sv.plot_image(annotated_image)
```
If you are running the segmentation model [`sv.MaskAnnotator`](annotators/#supervision.annotators.core.MaskAnnotator) is a drop-in replacement for [`sv.BoundingBoxAnnotator`](annotators/#supervision.annotators.core.BoundingBoxAnnotator) that will allow you to draw masks instead of boxes.