feat: store keypoints on detections (#2290)
* docs: add API design principles to contribution guidelines * feat: store keypoints on detections * test: add "keypoints" to internal test cases * docs: document keypoints field semantics and add docstring + dtype guard * refactor: deduplicate keypoints shape check and add K-mismatch guard * docs: clarify Detections.keypoints vs sv.KeyPoints decision rule and add KeyPoints filter example * feat(key_points): add KeyPoints.from_detections() cross-container adapter * test: extend keypoints test coverage — dtype guard, __eq__, dynamic field sets * test: expand keypoints test coverage (M7/M8) * fix: correct validate_xy expected_shape and dimensionality message * fix: add ndim guard in KeyPoints.from_detections * test: add unit tests for KeyPoints.from_detections adapter * refactor: fix class_id cast and import formatting in key_points --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Code <noreply@anthropic.com>
This commit is contained in:
Jirka Borovec
GitHub
parent
0c67942d32
commit
e03111e67c
|
|
@ -12,6 +12,7 @@ Please read and adhere to our [Code of Conduct](https://supervision.roboflow.com
|
|||
|
||||
- [Contribution Guidelines](#contribution-guidelines)
|
||||
- [Contributing Features](#contributing-features)
|
||||
- [API Design Principles](#api-design-principles)
|
||||
- [How to Contribute Changes](#how-to-contribute-changes)
|
||||
- [Installation for Contributors](#installation-for-contributors)
|
||||
- [Code Style and Quality](#code-style-and-quality)
|
||||
|
|
@ -41,6 +42,42 @@ For example, counting objects that cross a line anywhere on an image is a common
|
|||
|
||||
Before you contribute a new feature, consider submitting an Issue to discuss the feature so the community can weigh in and assist.
|
||||
|
||||
### API Design Principles
|
||||
|
||||
Supervision APIs should remain generic, composable, and predictable across model
|
||||
families. Before adding a new integration, annotator option, or data conversion
|
||||
method, check the existing `sv.Detections`, `sv.KeyPoints`, and annotator
|
||||
patterns and follow these principles:
|
||||
|
||||
1. **Model integrations normalize raw external outputs into existing Supervision
|
||||
containers.** Use `sv.Detections` for detection, segmentation, and other
|
||||
instance-level predictions that include boxes, masks, class ids, confidence
|
||||
scores, or extra per-instance fields. Use `sv.KeyPoints` for standalone
|
||||
keypoint or pose predictions when keypoints exist independently of detection
|
||||
boxes (e.g. pure pose estimation, landmark detection on pre-cropped images).
|
||||
Use `Detections.keypoints` when keypoints are always co-incident with boxes
|
||||
from the same model — the field stores an `(n, K, 2)` or `(n, K, 3)` array
|
||||
where the optional third channel is per-point confidence in `[0, 1]`.
|
||||
2. **Do not add a `from_<model>` method when the model already returns a
|
||||
Supervision object.** `from_*` methods are for converting raw outputs from
|
||||
external packages such as Ultralytics, Transformers, Inference, or MediaPipe.
|
||||
If a model's `predict()` method already returns `sv.Detections`, keep that
|
||||
result type and store additional structured payloads in `detections.data` or
|
||||
`detections.metadata` using documented keys.
|
||||
3. **Annotators render data; filtering and visibility are container state.**
|
||||
Filtering by confidence, class id, tracker id, geometry, or custom data should
|
||||
happen before annotation through the container slicing APIs, for example
|
||||
`detections[detections.confidence > 0.7]` or `key_points[key_points.confidence > 0.5]`.
|
||||
Per-point presentation state, such as a `KeyPoints.visible` mask, may
|
||||
live on the container and be honored consistently by annotators.
|
||||
4. **Annotator constructor arguments should describe visual presentation, not
|
||||
model-quality gates.** Use constructor arguments for color, thickness,
|
||||
opacity, text, position, style, and generic visualization parameters such as
|
||||
sigma levels. Annotators may skip invalid geometry defensively, including
|
||||
missing points, zero-area boxes, non-finite coordinates, or points marked
|
||||
invisible on the container. They should not introduce confidence thresholds or
|
||||
model-specific quality gates as rendering options.
|
||||
|
||||
## How to Contribute Changes
|
||||
|
||||
First, fork this repository to your own GitHub account. Click "fork" in the top corner of the `supervision` repository to get started:
|
||||
|
|
|
|||
12
AGENTS.md
12
AGENTS.md
|
|
@ -46,6 +46,18 @@ All work must follow the conventions of the `supervision` library
|
|||
- Follow existing naming patterns.
|
||||
- Maintain backward compatibility unless explicitly allowed.
|
||||
- Prefer functional utilities over complex classes unless justified.
|
||||
- Treat [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md#api-design-principles)
|
||||
as the canonical API design reference.
|
||||
- Keep model integrations aligned with existing containers: models that already
|
||||
return `sv.Detections` should continue to do so, with extra payloads stored
|
||||
in `data` or `metadata` under documented keys.
|
||||
- Reserve `from_*` methods for converting raw outputs from external packages
|
||||
into Supervision containers; do not add one-off adapters for outputs that are
|
||||
already `sv.Detections` or `sv.KeyPoints`.
|
||||
- Annotators render already-selected data. Do filtering by confidence, class id,
|
||||
tracker id, geometry, or custom fields before annotation with container
|
||||
slicing APIs, not annotator constructor arguments. Container-level visibility
|
||||
masks may be honored by annotators when documented consistently.
|
||||
|
||||
### Performance
|
||||
|
||||
|
|
|
|||
|
|
@ -135,6 +135,13 @@ class Detections:
|
|||
mask: An array of shape `(n, H, W)` containing the segmentation masks
|
||||
(`bool` data type), or `None` when masks are not available, or as
|
||||
:class:`~supervision.detection.compact_mask.CompactMask`.
|
||||
keypoints: An array of shape `(n, K, 2)` or `(n, K, 3)` containing
|
||||
keypoint coordinates for each detection, or `None` when keypoints
|
||||
are not available. `K` is the number of keypoints per detection (e.g.
|
||||
17 for COCO pose). The optional third channel is a per-point confidence
|
||||
score in `[0, 1]` (float32). Use `sv.KeyPoints` for standalone pose
|
||||
predictions without associated detection boxes; use this field when
|
||||
keypoints are always co-incident with boxes from the same model.
|
||||
confidence: An array of shape `(n,)` containing the confidence scores
|
||||
of the detections, or `None` when confidence values are not available.
|
||||
class_id: An array of shape `(n,)` containing the class ids of the
|
||||
|
|
@ -156,6 +163,7 @@ class Detections:
|
|||
tracker_id: npt.NDArray[np.generic] | None = None
|
||||
data: dict[str, npt.NDArray[np.generic] | list[Any]] = field(default_factory=dict)
|
||||
metadata: dict[str, Any] = field(default_factory=dict)
|
||||
keypoints: npt.NDArray[np.generic] | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
validate_detections_fields(
|
||||
|
|
@ -165,6 +173,7 @@ class Detections:
|
|||
class_id=self.class_id,
|
||||
tracker_id=self.tracker_id,
|
||||
data=self.data,
|
||||
keypoints=self.keypoints,
|
||||
)
|
||||
|
||||
def __len__(self) -> int:
|
||||
|
|
@ -188,6 +197,11 @@ class Detections:
|
|||
"""
|
||||
Iterates over the Detections object and yield a tuple of
|
||||
`(xyxy, mask, confidence, class_id, tracker_id, data)` for each detection.
|
||||
|
||||
Note:
|
||||
The `keypoints` field is intentionally excluded from iteration to preserve
|
||||
the stable 6-tuple shape that downstream code depends on. Access keypoints
|
||||
directly via `detections.keypoints`.
|
||||
"""
|
||||
for i in range(len(self.xyxy)):
|
||||
yield (
|
||||
|
|
@ -206,6 +220,7 @@ class Detections:
|
|||
[
|
||||
np.array_equal(self.xyxy, other.xyxy),
|
||||
np.array_equal(self.mask, other.mask),
|
||||
np.array_equal(self.keypoints, other.keypoints),
|
||||
np.array_equal(self.class_id, other.class_id),
|
||||
np.array_equal(self.confidence, other.confidence),
|
||||
np.array_equal(self.tracker_id, other.tracker_id),
|
||||
|
|
@ -2109,8 +2124,8 @@ class Detections:
|
|||
Merge a list of Detections objects into a single Detections object.
|
||||
|
||||
This method takes a list of Detections objects and combines their
|
||||
respective fields (`xyxy`, `mask`, `confidence`, `class_id`, and `tracker_id`)
|
||||
into a single Detections object.
|
||||
respective fields (`xyxy`, `mask`, `keypoints`, `confidence`, `class_id`, and
|
||||
`tracker_id`) into a single Detections object.
|
||||
|
||||
For example, if merging Detections with 3 and 4 detected objects, this method
|
||||
will return a Detections with 7 objects (7 entries in `xyxy`, `mask`, etc).
|
||||
|
|
@ -2171,6 +2186,7 @@ class Detections:
|
|||
class_id=detections.class_id,
|
||||
tracker_id=detections.tracker_id,
|
||||
data=detections.data,
|
||||
keypoints=detections.keypoints,
|
||||
)
|
||||
|
||||
xyxy = np.vstack([d.xyxy for d in detections_list])
|
||||
|
|
@ -2188,9 +2204,19 @@ class Detections:
|
|||
return CompactMask.merge(masks)
|
||||
# Mixed or all-ndarray: __array__ auto-converts any CompactMask.
|
||||
return np.vstack([np.asarray(m) for m in masks])
|
||||
if name == "keypoints":
|
||||
kp_arrays = [d.__getattribute__(name) for d in detections_list]
|
||||
shapes = [a.shape[1:] for a in kp_arrays]
|
||||
if len(set(shapes)) > 1:
|
||||
raise ValueError(
|
||||
f"All 'keypoints' arrays must share the same (K, channels); "
|
||||
f"got shapes: {[a.shape for a in kp_arrays]}"
|
||||
)
|
||||
return np.vstack(kp_arrays)
|
||||
return np.hstack([d.__getattribute__(name) for d in detections_list])
|
||||
|
||||
mask = stack_or_none("mask")
|
||||
keypoints = stack_or_none("keypoints")
|
||||
confidence = stack_or_none("confidence")
|
||||
class_id = stack_or_none("class_id")
|
||||
tracker_id = stack_or_none("tracker_id")
|
||||
|
|
@ -2208,6 +2234,7 @@ class Detections:
|
|||
tracker_id=tracker_id,
|
||||
data=data,
|
||||
metadata=metadata,
|
||||
keypoints=keypoints,
|
||||
)
|
||||
|
||||
def get_anchors_coordinates(self, anchor: Position) -> npt.NDArray[np.generic]:
|
||||
|
|
@ -2322,6 +2349,7 @@ class Detections:
|
|||
tracker_id=self.tracker_id[index] if self.tracker_id is not None else None,
|
||||
data=get_data_item(self.data, index),
|
||||
metadata=self.metadata,
|
||||
keypoints=self.keypoints[index] if self.keypoints is not None else None,
|
||||
)
|
||||
|
||||
def __setitem__(self, key: str, value: npt.NDArray[np.generic] | list[Any]) -> None:
|
||||
|
|
@ -2582,7 +2610,8 @@ def merge_inner_detection_object_pair(
|
|||
The resulting `confidence` of the merged object is calculated by the weighted
|
||||
contribution of each detection to the merged object.
|
||||
The bounding boxes and masks of the two input detections are merged into a
|
||||
single bounding box and mask, respectively.
|
||||
single bounding box and mask, respectively. If keypoints are present, keypoints
|
||||
from the winning detection are preserved.
|
||||
|
||||
Args:
|
||||
detections_1: The first Detections object.
|
||||
|
|
@ -2657,6 +2686,7 @@ def merge_inner_detection_object_pair(
|
|||
tracker_id=winning_detection.tracker_id,
|
||||
data=winning_detection.data,
|
||||
metadata=metadata,
|
||||
keypoints=winning_detection.keypoints,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -208,7 +208,6 @@ class VertexEllipseAnnotator(BaseKeyPointAnnotator):
|
|||
thickness: int = 2,
|
||||
sigma: float = 2.0,
|
||||
covariance_data_key: str = "covariance",
|
||||
confidence_threshold: float = 0.0,
|
||||
max_axis_length: float | None = None,
|
||||
line_style: Literal["solid", "dashed"] = "solid",
|
||||
dash_length: int = 16,
|
||||
|
|
@ -220,8 +219,6 @@ class VertexEllipseAnnotator(BaseKeyPointAnnotator):
|
|||
sigma: Number of standard deviations represented by the ellipse axes.
|
||||
covariance_data_key: Key in ``key_points.data`` containing covariance
|
||||
matrices with shape ``(N, K, 2, 2)``.
|
||||
confidence_threshold: Minimum keypoint confidence required for drawing.
|
||||
Ignored when ``key_points.confidence`` is ``None``.
|
||||
max_axis_length: Optional cap for ellipse semi-axis lengths in pixels.
|
||||
When ``None`` (default), near-singular precision matrices can produce
|
||||
extremely large eigenvalues and frame-spanning ellipses. Set this to
|
||||
|
|
@ -247,7 +244,6 @@ class VertexEllipseAnnotator(BaseKeyPointAnnotator):
|
|||
self.thickness = thickness
|
||||
self.sigma = sigma
|
||||
self.covariance_data_key = covariance_data_key
|
||||
self.confidence_threshold = confidence_threshold
|
||||
self.max_axis_length = max_axis_length
|
||||
self.line_style = line_style
|
||||
self.dash_length = dash_length
|
||||
|
|
@ -300,8 +296,6 @@ class VertexEllipseAnnotator(BaseKeyPointAnnotator):
|
|||
confidence = key_points.confidence[detection_index, point_index]
|
||||
if not np.isfinite(confidence):
|
||||
continue
|
||||
if confidence < self.confidence_threshold:
|
||||
continue
|
||||
ellipse = self._covariance_to_ellipse(
|
||||
covariance=covariances[detection_index, point_index]
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Iterable, Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Union, cast
|
||||
|
|
@ -11,9 +10,10 @@ import numpy.typing as npt
|
|||
from supervision.config import CLASS_NAME_DATA_FIELD
|
||||
from supervision.detection.core import Detections
|
||||
from supervision.detection.utils.internal import get_data_item, is_data_equal
|
||||
from supervision.validators import validate_key_points_fields
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
from supervision.validators import (
|
||||
validate_detection_keypoints,
|
||||
validate_key_points_fields,
|
||||
)
|
||||
|
||||
Index1D = Union[
|
||||
int,
|
||||
|
|
@ -26,94 +26,6 @@ Index1D = Union[
|
|||
Index2D = tuple[Index1D, Index1D]
|
||||
|
||||
|
||||
def _rfdetr_source_shape(
|
||||
rfdetr_detections: Detections,
|
||||
detections_count: int,
|
||||
) -> npt.NDArray[np.float32]:
|
||||
source_shape = rfdetr_detections.data.get("source_shape")
|
||||
if source_shape is None:
|
||||
raise ValueError(
|
||||
"RF-DETR detections with keypoint precision data must contain "
|
||||
"data['source_shape'] with shape (N, 2) where each row is "
|
||||
"(height, width) in pixels."
|
||||
)
|
||||
|
||||
source_shape_array = np.asarray(source_shape, dtype=np.float32)
|
||||
expected_shape = (detections_count, 2)
|
||||
if source_shape_array.shape != expected_shape:
|
||||
raise ValueError(
|
||||
"Expected RF-DETR source_shape shape "
|
||||
f"{expected_shape}, got {source_shape_array.shape}."
|
||||
)
|
||||
return source_shape_array
|
||||
|
||||
|
||||
def _rfdetr_precision_cholesky_to_pixel_covariance(
|
||||
precision_cholesky: npt.NDArray[np.float32],
|
||||
source_shape: npt.NDArray[np.float32],
|
||||
) -> npt.NDArray[np.float32]:
|
||||
if precision_cholesky.ndim != 3 or precision_cholesky.shape[2] != 3:
|
||||
raise ValueError(
|
||||
"Expected RF-DETR keypoint precision shape (N, K, 3), "
|
||||
f"got {precision_cholesky.shape}."
|
||||
)
|
||||
if precision_cholesky.shape[0] != source_shape.shape[0]:
|
||||
raise ValueError(
|
||||
"RF-DETR keypoint precision and source_shape must contain the same "
|
||||
"number of detections, got "
|
||||
f"{precision_cholesky.shape[0]} and {source_shape.shape[0]}."
|
||||
)
|
||||
|
||||
n_total = precision_cholesky.shape[0] * precision_cholesky.shape[1]
|
||||
n_non_finite = 0
|
||||
n_singular = 0
|
||||
n_overflow = 0
|
||||
|
||||
covariances = np.full(
|
||||
(*precision_cholesky.shape[:2], 2, 2), np.nan, dtype=np.float32
|
||||
)
|
||||
for detection_index, detection_precision in enumerate(precision_cholesky):
|
||||
height, width = source_shape[detection_index]
|
||||
scale = np.diag([width, height]).astype(np.float64)
|
||||
for keypoint_index, params in enumerate(detection_precision):
|
||||
if not np.isfinite(params).all():
|
||||
n_non_finite += 1
|
||||
continue
|
||||
log_l11 = float(np.clip(params[0], -20.0, 20.0))
|
||||
l21 = float(np.clip(params[1], -1.0e4, 1.0e4))
|
||||
log_l22 = float(np.clip(params[2], -20.0, 20.0))
|
||||
l11 = float(np.exp(log_l11))
|
||||
l22 = float(np.exp(log_l22))
|
||||
precision = np.array(
|
||||
[[l11 * l11, l11 * l21], [l11 * l21, l21 * l21 + l22 * l22]],
|
||||
dtype=np.float64,
|
||||
)
|
||||
try:
|
||||
covariance = np.linalg.inv(precision)
|
||||
except np.linalg.LinAlgError:
|
||||
n_singular += 1
|
||||
continue
|
||||
|
||||
pixel_covariance = scale @ covariance @ scale
|
||||
if np.isfinite(pixel_covariance).all():
|
||||
covariances[detection_index, keypoint_index] = pixel_covariance
|
||||
else:
|
||||
n_overflow += 1
|
||||
|
||||
n_failed = n_non_finite + n_singular + n_overflow
|
||||
if n_failed > 0:
|
||||
logger.warning(
|
||||
"%d of %d precision matrices failed: "
|
||||
"non_finite=%d, singular=%d, overflow=%d",
|
||||
n_failed,
|
||||
n_total,
|
||||
n_non_finite,
|
||||
n_singular,
|
||||
n_overflow,
|
||||
)
|
||||
return covariances
|
||||
|
||||
|
||||
def _optional_array_equal(
|
||||
first: npt.NDArray[np.generic] | None,
|
||||
second: npt.NDArray[np.generic] | None,
|
||||
|
|
@ -250,13 +162,6 @@ class KeyPoints:
|
|||
key_point = sv.KeyPoints.from_transformers(results[0])
|
||||
```
|
||||
|
||||
Note:
|
||||
[`sv.KeyPoints.from_rfdetr`][supervision.key_points.core.KeyPoints.from_rfdetr]
|
||||
accepts ``sv.Detections`` (not native RF-DETR output) because RF-DETR keypoints
|
||||
are attached as extra fields inside a ``sv.Detections`` object returned by
|
||||
``model.predict()``. Run that conversion first, then pass the result to
|
||||
``from_rfdetr``.
|
||||
|
||||
Attributes:
|
||||
xy: An array of shape `(n, m, 2)` containing
|
||||
`n` detected objects, each composed of `m` equally-sized
|
||||
|
|
@ -338,111 +243,6 @@ class KeyPoints:
|
|||
]
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_rfdetr(cls, rfdetr_detections: Detections) -> KeyPoints:
|
||||
"""
|
||||
Create a `sv.KeyPoints` object from RF-DETR `sv.Detections` output.
|
||||
|
||||
RF-DETR attaches keypoint coordinates to ``detections.data["keypoints"]``
|
||||
with shape ``(N, K, 3)`` where the last dimension stores ``[x, y,
|
||||
confidence]`` in pixel coordinates. When RF-DETR also provides
|
||||
``detections.data["keypoint_precision_cholesky"]``, this method converts
|
||||
those per-keypoint precision parameters into pixel-space covariance matrices
|
||||
and stores them in ``key_points.data["covariance"]`` for use with
|
||||
`sv.VertexEllipseAnnotator`.
|
||||
|
||||
Note:
|
||||
``detections.data["source_shape"]`` must have shape ``(N, 2)`` where each
|
||||
row is ``(height, width)`` in pixels — note this is HW order, not the WH
|
||||
order used by ``resolution_wh`` elsewhere in supervision.
|
||||
|
||||
Keypoint confidence values are stored as-is from RF-DETR output and are
|
||||
expected to be probabilities in the range ``[0, 1]``. If RF-DETR returns
|
||||
logits instead, user-supplied ``confidence_threshold`` values in
|
||||
`sv.VertexEllipseAnnotator` should be adjusted accordingly.
|
||||
|
||||
Args:
|
||||
rfdetr_detections: RF-DETR prediction returned by ``model.predict()``.
|
||||
|
||||
Returns:
|
||||
A `sv.KeyPoints` object containing RF-DETR keypoints and optional
|
||||
covariance matrices.
|
||||
|
||||
Raises:
|
||||
ValueError: If the RF-DETR detections do not contain valid keypoints,
|
||||
or if precision parameters are present without source shape data.
|
||||
|
||||
Examples:
|
||||
Basic usage — keypoints only:
|
||||
|
||||
>>> import numpy as np
|
||||
>>> import supervision as sv
|
||||
>>> kp_arr = np.array([[[50, 80, 0.9], [60, 90, 0.8]]], dtype=np.float32)
|
||||
>>> detections = sv.Detections(
|
||||
... xyxy=np.array([[10, 20, 100, 200]], dtype=np.float32),
|
||||
... data={"keypoints": kp_arr},
|
||||
... )
|
||||
>>> key_points = sv.KeyPoints.from_rfdetr(detections)
|
||||
>>> key_points.xy.shape
|
||||
(1, 2, 2)
|
||||
|
||||
With precision Cholesky parameters (produces covariance data):
|
||||
|
||||
>>> kp_arr2 = np.array([[[50, 80, 0.9], [60, 90, 0.8]]], dtype=np.float32)
|
||||
>>> chol = np.zeros((1, 2, 3), dtype=np.float32)
|
||||
>>> src = np.array([[480, 640]], dtype=np.float32)
|
||||
>>> detections_with_cov = sv.Detections(
|
||||
... xyxy=np.array([[10, 20, 100, 200]], dtype=np.float32),
|
||||
... data={
|
||||
... "keypoints": kp_arr2,
|
||||
... "keypoint_precision_cholesky": chol,
|
||||
... "source_shape": src,
|
||||
... },
|
||||
... )
|
||||
>>> kp = sv.KeyPoints.from_rfdetr(detections_with_cov)
|
||||
>>> "covariance" in kp.data
|
||||
True
|
||||
"""
|
||||
rfdetr_keypoints = rfdetr_detections.data.get("keypoints")
|
||||
if rfdetr_keypoints is None:
|
||||
raise ValueError("RF-DETR detections must contain data['keypoints'].")
|
||||
|
||||
keypoints = np.asarray(rfdetr_keypoints, dtype=np.float32)
|
||||
if keypoints.ndim != 3 or keypoints.shape[2] != 3:
|
||||
raise ValueError(
|
||||
f"Expected RF-DETR keypoints shape (N, K, 3), got {keypoints.shape}."
|
||||
)
|
||||
if keypoints.shape[0] == 0:
|
||||
return cls.empty()
|
||||
|
||||
data: dict[str, npt.NDArray[np.generic] | list[Any]] = {}
|
||||
precision_cholesky = rfdetr_detections.data.get("keypoint_precision_cholesky")
|
||||
if precision_cholesky is not None:
|
||||
precision_cholesky_array = np.asarray(precision_cholesky, dtype=np.float32)
|
||||
if precision_cholesky_array.shape[:2] != keypoints.shape[:2]:
|
||||
raise ValueError(
|
||||
"keypoint_precision_cholesky shape "
|
||||
f"{precision_cholesky_array.shape[:2]} does not match "
|
||||
f"keypoints shape {keypoints.shape[:2]}."
|
||||
)
|
||||
source_shape = _rfdetr_source_shape(
|
||||
rfdetr_detections, detections_count=keypoints.shape[0]
|
||||
)
|
||||
data["covariance"] = _rfdetr_precision_cholesky_to_pixel_covariance(
|
||||
precision_cholesky=precision_cholesky_array,
|
||||
source_shape=source_shape,
|
||||
)
|
||||
class_id: npt.NDArray[np.int_] | None = None
|
||||
if rfdetr_detections.class_id is not None:
|
||||
class_id = rfdetr_detections.class_id.astype(np.int_)
|
||||
|
||||
return cls(
|
||||
xy=keypoints[:, :, :2].astype(np.float32),
|
||||
confidence=keypoints[:, :, 2].astype(np.float32),
|
||||
class_id=class_id,
|
||||
data=data,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_inference(cls, inference_result: Any) -> KeyPoints:
|
||||
"""
|
||||
|
|
@ -1065,6 +865,60 @@ class KeyPoints:
|
|||
|
||||
self.data[key] = value
|
||||
|
||||
@classmethod
|
||||
def from_detections(cls, detections: Detections) -> KeyPoints:
|
||||
"""Convert a `sv.Detections` object to `sv.KeyPoints` using its keypoints field.
|
||||
|
||||
Use this adapter when passing `Detections.keypoints` to keypoint annotators
|
||||
such as `sv.VertexAnnotator`, `sv.EdgeAnnotator`, or `sv.VertexEllipseAnnotator`
|
||||
which accept `sv.KeyPoints` rather than raw NumPy arrays.
|
||||
|
||||
Args:
|
||||
detections: A `sv.Detections` object with a non-``None`` ``keypoints``
|
||||
field of shape ``(n, K, 2)`` or ``(n, K, 3)``.
|
||||
|
||||
Returns:
|
||||
A `sv.KeyPoints` instance. When the third channel is present it is
|
||||
interpreted as per-point confidence and stored in ``confidence``.
|
||||
|
||||
Raises:
|
||||
ValueError: If ``detections.keypoints`` is ``None``.
|
||||
|
||||
Examples:
|
||||
```pycon
|
||||
>>> import numpy as np
|
||||
>>> import supervision as sv
|
||||
>>> detections = sv.Detections(
|
||||
... xyxy=np.array([[10, 20, 30, 40]], dtype=np.float32),
|
||||
... keypoints=np.zeros((1, 17, 2), dtype=np.float32),
|
||||
... )
|
||||
>>> key_points = sv.KeyPoints.from_detections(detections)
|
||||
>>> key_points.xy.shape
|
||||
(1, 17, 2)
|
||||
|
||||
```
|
||||
"""
|
||||
if detections.keypoints is None:
|
||||
raise ValueError(
|
||||
"detections.keypoints is None; cannot convert to KeyPoints"
|
||||
)
|
||||
kp = detections.keypoints
|
||||
validate_detection_keypoints(kp, len(detections))
|
||||
if kp.shape[2] == 3:
|
||||
xy = kp[..., :2].astype(np.float32)
|
||||
confidence = kp[..., 2].astype(np.float32)
|
||||
else:
|
||||
xy = kp.astype(np.float32)
|
||||
confidence = None
|
||||
class_id: npt.NDArray[np.int_] | None = None
|
||||
if detections.class_id is not None:
|
||||
class_id = detections.class_id.astype(np.int_)
|
||||
return cls(
|
||||
xy=xy,
|
||||
confidence=confidence,
|
||||
class_id=class_id,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def empty(cls) -> KeyPoints:
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
from typing import Any
|
||||
from typing import Any, Optional
|
||||
|
||||
import numpy as np
|
||||
from deprecate import deprecated, void
|
||||
|
|
@ -63,6 +63,51 @@ def validate_mask(mask: Any, n: int) -> None:
|
|||
)
|
||||
|
||||
|
||||
def validate_detection_keypoints(keypoints: Any, n: int) -> None:
|
||||
"""Validate that keypoints is a numeric 3D array with shape (n, K, 2) or (n, K, 3).
|
||||
|
||||
The optional third channel encodes per-point confidence scores in ``[0, 1]``.
|
||||
Pass ``None`` when keypoints are absent; any other value must be a numeric
|
||||
``np.ndarray``.
|
||||
|
||||
Args:
|
||||
keypoints: The keypoints array to validate, or ``None``.
|
||||
n: Expected number of detections (first dimension of the array).
|
||||
|
||||
Raises:
|
||||
ValueError: If ``keypoints`` is not ``None`` and does not satisfy the shape
|
||||
or dtype constraints described above.
|
||||
|
||||
Examples:
|
||||
```pycon
|
||||
>>> import numpy as np
|
||||
>>> validate_detection_keypoints(None, 3)
|
||||
>>> validate_detection_keypoints(np.zeros((3, 17, 2), dtype=np.float32), 3)
|
||||
|
||||
```
|
||||
"""
|
||||
if keypoints is None:
|
||||
return
|
||||
expected_shape = f"({n}, K, 2) or ({n}, K, 3)"
|
||||
if not isinstance(keypoints, np.ndarray):
|
||||
raise ValueError(
|
||||
"keypoints must be a 3D np.ndarray with shape "
|
||||
+ f"{expected_shape}, but got {type(keypoints).__name__}"
|
||||
)
|
||||
if not np.issubdtype(keypoints.dtype, np.number):
|
||||
raise ValueError(
|
||||
f"keypoints must have a numeric dtype, but got dtype {keypoints.dtype}"
|
||||
)
|
||||
try:
|
||||
validate_xy(keypoints, n)
|
||||
except ValueError:
|
||||
actual_shape = str(keypoints.shape)
|
||||
raise ValueError(
|
||||
"keypoints must be a 3D np.ndarray with shape "
|
||||
+ f"{expected_shape}, but got shape {actual_shape}"
|
||||
)
|
||||
|
||||
|
||||
def validate_class_id(class_id: Any, n: int) -> None:
|
||||
expected_shape = f"({n},)"
|
||||
actual_shape = str(getattr(class_id, "shape", None))
|
||||
|
|
@ -140,16 +185,26 @@ def validate_data(data: dict[str, Any], n: int) -> None:
|
|||
raise ValueError(f"Value for key '{key}' must be a list or np.ndarray")
|
||||
|
||||
|
||||
def validate_xy(xy: Any, n: int, m: int) -> None:
|
||||
expected_shape = f"({n, m},)"
|
||||
def validate_xy(xy: Any, n: int, m: Optional[int] = None) -> None:
|
||||
actual_shape = str(getattr(xy, "shape", None))
|
||||
|
||||
is_valid = isinstance(xy, np.ndarray) and (
|
||||
xy.shape == (n, m, 2) or xy.shape == (n, m, 3)
|
||||
)
|
||||
if m is None:
|
||||
is_valid = (
|
||||
isinstance(xy, np.ndarray)
|
||||
and xy.ndim == 3
|
||||
and xy.shape[0] == n
|
||||
and xy.shape[2] in (2, 3)
|
||||
)
|
||||
expected_shape = f"({n}, K, 2) or ({n}, K, 3)"
|
||||
else:
|
||||
is_valid = isinstance(xy, np.ndarray) and (
|
||||
xy.shape == (n, m, 2) or xy.shape == (n, m, 3)
|
||||
)
|
||||
expected_shape = f"({n}, {m}, 2) or ({n}, {m}, 3)"
|
||||
|
||||
if not is_valid:
|
||||
raise ValueError(
|
||||
f"xy must be a 2D np.ndarray with shape {expected_shape}, but got shape "
|
||||
f"xy must be a 3D np.ndarray with shape {expected_shape}, but got shape "
|
||||
f"{actual_shape}"
|
||||
)
|
||||
|
||||
|
|
@ -161,10 +216,12 @@ def validate_detections_fields(
|
|||
confidence: Any,
|
||||
tracker_id: Any,
|
||||
data: dict[str, Any],
|
||||
keypoints: Any = None,
|
||||
) -> None:
|
||||
validate_xyxy(xyxy)
|
||||
n = len(xyxy)
|
||||
validate_mask(mask, n)
|
||||
validate_detection_keypoints(keypoints, n)
|
||||
validate_class_id(class_id, n)
|
||||
validate_confidence(confidence, n)
|
||||
validate_tracker_id(tracker_id, n)
|
||||
|
|
|
|||
|
|
@ -155,6 +155,46 @@ def test_detections_non_bool_mask_warns_with_migration_path() -> None:
|
|||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("keypoints", "exception"),
|
||||
[
|
||||
(np.array([[[1, 2], [3, 4]]], dtype=np.float32), DoesNotRaise()),
|
||||
(np.array([[[1, 2, 0.9], [3, 4, 0.8]]], dtype=np.float32), DoesNotRaise()),
|
||||
(
|
||||
np.array([[1, 2, 0.9], [3, 4, 0.8]], dtype=np.float32),
|
||||
pytest.raises(ValueError, match=r"keypoints must be a 3D np.ndarray"),
|
||||
),
|
||||
(
|
||||
np.array([[[1, 2, 0.9, 1], [3, 4, 0.8, 1]]], dtype=np.float32),
|
||||
pytest.raises(ValueError, match=r"keypoints must be a 3D np.ndarray"),
|
||||
),
|
||||
(
|
||||
np.array(
|
||||
[
|
||||
[[1, 2, 0.9], [3, 4, 0.8]],
|
||||
[[5, 6, 0.7], [7, 8, 0.6]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
),
|
||||
pytest.raises(ValueError, match=r"keypoints must be a 3D np.ndarray"),
|
||||
),
|
||||
(
|
||||
np.array([[["a", "b"]]], dtype=object),
|
||||
pytest.raises(ValueError, match=r"keypoints must have a numeric dtype"),
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_detections_keypoints_validation(
|
||||
keypoints: np.ndarray, exception: Exception
|
||||
) -> None:
|
||||
"""Validate that Detections rejects invalid keypoints arrays."""
|
||||
with exception:
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=keypoints,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("detections", "index", "expected_result", "exception"),
|
||||
[
|
||||
|
|
@ -304,6 +344,164 @@ def test_getitem(
|
|||
assert result == expected_result
|
||||
|
||||
|
||||
def test_getitem_preserves_keypoints() -> None:
|
||||
keypoints = np.array(
|
||||
[
|
||||
[[1, 2, 0.9], [3, 4, 0.8]],
|
||||
[[5, 6, 0.7], [7, 8, 0.6]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10], [20, 20, 30, 30]], dtype=np.float32),
|
||||
confidence=np.array([0.9, 0.8], dtype=np.float32),
|
||||
keypoints=keypoints,
|
||||
)
|
||||
|
||||
result = detections[[1]]
|
||||
|
||||
assert isinstance(result, Detections)
|
||||
np.testing.assert_array_equal(result.keypoints, keypoints[[1]])
|
||||
|
||||
|
||||
def test_merge_preserves_keypoints() -> None:
|
||||
detections_1 = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=np.array([[[1, 2, 0.9], [3, 4, 0.8]]], dtype=np.float32),
|
||||
)
|
||||
detections_2 = Detections(
|
||||
xyxy=np.array([[20, 20, 30, 30]], dtype=np.float32),
|
||||
keypoints=np.array([[[5, 6, 0.7], [7, 8, 0.6]]], dtype=np.float32),
|
||||
)
|
||||
|
||||
result = Detections.merge([detections_1, detections_2])
|
||||
|
||||
np.testing.assert_array_equal(
|
||||
result.keypoints,
|
||||
np.array(
|
||||
[
|
||||
[[1, 2, 0.9], [3, 4, 0.8]],
|
||||
[[5, 6, 0.7], [7, 8, 0.6]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_merge_rejects_mixed_keypoints_availability() -> None:
|
||||
"""Merging detections where only some have keypoints raises ValueError."""
|
||||
detections_1 = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=np.array([[[1, 2, 0.9], [3, 4, 0.8]]], dtype=np.float32),
|
||||
)
|
||||
detections_2 = Detections(
|
||||
xyxy=np.array([[20, 20, 30, 30]], dtype=np.float32),
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="All or none of the 'keypoints'"):
|
||||
Detections.merge([detections_1, detections_2])
|
||||
|
||||
|
||||
def test_getitem_with_none_keypoints() -> None:
|
||||
"""Integer-index slicing when keypoints=None returns None keypoints."""
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10], [20, 20, 30, 30]], dtype=np.float32),
|
||||
)
|
||||
|
||||
result = detections[[0]]
|
||||
|
||||
assert result.keypoints is None
|
||||
|
||||
|
||||
def test_getitem_preserves_keypoints_boolean_mask() -> None:
|
||||
"""Boolean-mask indexing propagates the selected keypoints rows."""
|
||||
keypoints = np.array(
|
||||
[[[1, 2, 0.9], [3, 4, 0.8]], [[5, 6, 0.7], [7, 8, 0.6]]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10], [20, 20, 30, 30]], dtype=np.float32),
|
||||
keypoints=keypoints,
|
||||
)
|
||||
|
||||
result = detections[np.array([True, False])]
|
||||
|
||||
np.testing.assert_array_equal(result.keypoints, keypoints[[0]])
|
||||
|
||||
|
||||
def test_getitem_preserves_keypoints_slice() -> None:
|
||||
"""Slice indexing propagates the selected keypoints rows."""
|
||||
keypoints = np.array(
|
||||
[[[1, 2, 0.9], [3, 4, 0.8]], [[5, 6, 0.7], [7, 8, 0.6]]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10], [20, 20, 30, 30]], dtype=np.float32),
|
||||
keypoints=keypoints,
|
||||
)
|
||||
|
||||
result = detections[1:]
|
||||
|
||||
np.testing.assert_array_equal(result.keypoints, keypoints[1:])
|
||||
|
||||
|
||||
def test_merge_preserves_keypoints_no_confidence() -> None:
|
||||
"""Merging (N, K, 2) keypoints (no confidence channel) concatenates correctly."""
|
||||
detections_1 = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=np.array([[[1, 2], [3, 4]]], dtype=np.float32),
|
||||
)
|
||||
detections_2 = Detections(
|
||||
xyxy=np.array([[20, 20, 30, 30]], dtype=np.float32),
|
||||
keypoints=np.array([[[5, 6], [7, 8]]], dtype=np.float32),
|
||||
)
|
||||
|
||||
result = Detections.merge([detections_1, detections_2])
|
||||
|
||||
np.testing.assert_array_equal(
|
||||
result.keypoints,
|
||||
np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]], dtype=np.float32),
|
||||
)
|
||||
|
||||
|
||||
def test_merge_all_none_keypoints() -> None:
|
||||
"""Merging detections where all keypoints are None yields None keypoints."""
|
||||
detections_1 = Detections(xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32))
|
||||
detections_2 = Detections(xyxy=np.array([[20, 20, 30, 30]], dtype=np.float32))
|
||||
|
||||
result = Detections.merge([detections_1, detections_2])
|
||||
|
||||
assert result.keypoints is None
|
||||
|
||||
|
||||
def test_merge_three_way_preserves_keypoints() -> None:
|
||||
"""Three-way merge concatenates keypoints from all detections in order."""
|
||||
kp1 = np.array([[[1, 2, 0.9]]], dtype=np.float32)
|
||||
kp2 = np.array([[[3, 4, 0.8]]], dtype=np.float32)
|
||||
kp3 = np.array([[[5, 6, 0.7]]], dtype=np.float32)
|
||||
detections_1 = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32), keypoints=kp1
|
||||
)
|
||||
detections_2 = Detections(
|
||||
xyxy=np.array([[20, 20, 30, 30]], dtype=np.float32), keypoints=kp2
|
||||
)
|
||||
detections_3 = Detections(
|
||||
xyxy=np.array([[40, 40, 50, 50]], dtype=np.float32), keypoints=kp3
|
||||
)
|
||||
|
||||
result = Detections.merge([detections_1, detections_2, detections_3])
|
||||
|
||||
np.testing.assert_array_equal(
|
||||
result.keypoints,
|
||||
np.array([[[1, 2, 0.9]], [[3, 4, 0.8]], [[5, 6, 0.7]]], dtype=np.float32),
|
||||
)
|
||||
|
||||
|
||||
def test_empty_detections_keypoints_is_none() -> None:
|
||||
"""Detections.empty() must have keypoints=None."""
|
||||
assert Detections.empty().keypoints is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("detections_list", "expected_result", "exception"),
|
||||
[
|
||||
|
|
@ -703,11 +901,42 @@ def test_get_anchor_coordinates(
|
|||
_create_detections(xyxy=[[10, 10, 20, 20]], data={"test_1": [3]}),
|
||||
False,
|
||||
), # detections with xyxy, and different data field values
|
||||
(
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32),
|
||||
keypoints=np.array([[[1.0, 2.0]]], dtype=np.float32),
|
||||
),
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32),
|
||||
keypoints=np.array([[[1.0, 2.0]]], dtype=np.float32),
|
||||
),
|
||||
True,
|
||||
), # equal non-None keypoints
|
||||
(
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32),
|
||||
keypoints=np.array([[[1.0, 2.0]]], dtype=np.float32),
|
||||
),
|
||||
Detections(xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32)),
|
||||
False,
|
||||
), # one has keypoints, other is None
|
||||
(
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32),
|
||||
keypoints=np.array([[[1.0, 2.0]]], dtype=np.float32),
|
||||
),
|
||||
Detections(
|
||||
xyxy=np.array([[0, 0, 1, 1]], dtype=np.float32),
|
||||
keypoints=np.array([[[9.0, 9.0]]], dtype=np.float32),
|
||||
),
|
||||
False,
|
||||
), # same shape, different keypoint values
|
||||
],
|
||||
)
|
||||
def test_equal(
|
||||
detections_a: Detections, detections_b: Detections, expected_result: bool
|
||||
) -> None:
|
||||
"""Verify Detections equality covers all fields including keypoints."""
|
||||
assert (detections_a == detections_b) == expected_result
|
||||
|
||||
|
||||
|
|
@ -880,6 +1109,25 @@ def test_merge_inner_detection_object_pair(
|
|||
assert result == expected_result
|
||||
|
||||
|
||||
def test_merge_inner_detection_object_pair_preserves_winning_keypoints() -> None:
|
||||
losing_keypoints = np.array([[[1, 2, 0.9], [3, 4, 0.8]]], dtype=np.float32)
|
||||
winning_keypoints = np.array([[[5, 6, 0.7], [7, 8, 0.6]]], dtype=np.float32)
|
||||
detection_1 = Detections(
|
||||
xyxy=np.array([[0, 0, 20, 20]], dtype=np.float32),
|
||||
confidence=np.array([0.1], dtype=np.float32),
|
||||
keypoints=losing_keypoints,
|
||||
)
|
||||
detection_2 = Detections(
|
||||
xyxy=np.array([[10, 10, 30, 30]], dtype=np.float32),
|
||||
confidence=np.array([0.9], dtype=np.float32),
|
||||
keypoints=winning_keypoints,
|
||||
)
|
||||
|
||||
result = merge_inner_detection_object_pair(detection_1, detection_2)
|
||||
|
||||
np.testing.assert_array_equal(result.keypoints, winning_keypoints)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("detections", "expected"),
|
||||
[
|
||||
|
|
|
|||
|
|
@ -214,7 +214,7 @@ class TestVertexEllipseAnnotator:
|
|||
"covariance": np.array([[[[25.0, 0.0], [0.0, 9.0]]]], dtype=np.float32)
|
||||
},
|
||||
)
|
||||
annotator = sv.VertexEllipseAnnotator(confidence_threshold=0.0)
|
||||
annotator = sv.VertexEllipseAnnotator()
|
||||
|
||||
result = annotator.annotate(scene=scene.copy(), key_points=key_points)
|
||||
|
||||
|
|
@ -241,33 +241,32 @@ class TestVertexEllipseAnnotator:
|
|||
with pytest.raises(ValueError, match="Expected covariance shape"):
|
||||
annotator.annotate(scene=scene.copy(), key_points=sample_key_points)
|
||||
|
||||
def test_confidence_threshold_filters_low_confidence_keypoints(self, scene):
|
||||
def test_pre_masked_keypoints_are_annotated(self, scene):
|
||||
"""
|
||||
Scenario: Two keypoints with confidences 0.3 and 0.7; threshold=0.5.
|
||||
Expected: Only the high-confidence keypoint is drawn.
|
||||
Scenario: Caller masks low-confidence keypoints before annotation.
|
||||
Expected: Only the already-selected keypoint is drawn.
|
||||
"""
|
||||
cov = np.array([[[[25.0, 0.0], [0.0, 9.0]]]], dtype=np.float32)
|
||||
key_points_low = sv.KeyPoints(
|
||||
xy=np.array([[[20.0, 20.0]]], dtype=np.float32),
|
||||
confidence=np.array([[0.3]], dtype=np.float32),
|
||||
data={"covariance": cov},
|
||||
key_points = sv.KeyPoints(
|
||||
xy=np.array([[[20.0, 20.0], [40.0, 40.0]]], dtype=np.float32),
|
||||
confidence=np.array([[0.3, 0.7]], dtype=np.float32),
|
||||
data={
|
||||
"covariance": np.tile(
|
||||
np.array([[[[25.0, 0.0], [0.0, 9.0]]]], dtype=np.float32),
|
||||
(1, 2, 1, 1),
|
||||
)
|
||||
},
|
||||
)
|
||||
key_points_high = sv.KeyPoints(
|
||||
xy=np.array([[[20.0, 20.0]]], dtype=np.float32),
|
||||
confidence=np.array([[0.7]], dtype=np.float32),
|
||||
data={"covariance": cov},
|
||||
)
|
||||
annotator = sv.VertexEllipseAnnotator(confidence_threshold=0.5)
|
||||
key_points.xy[key_points.confidence < 0.5] = 0.0
|
||||
annotator = sv.VertexEllipseAnnotator()
|
||||
|
||||
result_low = annotator.annotate(scene=scene.copy(), key_points=key_points_low)
|
||||
result_high = annotator.annotate(scene=scene.copy(), key_points=key_points_high)
|
||||
result = annotator.annotate(scene=scene.copy(), key_points=key_points)
|
||||
|
||||
assert np.array_equal(result_low, scene), (
|
||||
"low-confidence keypoint must be skipped"
|
||||
)
|
||||
assert not np.array_equal(result_high, scene), (
|
||||
"high-confidence keypoint must be drawn"
|
||||
np.testing.assert_array_equal(
|
||||
key_points.xy[0, 0], np.array([0.0, 0.0], dtype=np.float32)
|
||||
)
|
||||
assert not np.array_equal(result, scene)
|
||||
# The masked keypoint was moved to (0,0) but must not be drawn there.
|
||||
np.testing.assert_array_equal(result[:10, :10], scene[:10, :10])
|
||||
|
||||
def test_max_axis_length_caps_large_eigenvalue(self, scene):
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -3,7 +3,6 @@ from contextlib import nullcontext as DoesNotRaise
|
|||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from supervision.detection.core import Detections
|
||||
from supervision.key_points.core import KeyPoints
|
||||
from tests.helpers import (
|
||||
_create_key_points,
|
||||
|
|
@ -14,31 +13,6 @@ from tests.helpers import (
|
|||
_FakeYoloNasKeyPointResults,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def rfdetr_detections() -> Detections:
|
||||
keypoints = np.array(
|
||||
[
|
||||
[[10.0, 20.0, 0.9], [30.0, 40.0, 0.8]],
|
||||
[[50.0, 60.0, 0.7], [70.0, 80.0, 0.6]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
precision_cholesky = np.zeros((2, 2, 3), dtype=np.float32)
|
||||
return Detections(
|
||||
xyxy=np.array(
|
||||
[[0.0, 0.0, 40.0, 50.0], [10.0, 20.0, 90.0, 100.0]], dtype=np.float32
|
||||
),
|
||||
confidence=np.array([0.95, 0.85], dtype=np.float32),
|
||||
class_id=np.array([1, 1]),
|
||||
data={
|
||||
"keypoints": keypoints,
|
||||
"keypoint_precision_cholesky": precision_cholesky,
|
||||
"source_shape": np.array([[100, 200], [50, 100]], dtype=np.int64),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
KEY_POINTS = _create_key_points(
|
||||
xy=[
|
||||
[[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]],
|
||||
|
|
@ -54,73 +28,6 @@ KEY_POINTS = _create_key_points(
|
|||
)
|
||||
|
||||
|
||||
def test_key_points_from_rfdetr_loads_keypoints_and_covariance(
|
||||
rfdetr_detections: Detections,
|
||||
) -> None:
|
||||
key_points = KeyPoints.from_rfdetr(rfdetr_detections)
|
||||
|
||||
assert key_points.xy.shape == (2, 2, 2)
|
||||
np.testing.assert_allclose(
|
||||
key_points.xy, rfdetr_detections.data["keypoints"][:, :, :2]
|
||||
)
|
||||
np.testing.assert_allclose(
|
||||
key_points.confidence, rfdetr_detections.data["keypoints"][:, :, 2]
|
||||
)
|
||||
np.testing.assert_array_equal(key_points.class_id, rfdetr_detections.class_id)
|
||||
assert "covariance" in key_points.data
|
||||
covariance = key_points.data["covariance"]
|
||||
assert covariance.shape == (2, 2, 2, 2)
|
||||
np.testing.assert_allclose(
|
||||
covariance[0, 0], np.diag([200.0**2, 100.0**2]), rtol=1e-4, atol=1e-6
|
||||
)
|
||||
np.testing.assert_allclose(
|
||||
covariance[1, 0], np.diag([100.0**2, 50.0**2]), rtol=1e-4, atol=1e-6
|
||||
)
|
||||
|
||||
|
||||
def test_key_points_from_rfdetr_without_precision_omits_covariance(
|
||||
rfdetr_detections: Detections,
|
||||
) -> None:
|
||||
del rfdetr_detections.data["keypoint_precision_cholesky"]
|
||||
|
||||
key_points = KeyPoints.from_rfdetr(rfdetr_detections)
|
||||
|
||||
assert key_points.xy.shape == (2, 2, 2)
|
||||
assert "covariance" not in key_points.data
|
||||
|
||||
|
||||
def test_key_points_from_rfdetr_missing_keypoints_raises(
|
||||
rfdetr_detections: Detections,
|
||||
) -> None:
|
||||
del rfdetr_detections.data["keypoints"]
|
||||
|
||||
with pytest.raises(ValueError, match=r"data\['keypoints'\]"):
|
||||
KeyPoints.from_rfdetr(rfdetr_detections)
|
||||
|
||||
|
||||
def test_key_points_from_rfdetr_precision_requires_source_shape(
|
||||
rfdetr_detections: Detections,
|
||||
) -> None:
|
||||
del rfdetr_detections.data["source_shape"]
|
||||
|
||||
with pytest.raises(ValueError, match="source_shape"):
|
||||
KeyPoints.from_rfdetr(rfdetr_detections)
|
||||
|
||||
|
||||
def test_key_points_from_rfdetr_empty_keypoints_returns_empty(
|
||||
rfdetr_detections: Detections,
|
||||
) -> None:
|
||||
rfdetr_detections.xyxy = np.empty((0, 4), dtype=np.float32)
|
||||
rfdetr_detections.confidence = np.empty((0,), dtype=np.float32)
|
||||
rfdetr_detections.class_id = np.empty((0,), dtype=int)
|
||||
rfdetr_detections.data["keypoints"] = np.empty((0, 2, 3), dtype=np.float32)
|
||||
del rfdetr_detections.data["source_shape"]
|
||||
|
||||
key_points = KeyPoints.from_rfdetr(rfdetr_detections)
|
||||
|
||||
assert key_points == KeyPoints.empty()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("key_points", "index", "expected_result", "exception"),
|
||||
[
|
||||
|
|
@ -766,3 +673,56 @@ def test_from_mediapipe_input(mediapipe_results, resolution_wh, expected_key_poi
|
|||
mediapipe_results, resolution_wh=resolution_wh
|
||||
)
|
||||
assert key_points == expected_key_points
|
||||
|
||||
|
||||
class TestFromDetections:
|
||||
"""Verify KeyPoints.from_detections adapter behavior."""
|
||||
|
||||
def test_xy_only_input_no_confidence(self) -> None:
|
||||
"""(n, K, 2) keypoints: xy extracted, confidence is None."""
|
||||
from supervision.detection.core import Detections
|
||||
|
||||
kp = np.array([[[1.0, 2.0], [3.0, 4.0]]], dtype=np.float32)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=kp,
|
||||
)
|
||||
result = KeyPoints.from_detections(detections)
|
||||
np.testing.assert_array_equal(result.xy, kp)
|
||||
assert result.confidence is None
|
||||
|
||||
def test_xy_with_confidence_channel(self) -> None:
|
||||
"""(n, K, 3) keypoints: xy from first 2 channels, confidence from third."""
|
||||
from supervision.detection.core import Detections
|
||||
|
||||
kp = np.array([[[1.0, 2.0, 0.9], [3.0, 4.0, 0.7]]], dtype=np.float32)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=kp,
|
||||
)
|
||||
result = KeyPoints.from_detections(detections)
|
||||
np.testing.assert_array_equal(result.xy, kp[..., :2])
|
||||
np.testing.assert_array_equal(result.confidence, kp[..., 2])
|
||||
|
||||
def test_none_keypoints_raises_value_error(self) -> None:
|
||||
"""keypoints=None raises ValueError with a descriptive message."""
|
||||
from supervision.detection.core import Detections
|
||||
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
)
|
||||
with pytest.raises(ValueError, match="keypoints is None"):
|
||||
KeyPoints.from_detections(detections)
|
||||
|
||||
def test_class_id_propagated(self) -> None:
|
||||
"""class_id from Detections is forwarded to the KeyPoints object."""
|
||||
from supervision.detection.core import Detections
|
||||
|
||||
kp = np.array([[[1.0, 2.0], [3.0, 4.0]]], dtype=np.float32)
|
||||
detections = Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
keypoints=kp,
|
||||
class_id=np.array([42], dtype=int),
|
||||
)
|
||||
result = KeyPoints.from_detections(detections)
|
||||
np.testing.assert_array_equal(result.class_id, np.array([42]))
|
||||
|
|
|
|||
|
|
@ -1,73 +0,0 @@
|
|||
import numpy as np
|
||||
import pytest
|
||||
|
||||
import supervision as sv
|
||||
|
||||
|
||||
def test_keypoints_from_rfdetr_detections() -> None:
|
||||
"""Converts RF-DETR detections.data['keypoints'] into a KeyPoints object."""
|
||||
detections = sv.Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10], [10, 10, 20, 20]], dtype=np.float32),
|
||||
class_id=np.array([1, 3], dtype=int),
|
||||
data={
|
||||
"keypoints": np.array(
|
||||
[
|
||||
[[1.0, 2.0, 0.9], [3.0, 4.0, 0.8]],
|
||||
[[5.0, 6.0, 0.7], [7.0, 8.0, 0.6]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
key_points = sv.KeyPoints.from_rfdetr(detections)
|
||||
|
||||
assert key_points.xy.shape == (2, 2, 2)
|
||||
assert key_points.confidence is not None
|
||||
assert key_points.confidence.shape == (2, 2)
|
||||
assert key_points.class_id is not None
|
||||
assert np.array_equal(key_points.class_id, np.array([1, 3], dtype=int))
|
||||
|
||||
|
||||
def test_keypoints_from_rfdetr_missing_keypoints_raises_clear_error() -> None:
|
||||
"""Missing detections.data['keypoints'] raises a clear conversion error."""
|
||||
detections = sv.Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
class_id=np.array([0], dtype=int),
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match=r"data\['keypoints'\]"):
|
||||
sv.KeyPoints.from_rfdetr(detections)
|
||||
|
||||
|
||||
def test_keypoints_from_rfdetr_malformed_shape_raises_clear_error() -> None:
|
||||
"""Malformed keypoints shape raises a clear conversion error."""
|
||||
detections = sv.Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
class_id=np.array([0], dtype=int),
|
||||
data={"keypoints": np.array([[[1.0, 2.0]]], dtype=np.float32)},
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="shape \\(N, K, 3\\)"):
|
||||
sv.KeyPoints.from_rfdetr(detections)
|
||||
|
||||
|
||||
def test_keypoint_annotator_uses_vertex_and_edge_rendering() -> None:
|
||||
"""Converted RF-DETR keypoints are consumable by vertex and edge annotators."""
|
||||
scene = np.zeros((32, 32, 3), dtype=np.uint8)
|
||||
detections = sv.Detections(
|
||||
xyxy=np.array([[0, 0, 10, 10]], dtype=np.float32),
|
||||
data={
|
||||
"keypoints": np.array(
|
||||
[[[10.0, 10.0, 0.9], [20.0, 20.0, 0.8]]], dtype=np.float32
|
||||
)
|
||||
},
|
||||
)
|
||||
key_points = sv.KeyPoints.from_rfdetr(detections)
|
||||
|
||||
scene = sv.VertexAnnotator().annotate(scene=scene, key_points=key_points)
|
||||
scene = sv.EdgeAnnotator(edges=[(1, 2)]).annotate(
|
||||
scene=scene, key_points=key_points
|
||||
)
|
||||
|
||||
assert np.any(scene != 0)
|
||||
|
|
@ -121,46 +121,20 @@ class MockDataclass:
|
|||
(
|
||||
Detections.empty(),
|
||||
False,
|
||||
{
|
||||
"xyxy",
|
||||
"class_id",
|
||||
"confidence",
|
||||
"mask",
|
||||
"tracker_id",
|
||||
"data",
|
||||
"metadata",
|
||||
},
|
||||
set(Detections.__dataclass_fields__),
|
||||
DoesNotRaise(),
|
||||
),
|
||||
(
|
||||
Detections.empty(),
|
||||
True,
|
||||
{
|
||||
"xyxy",
|
||||
"class_id",
|
||||
"confidence",
|
||||
"mask",
|
||||
"tracker_id",
|
||||
"data",
|
||||
"metadata",
|
||||
"area",
|
||||
"box_area",
|
||||
"box_aspect_ratio",
|
||||
},
|
||||
set(Detections.__dataclass_fields__)
|
||||
| {"area", "box_area", "box_aspect_ratio"},
|
||||
DoesNotRaise(),
|
||||
),
|
||||
(
|
||||
Detections(xyxy=np.array([[1, 2, 3, 4]])),
|
||||
False,
|
||||
{
|
||||
"xyxy",
|
||||
"class_id",
|
||||
"confidence",
|
||||
"mask",
|
||||
"tracker_id",
|
||||
"data",
|
||||
"metadata",
|
||||
},
|
||||
set(Detections.__dataclass_fields__),
|
||||
DoesNotRaise(),
|
||||
),
|
||||
(
|
||||
|
|
@ -173,15 +147,7 @@ class MockDataclass:
|
|||
data={"key_1": [1, 2], "key_2": [3, 4]},
|
||||
),
|
||||
False,
|
||||
{
|
||||
"xyxy",
|
||||
"class_id",
|
||||
"confidence",
|
||||
"mask",
|
||||
"tracker_id",
|
||||
"data",
|
||||
"metadata",
|
||||
},
|
||||
set(Detections.__dataclass_fields__),
|
||||
DoesNotRaise(),
|
||||
),
|
||||
],
|
||||
|
|
|
|||
Loading…
Reference in New Issue