vision

ConfigurableCamera

ConfigurableCamera(**kwargs)

Bases: Camera

A generalized interface for adjusting camera parameters like exposure, brightness or fps.

camera_objects

camera_objects(
    camera_provider: CalibratableCameraProvider,
    camera_projector: CameraProjector,
    *,
    px_per_m: float = 10000,
    debug: bool = False,
    interval: float = 1.0
)

Bases: Group

This module provides a UI element for displaying cameras in a 3D scene.

It requires a camera provider as a source of cameras as well as a camera projector to show the current images projected on the ground plane. The px_per_m argument can be used to scale the camera frustums. With debug=True camera IDs are shown (default: False).

CameraProjector

CameraProjector(
    camera_provider: CalibratableCameraProvider,
    *,
    interval: float = 1.0
)

The camera projector computes a grid of projected image points on the ground plane.

It is mainly used for visualization purposes.

CameraProvider

CameraProvider()

Bases: Generic[T], PersistentModule

A camera provider holds a dictionary of cameras and manages additions and removals.

The camera dictionary should not be modified directly but by using the camera provider's methods. This way respective events are emitted and consistency can be taken care of.

The camera provider also creates an HTTP route to access camera images.

Events

Name	Description
CAMERA_ADDED	a new camera has been added (argument: camera)
CAMERA_REMOVED	a camera has been removed (argument: camera id)
NEW_IMAGE	a new image is available (argument: image)

Autoupload

Bases: Enum

Configures the auto-submitting of images to the Learning Loop

ALL class-attribute instance-attribute

ALL = 'all'

submit all images which are run through the detector

DISABLED class-attribute instance-attribute

DISABLED = 'disabled'

no auto-submitting

FILTERED class-attribute instance-attribute

FILTERED = 'filtered'

only submit images with novel detections and in an uncertainty range (this is the default)

Events

Name	Description
NEW_DETECTIONS	detection on an image is completed (argument: image)

Detector

Detector(*, name: str | None = None)

Bases: ABC

A detector allows detecting objects in images.

It also holds an upload queue for sending images with uncertain results to an active learning infrastructure like the Zauberzeug Learning Loop.

detect abstractmethod async

detect(
    image: Image,
    autoupload: Autoupload = Autoupload.FILTERED,
    tags: list[str] | None = None,
) -> Detections | None

Runs detections on the image. Afterwards the image.detections property is filled.

upload abstractmethod async

upload(
    image: Image, *, tags: list[str] | None = None
) -> None

Uploads the image to the Learning Loop.

The tags are added to the image. If the image has detections, they are also uploaded.

Events

Name	Description
NEW_DETECTIONS	detection on an image is completed (argument: image)

DetectorHardware

DetectorHardware(
    *, port: int = 8004, name: str | None = None
)

Bases: Detector

This detector communicates with a YOLO detector via Socket.IO.

It automatically connects and reconnects, submits and receives detections and sends images that should be uploaded to the Zauberzeug Learning Loop.

Note: Images must be smaller than MAX_IMAGE_SIZE bytes (default: 10 MB).

DetectorSimulation

DetectorSimulation(
    camera_provider: CalibratableCameraProvider,
    *,
    noise: float = 1.0,
    detection_delay: float = 0.4,
    name: str | None = None
)

Bases: Detector

This detector simulates object detection.

It requires a camera provider in order to check visibility using the cameras' calibrations. Individual camera IDs can be added to a set of blocked_cameras to simulate occlusions during pytests. A list of simulated_objects can be filled to define what can be detected. An optional noise parameter controls the spatial accuracy in pixels. An optional detection_delay parameter simulates the time it takes to process an image.

MultiCameraProvider

MultiCameraProvider(*camera_providers: CameraProvider)

Bases: CameraProvider

A multi-camera provider combines multiple camera providers into one.

This is useful if another module requires a single camera provider but the robot has multiple camera sources like USB and WiFi cameras.

RtspCameraProvider

RtspCameraProvider(
    *,
    frame_rate: int = 6,
    jovision_profile: int = 0,
    network_interface: str | None = None,
    auto_scan: bool = True
)

Bases: CameraProvider[RtspCamera], PersistentModule

This module collects and provides real RTSP streaming cameras.

SimulatedCameraProvider

SimulatedCameraProvider(
    *,
    simulate_failing: bool = False,
    auto_scan: bool = True
)

Bases: CameraProvider[SimulatedCamera], PersistentModule

This module collects and simulates cameras and generates synthetic images.

In the current implementation the images only contain the camera ID and the current time.

scan_for_cameras async

scan_for_cameras() -> AsyncGenerator[str, Any]

Simulated device discovery by returning all camera's IDs.

If simulate_device_failure is set, disconnected cameras are returned with a fixed probability.

UsbCameraProvider

UsbCameraProvider(*, auto_scan: bool = True)

Bases: CameraProvider[UsbCamera], PersistentModule

This module collects and provides real USB cameras.

Camera devices are discovered through video4linux (v4l) and accessed with openCV. Therefore the program v4l2ctl and openCV (including python bindings) must be available.