Skip to content

sim2bot.CameraFrame.image

image() -> Any

Decode the frame to a NumPy BGR image.

JPEG and raw RGBA frames are supported. Raw WebGL frames are vertically flipped and converted from RGBA to BGR. H.264 decoding is reserved for a future implementation.

Returns:

  • Any –

    A height × width × 3 uint8 BGR NumPy array. A truncated raw frame

  • Any –

    returns None.

Raises:

  • RuntimeError –

    If the optional OpenCV/NumPy dependencies are absent.

  • NotImplementedError –

    If the frame uses the reserved H.264 codec.

Examples:

Install sim2bot[cv2], then decode a received frame::

frame = stream.read(timeout=2.0)
if frame is not None:
    image_bgr = frame.image()
Source code in sim2bot/client.py 
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
def image(self) -> Any:
    """Decode the frame to a NumPy BGR image.

    JPEG and raw RGBA frames are supported. Raw WebGL frames are vertically
    flipped and converted from RGBA to BGR. H.264 decoding is reserved for a
    future implementation.

    Returns:
        A ``height × width × 3`` uint8 BGR NumPy array. A truncated raw frame
        returns ``None``.

    Raises:
        RuntimeError: If the optional OpenCV/NumPy dependencies are absent.
        NotImplementedError: If the frame uses the reserved H.264 codec.

    Examples:
        Install ``sim2bot[cv2]``, then decode a received frame::

            frame = stream.read(timeout=2.0)
            if frame is not None:
                image_bgr = frame.image()
    """
    if self.codec == _CODEC_H264:
        raise NotImplementedError("H.264 decode needs the sim2bot[av] extra.")
    try:
        import cv2  # type: ignore
        import numpy as np  # type: ignore
    except ImportError as exc:  # pragma: no cover - dependency hint
        raise RuntimeError(
            "Decoding camera frames needs OpenCV + numpy: pip install 'sim2bot[cv2]'"
        ) from exc
    if self.codec == _CODEC_RAW:
        # Uncompressed RGBA, bottom-up (WebGL order): reshape, flip, RGBA->BGR.
        need = self.width * self.height * 4
        if len(self.data) < need:
            return None
        arr = np.frombuffer(self.data, np.uint8, count=need)
        img = arr.reshape(self.height, self.width, 4)[::-1]
        return cv2.cvtColor(img, cv2.COLOR_RGBA2BGR)
    return cv2.imdecode(np.frombuffer(self.data, np.uint8), cv2.IMREAD_COLOR)

Practical examples

Examples use objects discovered from the current scene rather than hard-coded robot metadata.

Decode for OpenCV

# Install first: pip install "sim2bot[cv2]"
image_bgr = frame.image()
if image_bgr is not None:
    print(image_bgr.shape)

Save a frame

import cv2

image_bgr = frame.image()
if image_bgr is not None:
    cv2.imwrite("camera-frame.jpg", image_bgr)

Guidance

Usage tip

OpenCV uses BGR channel order. Convert to RGB before passing pixels to libraries that expect RGB.

Common error

H.264 decoding is not implemented yet; subscribe with codec="jpeg" or codec="raw".

See also