Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PixelMapping #251

Merged
merged 1 commit into from
Dec 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions modules/_zivid/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
infield_correction,
Matrix4x4,
data_model,
PixelMapping,
projection,
ProjectedImage,
presets,
Expand Down
1 change: 1 addition & 0 deletions modules/zivid/experimental/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from zivid.experimental._pixel_mapping import PixelMapping
34 changes: 34 additions & 0 deletions modules/zivid/experimental/_pixel_mapping.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
"""Module for experimental pixel mapping. This API may change in the future."""

import _zivid


class PixelMapping:
"""Pixel mapping from subsampled to full resolution.

Required when mapping an index in a subsampled point cloud to e.g. a full resolution 2D image.
"""

def __init__(self, row_stride=1, col_stride=1, row_offset=0.0, col_offset=0.0):
self.__impl = _zivid.PixelMapping(
row_stride, col_stride, row_offset, col_offset
)

@property
def row_stride(self):
return self.__impl.row_stride()

@property
def col_stride(self):
return self.__impl.col_stride()

@property
def row_offset(self):
return self.__impl.row_offset()

@property
def col_offset(self):
return self.__impl.col_offset()

def __str__(self):
return str(self.__impl)
30 changes: 30 additions & 0 deletions modules/zivid/experimental/calibration.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Module for experimental calibration features. This API may change in the future."""

import _zivid
from zivid.experimental import PixelMapping
from zivid.calibration import DetectionResult
from zivid.camera import Camera
from zivid.camera_intrinsics import _to_camera_intrinsics
Expand Down Expand Up @@ -71,6 +72,35 @@ def estimate_intrinsics(frame):
)


def pixel_mapping(camera, settings):
"""Return pixel mapping information given camera and settings.

When mapping from a subsampled point cloud to a full resolution
2D image it is important to get the pixel mapping correct. This
mapping depends on camera model and settings. This function provides
the correct parameters to map the 2D coordinates in a point cloud
captured using `settings` to the full resolution of the camera.

Args:
camera: Reference to camera instance.
settings: Reference to settings instance.

Returns:
A PixelMapping instance.
"""

pixel_mapping_handle = _zivid.calibration.pixel_mapping(
camera._Camera__impl, # pylint: disable=protected-access
_to_internal_settings(settings),
)
return PixelMapping(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is a bit strange that we here have a _zivid.PixelMapping, but we don't pass that into the constructor, instead the constructor creates another _zivid.PixelMapping.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It constructs and returns zivid.experimental._pixel_mapping which is mapped to zivid.experimental.PixelMapping.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a chicken and egg story here, right Eskil?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem was that we really needed two constructors:

  • One for the user that takes the four numbers
  • One for us that takes a _zivid.PixelMapping

Since Python can't really overload constructors, we opted to only have the first one.
Open to better solutions, but I think this is fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm fine with keeping this. We can find a cleaner solution in the future.

pixel_mapping_handle.row_stride(),
pixel_mapping_handle.col_stride(),
pixel_mapping_handle.row_offset(),
pixel_mapping_handle.col_offset(),
)


def capture_calibration_board(camera):
"""Capture the calibration board.

Expand Down
1 change: 1 addition & 0 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ set(SOURCES
Firmware.cpp
InfieldCorrection/InfieldCorrection.cpp
NodeType.cpp
PixelMapping.cpp
Projection.cpp
Presets.cpp
ReleasableArray2D.cpp
Expand Down
14 changes: 11 additions & 3 deletions src/Calibration/Calibration.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,16 @@ namespace ZividPython::Calibration
},
py::arg("camera"),
py::arg("settings_2d"))
.def("estimate_intrinsics", [](ReleasableFrame &releasableFrame) {
return Zivid::Experimental::Calibration::estimateIntrinsics(releasableFrame.impl());
});
.def("estimate_intrinsics",
[](ReleasableFrame &releasableFrame) {
return Zivid::Experimental::Calibration::estimateIntrinsics(releasableFrame.impl());
})
.def(
"pixel_mapping",
[](ReleasableCamera &releasableCamera, const Zivid::Settings &settings) {
return Zivid::Experimental::Calibration::pixelMapping(releasableCamera.impl(), settings);
},
py::arg("camera"),
py::arg("settings"));
Comment on lines +69 to +75
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is experimental in C++, and thus should really be in a experimental namespace in Python. It seems we have done a bad mistake for the other ones abvove here by not separating Experimental to a separate namespace/module.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I guess this is internal code only and that the public part is infact experimental. Then it looks Ok

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is how we've done it before, with for example infield correction.

}
} // namespace ZividPython::Calibration
18 changes: 18 additions & 0 deletions src/PixelMapping.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#pragma once

#include <Zivid/Experimental/PixelMapping.h>

#include <pybind11/pybind11.h>

namespace ZividPython
{
void wrapClass(pybind11::class_<Zivid::Experimental::PixelMapping> pyClass)
{
pyClass.def(pybind11::init(), "Initializes all values to their defaults")
.def(pybind11::init<int, int, float, float>())
.def("row_stride", &Zivid::Experimental::PixelMapping::rowStride)
.def("col_stride", &Zivid::Experimental::PixelMapping::colStride)
.def("row_offset", &Zivid::Experimental::PixelMapping::rowOffset)
.def("col_offset", &Zivid::Experimental::PixelMapping::colOffset);
}
} // namespace ZividPython
4 changes: 4 additions & 0 deletions src/Wrapper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#include <ZividPython/Firmware.h>
#include <ZividPython/InfieldCorrection/InfieldCorrection.h>
#include <ZividPython/Matrix4x4.h>
#include <ZividPython/PixelMapping.h>
#include <ZividPython/Presets.h>
#include <ZividPython/Projection.h>
#include <ZividPython/ReleasableArray2D.h>
Expand Down Expand Up @@ -69,4 +70,7 @@ ZIVID_PYTHON_MODULE // NOLINT
ZIVID_PYTHON_WRAP_NAMESPACE_AS_SUBMODULE(module, InfieldCorrection);
ZIVID_PYTHON_WRAP_NAMESPACE_AS_SUBMODULE(module, Projection);
ZIVID_PYTHON_WRAP_NAMESPACE_AS_SUBMODULE(module, Presets);

using PixelMapping = Zivid::Experimental::PixelMapping;
ZIVID_PYTHON_WRAP_CLASS(module, PixelMapping);
}
10 changes: 10 additions & 0 deletions src/include/ZividPython/PixelMapping.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#pragma once

#include <Zivid/Experimental/PixelMapping.h>

#include <pybind11/pybind11.h>

namespace ZividPython
{
void wrapClass(pybind11::class_<Zivid::Experimental::PixelMapping> pyClass);
} // namespace ZividPython
37 changes: 37 additions & 0 deletions test/experimental/test_pixel_mapping.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
def test_pixel_mapping(file_camera):
from zivid.experimental.calibration import pixel_mapping
from zivid.settings import Settings

pixel_mapping_handle = pixel_mapping(
camera=file_camera, settings=Settings(acquisitions=[Settings.Acquisition()])
)
assert isinstance(pixel_mapping_handle.row_stride, int)
assert isinstance(pixel_mapping_handle.col_stride, int)
assert isinstance(pixel_mapping_handle.row_offset, float)
assert isinstance(pixel_mapping_handle.col_offset, float)
Comment on lines +8 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also check the exact values that are used, IMO. To ensure that it is mapped correctly in the pybind wrapper.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert pixel_mapping_handle.row_stride == 1
assert pixel_mapping_handle.col_stride == 1
assert pixel_mapping_handle.row_offset == 0.0
assert pixel_mapping_handle.col_offset == 0.0

blue_subsample2x2_settings = Settings()
blue_subsample2x2_settings.acquisitions.append(Settings.Acquisition())
blue_subsample2x2_settings.sampling.pixel = Settings.Sampling.Pixel.blueSubsample2x2
pixel_mapping_handle = pixel_mapping(
camera=file_camera, settings=blue_subsample2x2_settings
)
assert pixel_mapping_handle.row_stride == 2
assert pixel_mapping_handle.col_stride == 2
assert pixel_mapping_handle.row_offset == 0.0
assert pixel_mapping_handle.col_offset == 0.0

red_subsample2x2_settings = Settings()
red_subsample2x2_settings.acquisitions.append(Settings.Acquisition())
red_subsample2x2_settings.sampling.pixel = Settings.Sampling.Pixel.redSubsample2x2
pixel_mapping_handle = pixel_mapping(
camera=file_camera, settings=red_subsample2x2_settings
)
assert pixel_mapping_handle.row_stride == 2
assert pixel_mapping_handle.col_stride == 2
assert pixel_mapping_handle.row_offset == 1.0
assert pixel_mapping_handle.col_offset == 1.0