Skip to content

Commit

Permalink
Improve PIL and COLOR_BGR context image decode performance (#1501)
Browse files Browse the repository at this point in the history
### Summary

- Ticket no. 141409
- There was a significant performance degradation on `decode_by_pil()`
if `ImageColorChannel.COLOR_BGR`.
- Improve it by returning a numpy array rather than `PIL.Image` and
using `cv2.cvtColor()` rather than `np.flip()`.

### How to test

- `tests/unit/test_image.py` covers this change.
- Compare performance with this code (dataset: Imagenette2)

| before | after |
| :-: | :-: |
|
![image](https://github.com/openvinotoolkit/datumaro/assets/26541465/3fd115b0-26b3-4a71-b058-0153cea96b98)
|
![image](https://github.com/openvinotoolkit/datumaro/assets/26541465/cb32adb6-cc70-4aef-9ab4-a47b862f1947)
|
| 00:51<00:00, 361.65it/s | 00:19<00:00, 859.39it/s |


### Checklist
<!-- Put an 'x' in all the boxes that apply -->
- [ ] I have added unit tests to cover my changes.​
- [ ] I have added integration tests to cover my changes.​
- [x] I have added the description of my changes into
[CHANGELOG](https://github.com/openvinotoolkit/datumaro/blob/develop/CHANGELOG.md).​
- [ ] I have updated the
[documentation](https://github.com/openvinotoolkit/datumaro/tree/develop/docs)
accordingly

### License

- [x] I submit _my code changes_ under the same [MIT
License](https://github.com/openvinotoolkit/datumaro/blob/develop/LICENSE)
that covers the project.
  Feel free to contact the maintainers if that's a concern.
- [x] I have updated the license header for each file (see an example
below).

```python
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT
```

---------

Signed-off-by: Kim, Vinnam <[email protected]>
  • Loading branch information
vinnamkim authored May 17, 2024
1 parent 4394c6a commit 3501e97
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 14 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
(<https://github.com/openvinotoolkit/datumaro/pull/1471>)
- Add ExtractedMask and update importers who can use it to use it
(<https://github.com/openvinotoolkit/datumaro/pull/1480>)
- Improve PIL and COLOR_BGR context image decode performance
(<https://github.com/openvinotoolkit/datumaro/pull/1501>)

### Bug fixes
- Split the video directory into subsets to avoid overwriting
Expand Down
27 changes: 13 additions & 14 deletions src/datumaro/util/image.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@
from datumaro.components.crypter import NULL_CRYPTER, Crypter
from datumaro.components.errors import DatumaroError

try:
# Introduced in 1.20
from numpy.typing import DTypeLike
except ImportError:
DTypeLike = Any
if TYPE_CHECKING:
try:
# Introduced in 1.20
from numpy.typing import DTypeLike
except ImportError:
DTypeLike = Any


class ImageBackend(Enum):
Expand All @@ -46,9 +47,6 @@ class ImageBackend(Enum):
from datumaro.util.image_cache import ImageCache
from datumaro.util.os_util import find_files

if TYPE_CHECKING:
from PIL.Image import Image as PILImage


class ImageColorChannel(Enum):
"""Image color channel
Expand Down Expand Up @@ -79,20 +77,21 @@ def decode_by_cv2(self, image_bytes: bytes) -> np.ndarray:

raise ValueError

def decode_by_pil(self, image_bytes: bytes) -> PILImage:
def decode_by_pil(self, image_bytes: bytes) -> np.ndarray:
"""Convert image color channel for PIL Image."""
from PIL import Image

img = Image.open(BytesIO(image_bytes))

if self == ImageColorChannel.UNCHANGED:
return img
return np.asarray(img)

if self == ImageColorChannel.COLOR_BGR:
return Image.fromarray(np.flip(np.asarray(img.convert("RGB")), -1))
img = np.asarray(img.convert("RGB"))
return cv2.cvtColor(img, cv2.COLOR_RGB2BGR)

if self == ImageColorChannel.COLOR_RGB:
return img.convert("RGB")
return np.asarray(img.convert("RGB"))

raise ValueError

Expand Down Expand Up @@ -289,13 +288,13 @@ def decode_image(image_bytes: bytes, dtype: DTypeLike = np.uint8) -> np.ndarray:

if IMAGE_BACKEND.get() == ImageBackend.cv2:
image = ctx_color_scale.decode_by_cv2(image_bytes)
image = image.astype(dtype)
elif IMAGE_BACKEND.get() == ImageBackend.PIL:
image = ctx_color_scale.decode_by_pil(image_bytes)
image = np.asarray(image, dtype=dtype)
else:
raise NotImplementedError()

image = image.astype(dtype)

assert len(image.shape) in {2, 3}
if len(image.shape) == 3:
assert image.shape[2] in {3, 4}
Expand Down

0 comments on commit 3501e97

Please sign in to comment.