Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug when using a "Connected file share" and "Segment size" parameter [UI & SDK] or "job_file_mapping" parameter [SDK]. (Regression since CVAT 2.23) #9146

Open
2 tasks done
jimytim opened this issue Feb 24, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@jimytim
Copy link

jimytim commented Feb 24, 2025

Actions before raising this issue

  • I searched the existing issues and did not find anything similar.
  • I read/searched the docs

Steps to Reproduce

Steps to reproduce via UI:

  1. Go to the "Tasks" view
  2. Click [+] "Create a new task"
  3. Set task name
  4. Create a label
  5. Select resource "Connected file share"
  6. Select 3 images (or at least 2 images)
  7. Click "Advanced configuration"
  8. Set "Segment size" to 2 (or at least 1)
  9. Click "Submit & Open"
  10. Click on 2nd job (one with a higher ID).
  11. See the error message "Could not receive image data"

Image
Image
Image
Image

Code sample to reproduce using the Python SDK:

from cvat_sdk import make_client
from cvat_sdk.core.proxies.tasks import ResourceType

images = [
    "./images/image_0.png",
    "./images/image_1.png",
    "./images/image_2.png",
]

with make_client(host="localhost", credentials=('user', 'password')) as client:

    task_A_spec = {
        "name": "TaskA",
        "segment_size": 2,
        "labels": [{"name": "LabelName", "color": "#0572f7"}],
    }

    task_B_spec = {
        "name": "TaskB",
        "labels": [{"name": "LabelName", "color": "#0572f7"}],
    }

    job_file_mapping = [[images[0], images[1]], [images[2]]]

    # Segment Size Test
    task_A = client.tasks.create(task_A_spec)

    task_A.upload_data(
        resources=images,
        resource_type=ResourceType.SHARE,
    )

    # Job File Mapping Test
    task_B = client.tasks.create(task_B_spec)

    task_B.upload_data(
        resources=images,
        resource_type=ResourceType.SHARE,
        params={"job_file_mapping": job_file_mapping},
    )

Expected Behavior

Image data from other jobs should be received.

Possible Solution

The bug is not present in CVAT 2.23

Context

Using a connected file share, we are trying to create multiple jobs using either the "segment size" parameter in the task creation form UI, or in the task specification with the Python SDK.
In the SDK, the bug also impacts the job_file_mapping parameter in task.upload_data().

Other information:

  • If the segment size parameter is not set, all the images are correctly visible in the annotation view.
  • If the job_file_mapping parameter is not used, all the images are correctly visible in the annotation view.
  • The images in the first job are always correctly uploaded, but the bug impacts all the other jobs. This happens regardless of the number of jobs; only the images in the first job are correctly uploaded.
  • There is no issue if the resource type is local (i.e when selecting "My computer" as the source in the task creation form)

Environment

- CVAT Server version: 2.30.0
- CVAT UI version: 2.30.0
- CVAT SDK version: 2.30.0

- Docker version:

Client:
 Version:           27.5.1
 API version:       1.47
 Go version:        go1.22.11
 Git commit:        9f9e405
 Built:             Wed Jan 22 13:37:19 2025
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.38.0 (181591)
 Engine:
  Version:          27.5.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.11
  Git commit:       4c9b3b0
  Built:            Wed Jan 22 13:41:25 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e946
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

- Are you using Docker Swarm or Kubernetes?
    No

- Operating System and version: 
    MacOS Sequoia 15.3

- Other diagnostic information / logs:
    Logs from `cvat_server` container:

2025-02-21 18:22:17 2025-02-21 23:22:17,899 DEBG 'uvicorn-0' stderr output:
2025-02-21 18:22:17 [2025-02-21 23:22:17,899] ERROR django.request: Internal Server Error: /api/jobs/58/preview
2025-02-21 18:22:17 Traceback (most recent call last):
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 518, in thread_handler
2025-02-21 18:22:17     raise exc_info[1]
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 42, in inner
2025-02-21 18:22:17     response = await get_response(request)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
2025-02-21 18:22:17     response = await wrapped_callback(
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 468, in __call__
2025-02-21 18:22:17     ret = await asyncio.shield(exec_coro)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 40, in run
2025-02-21 18:22:17     result = self.fn(*self.args, **self.kwargs)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 522, in thread_handler
2025-02-21 18:22:17     return func(*args, **kwargs)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
2025-02-21 18:22:17     return view_func(*args, **kwargs)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 124, in view
2025-02-21 18:22:17     return self.dispatch(request, *args, **kwargs)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
2025-02-21 18:22:17     response = self.handle_exception(exc)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
2025-02-21 18:22:17     self.raise_uncaught_exception(exc)
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
2025-02-21 18:22:17     raise exc
2025-02-21 18:22:17   File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
2025-02-21 18:22:17     response = handler(request, *args, **kwargs)
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/views.py", line 2431, in preview
2025-02-21 18:22:17     return data_getter()
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/views.py", line 910, in __call__
2025-02-21 18:22:17     return super().__call__()
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/views.py", line 782, in __call__
2025-02-21 18:22:17     data = frame_provider.get_preview()
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/frame_provider.py", line 553, in get_preview
2025-02-21 18:22:17     preview, mime = cache.get_or_set_segment_preview(self._db_segment)
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/cache.py", line 467, in get_or_set_segment_preview
2025-02-21 18:22:17     self._get_or_set_cache_item(
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/cache.py", line 192, in _get_or_set_cache_item
2025-02-21 18:22:17     return self._create_cache_item(
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/cache.py", line 267, in _create_cache_item
2025-02-21 18:22:17     wait_for_rq_job(rq_job)
2025-02-21 18:22:17   File "/home/django/cvat/apps/engine/cache.py", line 113, in wait_for_rq_job
2025-02-21 18:22:17     raise exc_type(*exc_args)
2025-02-21 18:22:17 FileNotFoundError: [Errno 2] No such file or directory
@jimytim jimytim added the bug Something isn't working label Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant