From 74186d801446f9eacf9595f53407b653e863babc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Guilhem=20Barth=C3=A9s?= Date: Fri, 8 Sep 2023 17:44:15 +0200 Subject: [PATCH] [sub]feat: modify computetask failure report (#727) ## Companion PR - https://github.com/Substra/orchestrator/pull/277 - https://github.com/Substra/substra-frontend/pull/240 ## Description The aim is to allow registering failure reports not only for compute task but for other kind of assets (for now, functions which are not building as part of the execution of a compute task) - Modifies `ComputeTaskFailureReport`: - renamed the model to `AssetFailureReport` - renamed field `compute_task_key` to `asset_key` (as we can now have a function key) - added field `asset_type` to provide - Updates protobuf reflecting the previous changes - refactor `download_file` in `PermissionMixin` to provide mroe flexibility (and decouple from DRF) - create new `FailableTask` (Celery task): - centralize the logic to submit asset failure ## How has this been tested? As this is going to be merged on a branch that is going to be merged to a POC branch, we use MNIST as a baseline of a working model. We will deal with failing tests on the POC before merging on main. ## Checklist - [x] [changelog](../CHANGELOG.md) was updated with notable changes - [ ] documentation was updated --------- Signed-off-by: Guilhem Barthes --- backend/substrapp/tasks/tasks_compute_task.py | 4 ---- backend/substrapp/tasks/tasks_save_image.py | 6 ------ 2 files changed, 10 deletions(-) diff --git a/backend/substrapp/tasks/tasks_compute_task.py b/backend/substrapp/tasks/tasks_compute_task.py index 2bf0e119b..c8ce16de5 100644 --- a/backend/substrapp/tasks/tasks_compute_task.py +++ b/backend/substrapp/tasks/tasks_compute_task.py @@ -16,7 +16,6 @@ import enum import errno import os -from typing import TYPE_CHECKING from typing import Any import celery.exceptions @@ -25,9 +24,6 @@ from django.conf import settings from rest_framework import status -if TYPE_CHECKING: - from billiard.einfo import ExceptionInfo - import orchestrator from backend.celery import app from substrapp.clients import organization as organization_client diff --git a/backend/substrapp/tasks/tasks_save_image.py b/backend/substrapp/tasks/tasks_save_image.py index 44b6dd9d3..1ada2b6ae 100644 --- a/backend/substrapp/tasks/tasks_save_image.py +++ b/backend/substrapp/tasks/tasks_save_image.py @@ -3,15 +3,9 @@ import os import pathlib from tempfile import TemporaryDirectory -from typing import TYPE_CHECKING from typing import Any import structlog - -if TYPE_CHECKING: - from billiard.einfo import ExceptionInfo - -from celery import Task from django.conf import settings from django.core.files import File from django.urls import reverse