Add more fields to /tasks endpoint #526

rochacbruno · 2020-10-22T22:59:24Z

Replaces #488

/tasks/

GET /api/automation-hub/v3/tasks/
HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "meta": {
        "count": 2
    },
    "links": {
        "first": "/api/automation-hub/v3/tasks/?limit=10&offset=0",
        "previous": null,
        "next": null,
        "last": "/api/automation-hub/v3/tasks/?limit=10&offset=0"
    },
    "data": [
        {
            "pulp_id": "4a02f4b2-201c-4c71-8d5b-56a717332d20",
            "name": "pulp_ansible.app.tasks.collections.sync",
            "repository": "rh-certified",
            "state": "failed",
            "started_at": "2020-11-09T16:32:47.436947Z",
            "finished_at": "2020-11-09T16:32:48.340957Z",
            "href": "/api/automation-hub/v3/tasks/4a02f4b2-201c-4c71-8d5b-56a717332d20/"
        },
        {
            "pulp_id": "51d13b1d-328b-4354-b832-b1c706055522",
            "name": "pulp_ansible.app.tasks.collections.sync",
            "repository": "community",
            "state": "failed",
            "started_at": "2020-11-05T21:10:37.050173Z",
            "finished_at": "2020-11-05T21:10:38.158190Z",
            "href": "/api/automation-hub/v3/tasks/51d13b1d-328b-4354-b832-b1c706055522/"
        }
    ]
}

/tasks/pk/

GET /api/automation-hub/v3/tasks/4a02f4b2-201c-4c71-8d5b-56a717332d20/
HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "pulp_id": "4a02f4b2-201c-4c71-8d5b-56a717332d20",
    "name": "pulp_ansible.app.tasks.collections.sync",
    "created_at": "2020-11-09T16:32:47.310079Z",
    "updated_at": "2020-11-09T16:32:48.341382Z",
    "finished_at": "2020-11-09T16:32:48.340957Z",
    "started_at": "2020-11-09T16:32:47.436947Z",
    "state": "failed",
    "error": {
        "traceback": "  File \"/venv/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/venv/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/venv/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 82, in sync\n    d_version.create()\n  File \"/venv/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.6/asyncio/base_events.py\", line 484, in run_until_complete\n    return future.result()\n  File \"/venv/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/venv/lib64/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n    await self.run()\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 290, in run\n    async for metadata in self._fetch_collections():\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 441, in _fetch_collections\n    count = await _loop_through_pages(not_done)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 399, in _loop_through_pages\n    url = await _get_url(1, versions_url)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 387, in _get_url\n    api_endpoint, api_version = await _get_collection_api(remote.url)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/tasks/collections.py\", line 361, in _get_collection_api\n    api_data = parse_metadata(await downloader.run())\n  File \"/venv/lib64/python3.6/site-packages/pulpcore/download/base.py\", line 227, in run\n    return await self._run(extra_data=extra_data)\n  File \"/venv/lib64/python3.6/site-packages/backoff/_async.py\", line 133, in retry\n    ret = await target(*args, **kwargs)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/downloaders.py\", line 88, in _run\n    return await super()._run(extra_data=extra_data)\n  File \"/venv/lib64/python3.6/site-packages/backoff/_async.py\", line 133, in retry\n    ret = await target(*args, **kwargs)\n  File \"/venv/lib64/python3.6/site-packages/pulpcore/download/http.py\", line 210, in _run\n    self.raise_for_status(response)\n  File \"/venv/lib64/python3.6/site-packages/pulp_ansible/app/downloaders.py\", line 66, in raise_for_status\n    response.raise_for_status()\n  File \"/venv/lib64/python3.6/site-packages/aiohttp/client_reqrep.py\", line 946, in raise_for_status\n    headers=self.headers)\n",
        "description": "401, message='Unauthorized', url=URL('https://cloud.redhat.com/api/automation-hub/v3/collections/api/')"
    },
    "worker": {
        "name": "1@d038415f3345",
        "missing": false,
        "last_heartbeat": "2020-11-09T20:22:33.770831Z"
    },
    "parent_task": null,
    "child_tasks": [],
    "repository": "rh-certified",
    "progress_reports": [
        {
            "message": "Parsing CollectionVersion Metadata",
            "code": "parsing.metadata",
            "state": "failed",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Parsing Galaxy Collections API",
            "code": "parsing.collections",
            "state": "failed",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Downloading Artifacts",
            "code": "downloading.artifacts",
            "state": "canceled",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Downloading Artifacts",
            "code": "downloading.artifacts",
            "state": "canceled",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Associating Content",
            "code": "associating.content",
            "state": "canceled",
            "total": null,
            "done": 0,
            "suffix": null
        }
    ]
}

galaxy_ng/app/api/v3/serializers/task.py

Currently it is only possible to import ProgressReportSerializer through `pulpcore.app.serializers` and it is useful to have it on `pulpcore.plugin.serializers` Related PR: ansible/galaxy_ng#526 [noissue]

Currently it is only possible to import ProgressReportSerializer through `pulpcore.app.serializers` and it is useful to have it on `pulpcore.plugin.serializers` Related PR: ansible/galaxy_ng#526 closes #7759

galaxy_ng/app/api/v3/serializers/task.py

- Added 7 more fields to /tasks endpoint - Added information about worker and errors - Added progress_reports from pulp api Issue: AAH-16

No-Issue

temporary solution until pulp/pulpcore#982 is merged No-Issue

No-Issue

alikins · 2020-12-01T15:31:57Z

First glance, I like the pr. It will resolve some issues I've ran into when troubleshooting tasks (notably, that the /tasks from galaxy_ng are missing lots of info that exists in the corresponding pulp task).

Main concern would be making sure the changes are compatible with ansible-galaxy's use of the galaxy_ng task end points. But I haven't dug into that yet.

alikins

LGTM

rochacbruno requested review from alikins, awcrosby, chouseknecht, cutwater and newswangerd as code owners October 22, 2020 22:59

rochacbruno mentioned this pull request Oct 22, 2020

Add sync/task endpoint to list all sync tasks #488

Closed

rochacbruno commented Oct 23, 2020

View reviewed changes

galaxy_ng/app/api/v3/serializers/task.py Show resolved Hide resolved

rochacbruno mentioned this pull request Oct 23, 2020

Expose ProgressReportSerializer on pulpcore.plugin pulp/pulpcore#982

Merged

rochacbruno changed the title ~~[WIP] Add more fields to /tasks endpoint~~ Add more fields to /tasks endpoint Oct 26, 2020

rochacbruno force-pushed the aah-16 branch from 69b9c63 to e4177e5 Compare October 28, 2020 18:32

newswangerd reviewed Oct 28, 2020

View reviewed changes

galaxy_ng/app/api/v3/serializers/task.py Show resolved Hide resolved

rochacbruno changed the title ~~Add more fields to /tasks endpoint~~ [WIP] Add more fields to /tasks endpoint Nov 9, 2020

rochacbruno added the status/do NOT merge label Nov 9, 2020

rochacbruno added 4 commits November 13, 2020 17:09

Add more fields to /tasks endpoint

6ad0043

- Added 7 more fields to /tasks endpoint - Added information about worker and errors - Added progress_reports from pulp api Issue: AAH-16

Add progress_reports from pulpcore

627c8f6

No-Issue

Adding ProgressReportSerializer to exception list

f1c3772

temporary solution until pulp/pulpcore#982 is merged No-Issue

Add unittests

7bfd751

No-Issue

rochacbruno force-pushed the aah-16 branch 4 times, most recently from 5837b9f to bd08397 Compare November 13, 2020 21:40

rochacbruno added 2 commits November 13, 2020 18:41

Adding a Detail view for tasks

109ff8b

No-Issue

Add Summary Serializer and href field

bd08397

No-Issue

rochacbruno removed the status/do NOT merge label Nov 13, 2020

rochacbruno changed the title ~~[WIP] Add more fields to /tasks endpoint~~ Add more fields to /tasks endpoint Nov 13, 2020

rochacbruno requested a review from newswangerd November 13, 2020 21:50

alikins approved these changes Jan 11, 2021

View reviewed changes

alikins merged commit 7901955 into ansible:master Jan 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add more fields to /tasks endpoint #526

Add more fields to /tasks endpoint #526

rochacbruno commented Oct 22, 2020 •

edited

Loading

alikins commented Dec 1, 2020

alikins left a comment

Add more fields to /tasks endpoint #526

Add more fields to /tasks endpoint #526

Conversation

rochacbruno commented Oct 22, 2020 • edited Loading

alikins commented Dec 1, 2020

alikins left a comment

Choose a reason for hiding this comment

rochacbruno commented Oct 22, 2020 •

edited

Loading