Kucalc: API copy status, Take 2

[haraldschilly]

Kucalc API: copy a file without blocking

The terminal session below is from my local computer hitting the test namespace. I've updated all hubs for that (but I think just the api hub is enough ... )

What's new is

• copy_path_between_projects gets a switch -d wait_until_done=false
• copy_path_status with -d copy_path_id=[copy_path_id]

copying an existing file from A to B

hsy@x1:~/d/cocalc/test$ curl -s -X POST -u `cat test_api`: -d src_project_id=`cat p1` -d target_project_id=`cat p2` -d wait_until_done=false -d src_path=x.md https://test.cocalc.com/api/v1/copy_path_between_projects | jq
{
  "event": "copy_path_between_projects_response",
  "id": "354f6977-47e0-435f-80c2-ccb3593f1217",
  "copy_path_id": "b83f8dbf-8c0c-49ab-913a-24a20d862792",
  "note": "Query copy_path_status with the copy_path_id to learn if the copy operation was successful."
}

retrieving the status of the above -- no error!

hsy@x1:~/d/cocalc/test$ curl -s -X POST -u `cat test_api`: -d copy_path_id=b83f8dbf-8c0c-49ab-913a-24a20d862792  https://test.cocalc.com/api/v1/copy_path_status | jq{
  "event": "copy_path_status_response",
  "id": "e03a7e75-b70c-4d03-95b1-89106eb3c801",
  "data": {
    "time": "2019-08-02T14:00:03.786Z",
    "source_project_id": "c37fbd83-c4c3-4f92-b66c-37b8d2c8cdf1",
    "source_path": "x.md",
    "target_project_id": "9282d61d-8d27-4b9f-ae0f-2fc9bac64203",
    "target_path": "x.md",
    "overwrite_newer": false,
    "delete_missing": false,
    "backup": false,
    "started": "2019-08-02T14:00:03.798Z",
    "finished": "2019-08-02T14:00:04.190Z"
  }
}

copying a file that does not exist from A to B

hsy@x1:~/d/cocalc/test$ curl -s -X POST -u `cat test_api`: -d src_project_id=`cat p1` -d target_project_id=`cat p2` -d wait_until_done=false -d src_path=unknown-path.md https://test.cocalc.com/api/v1/copy_path_between_projects | jq
{
  "event": "copy_path_between_projects_response",
  "id": "e09847d0-2ff0-46e2-90f9-810296e377e8",
  "copy_path_id": "510f5acd-7955-4067-af3d-d664584d6f45",
  "note": "Query copy_path_status with the copy_path_id to learn if the copy operation was successful."
}

retrieving the error message of the event above

hsy@x1:~/d/cocalc/test$ curl -s -X POST -u `cat test_api`: -d copy_path_id=510f5acd-7955-4067-af3d-d664584d6f45  https://test.cocalc.com/api/v1/copy_path_status | jq{
  "event": "copy_path_status_response",
  "id": "f463e4c7-c347-4ae8-ba1d-ec3aa1bfc45a",
  "data": {
    "time": "2019-08-02T14:00:42.046Z",
    "source_project_id": "c37fbd83-c4c3-4f92-b66c-37b8d2c8cdf1",
    "source_path": "unknown-path.md",
    "target_project_id": "9282d61d-8d27-4b9f-ae0f-2fc9bac64203",
    "target_path": "unknown-path.md",
    "overwrite_newer": false,
    "delete_missing": false,
    "backup": false,
    "started": "2019-08-02T14:00:42.056Z",
    "finished": "2019-08-02T14:00:42.448Z",
    "error": "{\"errno\":-2,\"code\":\"ENOENT\",\"syscall\":\"lstat\",\"path\":\"/home/manage/projects/c37fbd83-c4c3-4f92-b66c-37b8d2c8cdf1/unknown-path.md\"}"
  }
}

backwards compatible? synchronized copying

hsy@x1:~/d/cocalc/test$ time curl -s -X POST -u `cat test_api`: -d src_project_id=`cat p1` -d target_project_id=`cat p2`  -d src_path=unknown-path.md https://test.cocalc.com/api/v1/copy_path_between_projects | jq
{
  "event": "error",
  "id": "f429442c-6f23-4252-9f0f-879d19686f8b",
  "error": "{\"errno\":-2,\"code\":\"ENOENT\",\"syscall\":\"lstat\",\"path\":\"/home/manage/projects/c37fbd83-c4c3-4f92-b66c-37b8d2c8cdf1/unknown-path.md\"}"
}

battle testing

to test this even further, I issued a copy to a project which doesn't exist. I immediately got the copy id, and a start time.... Then the process stupidly tries for 2 minutes, then finally gives up and records this:

hsy@x1:~/d/cocalc/test$ curl -s -X POST -u `cat test_api`: -d copy_path_id=46ead3df-87f1-4f3b-b115-9062b7c76951  https://test.cocalc.com/api/v1/copy_path_status | jq
{
  "event": "copy_path_status_response",
  "id": "f4c1d04d-f99a-4b7d-8dd9-1833a2829cb1",
  "data": {
    "time": "2019-08-02T14:13:59.563Z",
    "source_project_id": "c37fbd83-c4c3-4f92-b66c-37b8d2c8cdf1",
    "source_path": "x.md",
    "target_project_id": "fe36d5d7-b126-4431-a9ee-977d14de2b4b",
    "target_path": "x.md",
    "overwrite_newer": false,
    "delete_missing": false,
    "backup": false,
    "started": "2019-08-02T14:13:59.572Z",
    "finished": "2019-08-02T14:15:55.298Z",
    "error": "maximum time (=120000ms) exceeded - last error \"no project with given id in the database?!\""
  }
}

Relevant Issues

Checklist:

• No debugging console.log messages.
• All new code is actually used.
• Non-obvious code has some sort of comments.

Front end:

• Restart at least one project and check its ~/.smc/local_hub/local_hub.log
• Completely restart Webpack with ./w in /src
• Completely restart the hub by killing and restarting ./start_hub.py in /src/dev/project
• Screenshots if relevant.

[williamstein]

Everything looks good, but I don't understand why you made this default a STRING 'true' and not just the boolean true?

[haraldschilly]

maybe I don't know how to use curl -X POST -d key=true, but when I do that, I get a string. I wanted to make this consistent with what's coming in, that's all. The code itself checks for boolean val or the default string.

[williamstein]

Looks fine except for one comment about true v 'true'.