Allow renaming datasets & dataset with duplicate names #8075
Conversation
…ir dataset addressing scheme
… the check for already existing name)
…gaId - Includes moving ObjectId to uitls package
…store & add legacy routes - Undo renaming DataSourceId to LegacyDataSourceId
…ow-dataset-renaming
app/models/task/TaskService.scala
Outdated
| "dataSet" -> dataset.name, | ||
| "datasetName" -> dataset.name, | ||
| "datasetId" -> dataset._id, // Only used for csv serialization in frontend. |
| creationInfo: null, | ||
| dataSet: '2012-06-28_Cortex', | ||
| datasetId: '570b9f4e4bb848d0885ee711', | ||
| datasetName: '2012-06-28_Cortex', | ||
| editPosition: [ |
There was a problem hiding this comment.
@fm3 I just noticed by check which files where updated. I thought we wanted to avoid having datasetName and dataSet in parallel?
Although checking: #8075 (comment) shows that we did not clearly have this decision. Should I include a legacy route for anything that returns / receives a task? There is already a legacy route for creating and updating a task, but the publicWrites in TaskService still returns
"dataSet" -> dataset.name,
"datasetName" -> dataset.name,
"datasetId" -> dataset._id, // Only used for csv serialization in frontend.
Should I add a legacy route for this to not always send dataSet and datasetName?
There was a problem hiding this comment.
This can also be seen in tasks.e2e.js.md
There was a problem hiding this comment.
I think it would be cleaner to send the old info only in legacy routes (if the frontend doesn’t use it now). However, leaving in the redundant property does not hurt much either. So I’ll leave this decision up to you. If you think it’s quick and easy enough to clean this up now, go for it. Otherwise, I’m also happy if we can merge this soon, and perhaps schedule a follow-up issue for this (then we’ll have to bump the api version again, but that should not be a problem).
CHANGELOG.unreleased.md
Outdated
| @@ -15,6 +15,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released | |||
|
|
|||
| ### Changed | |||
| - Reading image files on datastore filesystem is now done asynchronously. [#8126](https://github.com/scalableminds/webknossos/pull/8126) | |||
| - Dataset can now be renamed and can have duplicate names. [#8075](https://github.com/scalableminds/webknossos/pull/8075) | |||
There was a problem hiding this comment.
| - Dataset can now be renamed and can have duplicate names. [#8075](https://github.com/scalableminds/webknossos/pull/8075) | |
| - Datasets can now be renamed and can have duplicate names. [#8075](https://github.com/scalableminds/webknossos/pull/8075) |
…d required legacy adaption
- no legacy routes needed as not used by wklibs
|
I now removed the I'll do testing of the new legacy routes later |
Looks right to me! |
…ow-dataset-renaming
- also remove unised injections and imports
|
Hi @fm3, I added the required legacy routes to remove For the curl commands, you first need to fill in the id cookie before you can run the scripts. Before testing here is one more important thing to double check: The legacy routes now include both: Checklist for testing new Legacy Routes
Testing create & update task -> :shurg: I did this manually in firefox with the edit & resent feature on the dev instance. In case you want to test more complicated routes:curl command for checking task creation (setting id cookie is required)curl 'https://allowdatasetrenaming.webknossos.xyz/api/tasks' \
-H 'accept: application/json' \
-H 'accept-language: en-US,en;q=0.9,de;q=0.8' \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-H 'cookie: id=<fill-me-in>' \
-H 'origin: https://allowdatasetrenaming.webknossos.xyz' \
-H 'pragma: no-cache' \
-H 'priority: u=1, i' \
-H 'referer: https://allowdatasetrenaming.webknossos.xyz/tasks/create' \
-H 'sec-ch-ua: "Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-origin' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36' \
--data-raw '[{"taskTypeId":"63721e2cef0100470266c485","neededExperience":{"domain":"sampleExp","value":1},"pendingInstances":1,"projectName":"sampleProject","boundingBox":null,"dataSet":"kiwi","editPosition":[0,0,0],"editRotation":[0,0,0],"baseAnnotation":null}]'Test task update: (setting id cookie is required)curl 'https://allowdatasetrenaming.webknossos.xyz/api/tasks/6746ecc6010000d80015ea8c' \
-X 'PUT' \
-H 'accept: application/json' \
-H 'accept-language: en-US,en;q=0.9,de;q=0.8' \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-H 'cookie: id=<fill-me-in>' \
-H 'origin: https://allowdatasetrenaming.webknossos.xyz' \
-H 'pragma: no-cache' \
-H 'priority: u=1, i' \
-H 'referer: https://allowdatasetrenaming.webknossos.xyz/tasks/6746ecc6010000d80015ea8c/edit' \
-H 'sec-ch-ua: "Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "Linux"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-origin' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36' \
--data-raw '{"taskTypeId":"63721e2cef0100470266c485","neededExperience":{"domain":"sampleExp","value":1},"pendingInstances":2,"projectName":"sampleProject","boundingBox":null,"datasetId":"kiwi","editPosition":[0,0,0],"editRotation":[0,0,0]}' |
| sil: Silhouette[WkEnv])(implicit ec: ExecutionContext, bodyParsers: PlayBodyParsers) | ||
| extends Controller { | ||
|
|
||
| /* to provide v8, remove legacy routes */ |
There was a problem hiding this comment.
Not sure I understand this comment. What legacy routes are removed? (I mean, the PR removes some, but this comment appears to be about the following code lines). Maybe it should be something like “to provide v8, add dataSet to task json?” Or maybe remove these comments altogether.
There was a problem hiding this comment.
Ah damn it. This is a mixture of a todo comment which I forgot to remove and copy & paste mistake.
I renamed all such occurrences to simple /* provide vX */
There was a problem hiding this comment.
Or maybe remove these comments altogether.
Something like these comments were present previously (I think). Thus, I kept them to "order" the code a little :)
I can also remove them in case you prefer that :)
CHANGELOG.unreleased.md
Outdated
| @@ -26,6 +27,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released | |||
| - Fix a bug where dataset uploads would fail if the organization directory on disk is missing. [#8230](https://github.com/scalableminds/webknossos/pull/8230) | |||
|
|
|||
| ### Removed | |||
| - Removed legacy routes for versions 2,3 and 4. [#8075](https://github.com/scalableminds/webknossos/pull/8075) | |||
There was a problem hiding this comment.
| - Removed legacy routes for versions 2,3 and 4. [#8075](https://github.com/scalableminds/webknossos/pull/8075) | |
| - Removed support for HTTP API versions 3 and 4. [#8075](https://github.com/scalableminds/webknossos/pull/8075) |
(v2 was already removed previously) Could you also mention this in the migration guide please? (If someone has old code interacting with the HTTP API, they need to update)
…ow-dataset-renaming
- fix changelog entry - add migration entry about dropped api versions
|
merge conflicts & your two comments are done now. So it's ready for 🚢 |
|
🎉 🤞 |
Further Notes:
URL of deployed dev instance (used for testing):
Steps to test:
TODOs:
organization_namein worker toorganization_id. see Rename organization_name to organization_id in worker args #8038LinkedLayerIdentifierstill uses the datasetName as an identifier[ ] the datasetThe dataset seems to be broken. Could reproduce this on other branchesC555_tps_demohas quite some bucket loading errors. Unsure why some buckets do not workDatasetURIParserIssues:
(Please delete unneeded items, merge only when none are left open)