Zooniverse image annotation frontend library?

[csbrown]

Hello. I'm doing some research into image annotation tools. Poking around the Zooniverse repo, it seems like your image annotation tools are all custom-built and baked into the front-end-monorepo monolith. Is this more-or-less accurate? Do you think it would be very difficult to fork/customize the Zooniverse image annotation tools for use as a standalone image annotation tool, or is it inextricably tied to the rest of the Zooniverse monolith?

[goplayoutside3]

Hi there! Thanks for the discussion question. Unfortunately the short answer is yes, the image annotation tools are custom-built as part of Zooniverse’s frontend. The classifier interface where volunteers draw on images is coupled with the project builder (currently resides in the legacy repo Panoptes-Front-End), so using annotation code outside of the Zooniverse ecosystem would be difficult.

Taking inspiration from Zooniverse’s open source code to build your own tool is definitely encouraged though, and I suggest starting with code in the InteractionLayer and drawingTools folder if you’re going to dig in.

[csbrown]

"Pantoptes-Front-End" ahhhhh, there's all the code I was looking for. :D

So, the Zooniverse plan is to refactor all of the annotation tools (and other features) from Panoptes-Front-End to front-end-monorepo moving forward?

Thanks for the info. Sorry for being nosy, but a lot of other well-maintained open-source annotation tools are governed by for-profits which are incentivized to "pull the rug out", so-to-speak. So, would much prefer to be involved with something like the Zooniverse ecosystem if possible.

[trouille]

@csbrown If you have a specific project or goal in mind, great if you email [email protected] and I can see if there are collaboration opportunities/pathways and any other support I can provide as you explore. Thanks! - Laura, Zooniverse PI

[goplayoutside3]

No problem - happy to be transparent about plans for the Zooniverse frontend. Re:

plan is to refactor all of the annotation tools (and other features) from Panoptes-Front-End to front-end-monorepo moving forward?

Yes, Panoptes-Front-End (PFE) is the older version of Zooniverse's website, and some projects still use that repo for classification. FEM is the newer version, and Zooniverse is eventually migrating all projects to this repo. A few projects already use FEM's annotation tools. For example Elephant ID uses the point tool and freehand line tool, while many transcription annotation projects also exist, such as Deciphering Secrets.

Echoing Laura's comment, please don't hesitate to reach out to [email protected] for project building questions.

[csbrown]

This is all extremely helpful @goplayoutside3 and @eatyourgreens .

For context I am needing to ingest data from an existing Zooniverse project (OceanEYES), but also from other annotation tools. Also, I am trying to take stock of the existing annotation-tool-ecosystem, as we have need to get annotations from citizen-scientists but also professional-scientists who are sometimes in an office but sometimes on a boat in the middle of the Pacific. :) @trouille in your opinion would attempting to shoehorn a Zooniverse image-annotation workflow for use in the field (i.e. by hosting it locally) be more trouble than it's worth?

[lcjohnso]

Hi @csbrown -- I'm Cliff, a Zooniverse co-director. I'm chiming in to say that I'll be scheduling a call with you, Ben, and others about next steps for OceanEYEs after Ben reached out to me. We can chat about ML integration, annotation tools, and more, and we can dig into details and options. Talk soon!

[eatyourgreens]

If you’re looking for an open source image annotation library, Annotorius v3 was just released yesterday. It’s an open source tool, based on the W3C Web Annotation standard, with both vanilla JS and React bindings.

https://annotorious.dev/

[eatyourgreens]

@csbrown I originally designed the drawing tools to be a self-contained package within the monorepo. They've been through a few changes since I wrote them, but (in theory) you should be able to extract the drawingTools directory from the classifier without it depending on any other files in the monorepo:
https://github.com/zooniverse/front-end-monorepo/tree/master/packages/lib-classifier/src/plugins/drawingTools

That gives you the individual drawing tools, plus an API for creating, editing and deleting marks. The Readme gives an overview of the drawing API, with some examples:
https://github.com/zooniverse/front-end-monorepo/blob/master/packages/lib-classifier/src/plugins/drawingTools/README.md

The InteractionLayer component, in the classifier package, is an example implementation of a drawing canvas, as an SVG <rect /> with pointer event handlers for creating, modifying and deleting marks via the drawing tools API:
https://github.com/zooniverse/front-end-monorepo/blob/master/packages/lib-classifier/src/components/Classifier/components/SubjectViewer/components/InteractionLayer/InteractionLayer.js

[eatyourgreens]

If you want to consume Zooniverse annotations/classifications in a tool that uses the Web Annotation standard, this repo has examples of code that publishes drawn rectangles, with linked text, as web annotations: https://github.com/zooniverse/iiif-annotations

In particular, this file has the code that converts Zooniverse rectangles to web annotations, where each annotation is a rectangular area of the image with a linked text string:
https://github.com/zooniverse/iiif-annotations/blob/main/annotations/titles/titles.11tydata.js

Here's an example of JSON-LD produced by that code. It's a collection of annotations for a single image, where the image is a scanned playbill and the annotations are performance titles printed on the playbill.
https://zooniverse.github.io/iiif-annotations/annotations/titles/0.json

[Rezanmonis]

We are looking for a frontend developer as our teammate for our edtech project [email protected]

[Rezanmonis]

We are looking for frontend and backend developers as our teammates for our edtech project [email protected]

[csbrown]

@Rezanmonis I'm currently spoken for at the moment, but I will forward this to folks in my circle. It looks like Zooniverse works in a rails stack? Will the edtech project be different?