Add a visualisation example of OMERO.tables

[fmeyenhofer]

As a result of the discussions in the forum:
https://forum.image.sc/t/omero-tables-display-on-images-link-questions/87870

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-tables-display-on-images-link-questions/87870/5

[will-moore]

This can just be images = dataset.listChildren(). That gives you a generator of ImageWrappers.
or if you expect to iterate through the images multiple times, then convert the generator to a list:
images = list(dataset.listChildren())

[will-moore]

Thanks for the contribution.
I wonder if you tried the example at https://forum.image.sc/t/omero-tables-display-on-images-link-questions/87870/4 as I think it would be good to include a couple of things from that.

• Since omero2pandas is quite new, we haven't mentioned it in our docs, but I think it has some advantages in terms of much less code to write. The only limitation is that it apparently doesn't support linking to Projects, but I imagine that is easily fixable. At least if we keep the full code example, we should still mention omero2pandas.
• I think it's a bit nicer to use cli_login to handle connection to OMERO. Even with running your example a couple of times, it gets annoying to keep entering your password, whereas cli_login will remember it (and the connection can be reused across other omero cli commands). Again, we haven't mentioned cli_login in our docs, but I've been meaning to do that more and this is a good time.

We have some OMERO.tables example python code at https://omero.readthedocs.io/en/v5.6.9/developers/Python.html#omero-tables and it would be good to avoid duplication (but also make sure that it's findable (I don't know if you've read those docs - linking to that from the OMERO.tables page might be a good idea). I think that example is missing the "Image" column. We could replace that table creation code with your example.

Thanks!

[will-moore]

It's kinda minor point but the convention we've adopted now is to use Image rather than image as the column name. E.g. see https://github.com/ome/omero-metadata#populate. I realise we've used image in some of the older IDR tables (and it certainly works), but more recent ones (created by the current omero-metadata tool) use Image.

[will-moore]

Instead of requiring group_id as an input, you can first lookup the Dataset since by default the gateway will do a cross-group query (or you can enforce that with gateway.SERVICE_OPTS.setOmeroGroup(-1)).
Then you can get the group from the Dataset and use this, which you will need for saving table etc.

    group_id = dataset.getDetails().group.id.val
    conn.SERVICE_OPTS.setOmeroGroup(group_id)

[will-moore]

By convention we use conn for the BlitzGateway connection everywhere else in our docs etc.
If you want to create this from cli_login, you can do:

    with cli_login() as cli:
        conn = BlitzGateway(client_obj=cli._client)

[will-moore]

In editing your example with the various points I was making above, I thought I'd push it so you can see: https://github.com/will-moore/python-scripts/blob/main/omero_tables_docs.py

[fmeyenhofer]

No, I had not tried the example your posted in the forum with the omero2pandas module. I stumbled across it before, but decided that I would like to do without that dependency - given that the module is recent.

Your code review contains good lessons. Obviously it is neater without the little listChildren() blunder.

On the code page, I think an example that can be easily used on one's own OMERO.server instance would make things easier to get started.

On this page, an explicit, visible mention of the image-column-name-convention would be really necessary in my opinion.
I was also wondering if the column name of ImageColumn should not just be hardcoded as "Image" (since OMERO.web is relying on it) , that would render the above obsolete.

Finally a small diagram and/or screenshot of the table display might help to understand how OMERO.tables are intended to function:

Dataset                    <- table (attachement)
   |- Image 1                <- table (link)  + row value display (table panel)
   |- Image 2                ...
   ...

[will-moore]

I understand the preference for writing your own code that you understand and have control over, instead of adding an unknown dependency.
However, I think you can trust the omero2pandas tool as it is developed by Glencoe which is the commercial part of the OME project. They announced the tool at https://www.glencoesoftware.com/blog/2023/05/15/Synchronizing-data-science-management.html

I suggest you:

• update the code sample on the Python page with your changes (random numbers, Image column etc)
• add a link to that from the Tables page and mention the 'Image' column naming requirement
• Keep the screenshot and/or diagram to illustrate

Hardcoding the ImageColumn to be named Image would require server-side changes and would break any existing data that has been created before that change. However, it might make sense for client code to look for a column named "Image" OR a column of type ImageColumn when searching for image IDs. 👍

[fmeyenhofer]

However, I think you can trust the omero2pandas tool as it is developed by Glencoe which is the commercial part of the OME project. They announced the tool at https://www.glencoesoftware.com/blog/2023/05/15/Synchronizing-data-science-management.html

I will try it for my next script.

Hardcoding the ImageColumn to be named Image would require server-side changes and would break any existing data that has been created before that change.

Sure, it was just a thought.

However, it might make sense for client code to look for a column named "Image" OR a column of type ImageColumn when searching for image IDs.

To sum up: For the client OMERO.web to display the rows for each image in the table panel, the table has to contain a column

• with a header (column name) of value Image (or image)
• the column has to be of the type ImageColumn or LongColumn

...or could it be a DoubleColumn , StringColumn or another column type as well?

Your suggestions resonate with me. One more comment concerning your third point

Keep the screenshot and/or diagram to illustrate

Instead of the initially proposed subsection, a new section (heading 2) named something like: Data visualisation (in OMERO.web) might make more sense.
The screenshot, data structure and conditions for the client to be able to fetch the data (here above) could be there.
=> Under Examples the only thing to add - as you mentioned - is the link to the Python examples

Do you want me to pack this in another pull request, or do you prefer to do the changes yourself?

[will-moore]

Sounds great - I opened a code PR at #2355, using a combination of your code sample and the one that was there already.

[fmeyenhofer]

Replaced by #2357