Interface with Scipion framework

[xulabs]

We will gradually interface AITom with the Scipion framework as a mid-term effort.

https://github.com/I2PC/scipion/wiki

https://github.com/scipion-em

https://journals.iucr.org/d/issues/2019/10/00/ic5108/index.html

[pconesa]

Hi! Count with our assistance... I think now is the right timing for this. We are preparing a release of the scipion-tomo set of plugins with Eman, imod, some xmipp methods....dynamo is on board too.

[pconesa]

Better this page, we very soon will update it

https://scipion-em.github.io/docs/docs/developer/developers.html

[xulabs]

@pconesa great! If you could help us with the interfacing, that would be wonderful!
The usage of current functions can be found in the following tutorials.
https://github.com/xulabs/aitom_doc/tree/master/tutorials

[pconesa]

What would be the simplest method to integrate...my idea is to:

1.- We create a basic plugin for you with one of the methods integrated.
2.- We share it with you and start co-developing it and co-maintaining it.

How does it sound?

[xulabs]

Sounds good!

The simplest method we have is particle picking.
https://github.com/xulabs/aitom_doc/tree/master/tutorials/008_particle_picking

Perhaps we could start from here.

[pconesa]

Great, I've got an skeleton ready. Soon will need your input to call the picker. El jue., 21 nov. 2019 2:53, xulabs <[email protected]> escribió:

…

Sounds good! The simplest method we have is particle picking. https://github.com/xulabs/aitom_doc/tree/master/tutorials/008_particle_picking Perhaps we could start from here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9?email_source=notifications&email_token=AAF7ZYLQ3KZDIHM5OX4BFGDQUXSYJA5CNFSM4IG3WVRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYHMWY#issuecomment-556824155>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF7ZYK2A6UD4ZVDJCGHR6DQUXSYJANCNFSM4IG3WVRA> .

[xulabs]

Great. Thanks.

[pconesa]

HI @xulabs , I've pushed the scipion plugin for Aitom (a VERY basic one) to scipion-em organization.

It is perfectly ok if you want to have it here under xulabs.

You can find in the readme what do you need....there are though a few items I'd like to comment:

1.- Installation: The plugin is able to install aitom in a conda VE, but for that I needed a release (thus I forked aitom repo and just made a release to have the tar.gz files). There are better ways to do this, this is just a quick hack. The usual way to do it is to use pypi and just do pip install xxxxx.

2.- To invoke the picking I also created an entry point, so in a terminal you can run

picking pathtotomogram.mrc pathtojsonoutputfile.json

The picking script is a slight adaptation os your picking tutorial script.

3.- Picking itself....It is parsing the coordinates in the json, but ....there is no boxsize, I've hardcoded it to 100 but I'm sure this is incorrect. My doubt is if there any information about the boxsize per coordinate. How does it work?

[xulabs]

Thanks a lot @pconesa
Currently, almost all of the aitom developers are student volunteers, and they are busy at the end of this semester. Once they are available, I will find a student work on this.
Thanks again!

[pconesa]

Great! We can add whoever you want to our slack channel.

[xiangruz]

@zhuzhenxi

[zhuzhenxi]

HI @pconesa , I successfully install 'scipion-em' in devel mode but meet some problems while testing (after running 'scipion test aitom.tests.tests_picking.TestAitomPicking'). It seems 'picking' command not found.

ERROR running protocol aitom - picking
FAILED with error: Protocol failed: Command ' conda activate aitom-0.1 && picking "/home/zzx/ScipionUserData/projects/TestAitomPicking/Runs/000002_ProtImportTomograms/extra/import_aitom_demo_single_particle_tomogram.mrc" "Runs/000064_AiTomPicking/extra/import_aitom_demo_single_particle_tomogram.mrc.json"' returned non-zero exit status 1

[pconesa]

We have seen many issues when we run conda apps.
What has better worked for us is to:
1.- Do not have conda in the path (remove any conda initiatization in the bash.rc or equivalent.
2.- Tell Scipion how to activate conda by setting the variable CONDA_ACTIVATION_CMD to something like:
I have this working for my laptop in my <SCIPION_HOME>/config/scipion.conf , under [BUILD] section :
CONDA_ACTIVATION_CMD= eval "$(/extra/miniconda3/bin/conda shell.bash hook)"

Note that you need to adapt at least the path or even the shell type (in case you don't use bash))

[pconesa]

@zhuzhenxi when you say scipion-em .....what actually do you mean?
We are working on scipion 3 (in python3) and the latest released version of scipion is scipion2.0.

Which one have you installed. The plugin I initiated targets Scipion 2.0, so python 2.7

[zhuzhenxi]

Thanks a lot @pconesa , I will try again.

[zhuzhenxi]

Conda seems to work, but 'picking' is still 'command not found'(non-zero exit status 127). @pconesa

[pconesa]

Hi @zhuzhenxi , that might be because I changed slightly your code to have an entry point for the picking. Please do not take it literally it was just an approach that worked for me:
master...pconesa:master

I find it very easy to have those entry points, so the user/scipion does not need to know where the scripts are, they are available as commands.

[zhuzhenxi]

Hi @pconesa , the plugin works pretty well, thanks a lot.
A release is available (https://github.com/xulabs/aitom/releases) and code with entry point has been merged (https://github.com/xulabs/aitom/tree/master/aitom/bin). Is it possible now for the plugin to install AITom automatically?

[pconesa]

Nice! It should work, I actually did it getting the release from my fork, but now that you have your own it should be a matter of pointing to your release.

[zhuzhenxi]

Actually, I manually install AITom in conda VE 'aitom-0.1'. So how can I depend on the release and maybe install AITom automatically like other Scipion plugins(more friendly to users)?

[pconesa]

Yep, I've added you as admin of the plugin repository.

This line is the one grabbing the aitom release from my fork, just need to be adapted to take your url.

https://github.com/scipion-em/scipion-em-aitom/blob/devel/aitom/__init__.py#L133

[pconesa]

Your release url should be this one: https://github.com/xulabs/aitom/archive/0.1.tar.gz

Optionally you can deploy aitom to pipy and then the installation will become a "pip install aitom"

[zhuzhenxi]

Ok, I will discuss with Xu for further development.

[xulabs]

@zhuzhenxi yes we may deploy it to pipy. Please check how to do it and let me know. Thanks.

[pconesa]

You are almost there, your code is ready to be deploy, you just need an account and push it.

We have some instructions for our plugins but there is nothing specific to Scipion there, all is standard pypi.org commands:

https://scipion-em.github.io/docs/docs/developer/creating-a-plugin#create-and-upload-distribution

Except fo the -c "scipion-2.0" (you don't need that for uploading aitom.)

[xulabs]

@pconesa got it. Will do. Thank you.

[xulabs]

@pconesa @zhuzhenxi the code is now available at
https://test.pypi.org/project/aitom/0.0.1

@zhuzhenxi I have modified the version from 0.1 to 0.01 because it is a very alpha release. Please change the related code and tutorial accordingly if needed.

[pconesa]

Great, now we just need to do the pip install using it, not sure how to use a pip package from test.pipy but should be documented somewhere...I can make a PR with that change.

[pconesa]

Hi, this completely went out of my radar.

We even organized a tomo course and didn't mention this integration effort.

Maybe is a good time to catch up? How are things going? Do you need any assistance?