Skip to content
This repository has been archived by the owner on Apr 26, 2023. It is now read-only.

Scipion demo for ICS course (configuration)

Jump to bottom
David Maluenda Niubó edited this page May 30, 2019 · 8 revisions

Scipion demo for the course of International School of Crystallography

Setting up the machines

Here you can find the instructions to set up your machine to be able to follow the Scipion tutorial session.

There will be two options to follow up the tutorial session:

  1. To use a virtual machine on Amazon cloud (AWS): A number of AWS instances will be ready during the session to be used by the students. Connections details will be given at that time, and the only required software will be a web browser and a reasonable internet connection.

    Get access (remember to accept the https risk):

       https://IP.xxx.yyy.zzz/vnc.html  --> psswd: scipionICS

    Resize the screen with the gear at left edge menu --> remote size.

  2. The super alternative plan: To set up your own Scipion machine using VirtualBox: A VirtualBox virtual appliance, that you could easily launch from your own computer using Oracle VirtualBox application, is ready to be downloaded from here. Then follow the next steps:

    1.1. Create a new Virtual Machine (Type Linux and Version Ubuntu).

    1.2. Assign at least 6 GB memory and 4 CPUs (Settings > System > Motherboard/Processor).

    1.3. Choose as existing hard disk the virtual appliance downloaded (vdi).

    1.4. Create.

    1.5. Start it and login with user scipion and password scipionICS. By default the virtual machine screen is quite small, to resize it open a terminal and type xrandr -q and choose the resolution that fits better your laptop screen and then set it up with xrandr -s wxh (e.g. 1280x1024) or by Start > Preferences > Monitor Settings.

You can open Scipion by clicking on the desktop shortcut. There are two demo projects that you can visualize prior to the session (you could try to run some steps yourself but processing might be slow since number of cores and RAM are virtualized and not very high and GPU is not present). Raw movies are not present either to save some disk space. Project ScipionCombined shows a representative workflow where many of the Scipion possibilities are shown while project AcquisitionDemo presents a finished workflow of an Acquisition Simulation.

Datasets

The dataset is the beta-galactosidase EMPIAR 10061 (4k with 38 frames).

The full set to simulate the large acquisition is made of ~150 movies (~250Gb) and it can be gotten from /home/dmaluenda/dataLocal/rawData/10061_dwnsp2/* (only for the Jose Maria's machine).

The sample of that dataset (to be used by the students) can be reached using the pattern /home/dmaluenda/dataLocal/rawData/10061_dwnsp2/*1_*.mrc.

This datasets should be placed where RAWDATA_SIM at ~/em-facilities/usingAPI_demo/scipionbox.conf is pointing out (~/rawData/EMPIAR_10062_2dwsamp, by default).

Projects

We will have two stored projects ScipionCombined and AcquisitionDemo. They can be reached from carver at /home/dmaluenda/ScipionUserData/ScipionCombined and /home/dmaluenda/ScipionUserData/AcquisitionDemo.

Streaming workflows

We will run two streaming workflows, one from a JSON template and, the other from a Scipion's API script.

From JSON template

Foe this, you do not need any special configuration. Just open a web browser, download the All2Monitor_GPU_.... workflow from the http://workflows.scipion.i2pc.es. Then import the downloaded JSON to an empty Scipion project.

If no GPU device is available, you can download the All2Monitor_CPU_.... template.

Check that the movies path is pointing to where the real data is placed, ~/rawData/EMPIAR_10062_2dwsamp by default. After editing some form parameter, SAVE the protocol, but do NOT Execute any protocol from the form!.

When you are happy with all the parameters of all the protocols , you can launch the whole workflow by right-click in the Import Movies + Restart workflow.

If a protocol was Executed from the form, that protocol will be Scheduled and, then, the Restart workflow raise an error. Don't panic! You must just execute all the other protocols from the form.

From Scipion's API script

  • The script is in a git repository, thus
   git clone https://github.com/I2PC/em-facilities

  • All needed code is in the usingAPI_demo directory.

  • The launcher is assuming that Scipion is in ~/scipion and this repository at ~/em-facilities, if not, please edit in the first lines of the launcher (usingAPI_demo/script.sh).

  • Also, usingAPI_demo/scipionbox.conf contains config variables. The most machine dependent variables are:

    • DEPOSITION_PATH: any directory where the script will simulate the acquisition creating symbolic links.

    • SCIPION_PROJECT: Where to put the Scipion's project (~/ScipionUserData/projects by default).

    • RAWDATA_SIM: Where the real data is located (20 or 200 movies, see datasets sections above).

    • NUM_CPU: Number of CPUs of the machine.

  • Once the configuration is ready, we can launch the Acquisition Simulation by

   sh ~/em-facilities/usingAPI_demo/script.sh

NOTE: Several threads are called by this script, so to Stop it you need either to look for all the process and kill them or close the terminal where it was launch (also a killall python works, but be careful in AWS because vnc use a python script...). To avoid this annoying situation, we can make a desktop launcher/shortcut calling the same command above. Then, when we close the opened terminal by the shortcut, all tasks will be killed.

Clone this wiki locally