Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

iSeg -> Open Source #15

Closed
6 tasks done
esraneufeld opened this issue Feb 1, 2018 · 6 comments
Closed
6 tasks done

iSeg -> Open Source #15

esraneufeld opened this issue Feb 1, 2018 · 6 comments
Assignees
@dyollb @odeimaiz @sanderegg
Milestone
Aquaria

Comments

@esraneufeld
Copy link
Member

esraneufeld commented Feb 1, 2018 •
edited by dyollb
Loading

USER STORY

As external contributor I would like to get access to iSeg code and add features or fix issues.

DEFINITION OF DONE

  • Code is cleaned up (using what metric?)
  • Carfully check 3rd party dependencies for legal issues w.r.t. to the iseg license (tbd)
  • The code is published somewhere
  • we assume that Esra is able to check out/compile and run it only being provided with the github repository link

As outcome from the review:

  • also provide precompiled version of it on github
  • still needs some minor cleanup and renaming

BACKGROUND

As part of our agreement, iSeg shall be openly available for the public.
@mguidon
Copy link
Member

mguidon commented Feb 2, 2018

iSeg is using the SuperMash licensing model
Expected delivery end of March
create osparc-iseg repository
simple to compile
check licensed features and third party tools in current version

@odeimaiz
Copy link
Member

odeimaiz commented Mar 6, 2018 •
edited
Loading

Third party libraries currently used by iSeg:

@mguidon
Copy link
Member

mguidon commented Mar 7, 2018 •
edited
Loading

  1. Check list above for completeness/correctness
  2. Remove proprietary stuff
  3. Clean up and add Readme
  4. Push to osparc-iseg
  5. Automated build procedure for iSeg and 3rd party libs (chocolatey)
  6. Requirements for build tools

@mguidon mguidon added this to the Aquaria milestone Mar 7, 2018
@drniiken
Copy link
Member

drniiken commented Apr 4, 2018

  • SA

  • via github

  • osparc iseg

  • license under MIT terms

  • see dedicated website, + documentation, readme

  • commercial mirror in our gitlab repo

  • also provide precompiled version of it on github

  • still needs cleanup and renaming

  • maybe look at build automation too

  • to become popular, it needs to be very easy to compile, use, extend, etc. !

@sanderegg
Copy link
Member

sanderegg commented Apr 10, 2018 •
edited
Loading

pre-compiled binaries for windows located in: https://github.com/ITISFoundation/osparc-iseg/releases/tag/opensource-v1.0

@drniiken
Copy link
Member

drniiken commented May 2, 2018

  • closed completely

  • use internally now

  • check mac version

  • how to market iSeg

  • plugin system

  • we need to market it, tell the world

  • BL+Nik, come up with action items

  • icons being updated

odeimaiz referenced this issue in odeimaiz/osparc-simcore Sep 13, 2018
@sanderegg
Merge pull request #15 from odeimaiz/feature/prj_model_w_server
e10ed0c 
Feature/prj model w server
mguidon referenced this issue in mguidon/osparc-simcore Nov 14, 2018
@odeimaiz
Merge pull request #15 from sanderegg/fix_port_key
5ce6f70 
Fix port key
sanderegg added a commit that referenced this issue Nov 16, 2018
@sanderegg
Dynamic services to use storage (#334)
3b62a0f 
* Fix pytest on travis

* Added SMTP_HOST and SMTP_PORT to the docker-composes

* No more fake files

* Fixed webserver<->storage specs

* File Picker get nodeModel and projectId

* Get MyDocuments

* Trying to upload files

* fix swarm config for storage

* Increased PLATFORM_VERSION to 3.19

* Revert "Increased PLATFORM_VERSION to 3.19"

This reverts commit cd37313.

* Increased PLATFORM_VERSION to 3.19

* upgraded openapi code generator
re-generated storage client sdk

* go back to 3.2.3

* renamed package name

* fixed crash when no datcore present

* fixed openapi definition for getting locations - was not returning enveloped response

* improved script to automatically remove output.yaml

* modified openapi to specify it actually returns envelopes
re-generated client

* created sample file for testing client

* fixed usage of upload link

* defines link as required

* removed python notebook

* removed log

* changed dependency from s3wrapper to storage client sdk

* make use of Path to ensure correct /

* change name of package to follow conventions

* renamed requirements for production

* code cleanup

* added necesary config entries

* moved includes to fixtures to top'level conftest

* pylint

* fixed storage setup script to copy the openapi correctly

* fixed dependencies to servicelib

* fixed naming

* added logs

* added exception for storage connection error

* return value of location id is an integer not a number

* strip " from none

* added fixture to start storage servicemoved minio out of the docker stack so that pre-signed urls are correctmade filemanager asynctest filemanager complete and functional

* filemanager tests are passingadded exceptions to handle storage server errors

* added configuration for store and bucket

* moved fixtures to conftest
added teardowns for DB
added external minio to get functional pre-signed keys

* added new way of creating file id

* parts of nodeports test passing
made parts async

* added more helpers functions for testing

* fixes the warnings regarding ssl

* refactoring
fixed file_uuid fixture

* fixed await missing

* refactoring storage fixture
fixed issues

* missing files

* fixed tests for getting files and values from previous nodes
things are more flexible

* nodeports tests are passing again

* download using async library aiofiles

* pylint

* code cleanup

* pylint

* updated code generator to latest version

* removed usage of prance to resolve openapi as this is now fixed by fixing the openapi specifications
updated the storage sdk client code
removed wrong nullable entry where a non null response is expected
set openapi schema file to include the components and schemas namespaces to fix validation error about unknown schema
fixed all references by using / after the #

* changed image to use for testing

* replace the minio fixture with the new external to the docker-compose stack

* fixed endpoint without scheme

* storage build for tests

* update docker-compose on travis

* fix issue with not creating parent paths

* pylint

* fixed issue with windows invalid path

* pylint

* Little fixes (#9)

* update docker-compose on travis

* fix issue with not creating parent paths

* pylint

* fixed issue with windows invalid path

* pylint

* fix deps for mock

* try to fix travis hang

* pylint

* try to satisfy travis with log output

* fixed dependencies of sidecar

* another travis test

* fixed dependencies

* re-arranging nodeports

* re-arranged test folder

* fixed installation dependencies

* fixed imports

* re-arranged nodeports for simplicity

* use the latest minio only

* removed the timeout

* wrap async calls in sidecar

* change configuration to STORAGE_ENDPOINT

* Slightly change API

* updated the client code

* pylint

* adapted node ports to new storage api

* fixed setting up of storage enpoint in test

* fixed dependencies. without this requirements, the pdb debugger does not stop

* makefile cleanup

* adapted to new API

* fixes return value

* added mark as pytest asyncio

* set as internal port

* pylint

* only build storage

* export DOCKER_GID before building storage

* wrong env

* fixes tests

* Revert back to old api

* Remove unused field

* FakeData updated to new data structure

* minor

* presignedURL request fixed

* minor

* minor

* cleanup

* updated storage client sdk

* reverted storage api

* mark asyncio tests as async

* removed usage of psycopg2

* remove location from fileuuid

* Merge sandereggs changes

* removing SIMCORE_PIPELINE_ID

* location name is not part of file ID anymore

* fix issues with openapi files

* Forgot those

* refactoring

* allow to also change inputs

* added storage to the devel environment
added STORAGE_ENDPOINT to the config variable

* simplified notebook client

* code refactoring

* renamed

* Fix rerouting to storage

* fixed typo

* renamed folder

* added custom.js to notebook to prevent opening in a new tab

* yaml, we said yAml!!

* user id needs to be passed by the director

* only allow doing stuff in the notebooks folder

* Remove bucket name from file_uuid. It is already defined by the location_id

* Make api for copy more clear

* Make api for copy more cleard-s

* improved theming

* added comments

* Implements copy datcore->s3

* removed bucket from file id

* minor

* try pytest fold

* updated director api to pass user id

* re-generated director server stubs

* refactoring of producer
pass user id to dynamic services

* re-generated director client sdk code

* fixed typos, variable checks

* datcore fakedata updated

* registry proxy is now async

* producer now async

* handlers now using async

* eslint

* fix test

* Adds test for file_uuid creator

* fixed configuration of director sdk

* ISSUE: fixed regular expression in generated sdk code

* Squash from PR#324 webserver uses apihub

Squashed commit of the following:

commit eb33599
Merge: a715899 f7eec98
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 14 11:41:20 2018 +0100

    Bad merge
     Conflicts:
    	services/director/requirements/base.txt

commit a715899
Merge: 0806917 b1f7ad9
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 14 11:36:07 2018 +0100

    Merge branch 'is309/webserver-uses-apihub' of github.com:pcrespov/osparc-simcore into is309/webserver-uses-apihub

commit f7eec98
Merge: 0806917 b1f7ad9
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 14 11:36:07 2018 +0100

    Merge branch 'is309/webserver-uses-apihub' of github.com:pcrespov/osparc-simcore into is309/webserver-uses-apihub

commit 0806917
Merge: 89563a3 ed07bfd
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 14 11:35:25 2018 +0100

    Merge branch 'master' into is309/webserver-uses-apihub

commit ed07bfd
Merge: 0156ec8 c258483
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 14 11:31:37 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit c258483
Author: Tobias Oetiker <[email protected]>
Date:   Wed Nov 14 11:29:25 2018 +0100

    Build Environment Fixes (#314)

    * make swarm work on osx

    * use compose to create input for docker stack

    * have a separate target for launching the qxbuild

    * do NOT put explicit dns into the compose file!!!
    rather make sure the /etc/resolve.conf on the host where docker is
    running is setup correctly!

    * do not rely on boot.sh exec permissions when starting sidecar

    * allow override of REGISTRY_URL from .env

    * better targets for fontent dev

    * Working Makefile for PowerShell

commit b1f7ad9
Merge: 89563a3 ff0ac55
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 22:11:36 2018 +0100

    Merge branch 'master' into is309/webserver-uses-apihub

commit 0156ec8
Merge: e71ed59 ff0ac55
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 21:31:06 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit 89563a3
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 19:26:45 2018 +0100

    Fixes retrieving openapi to webserver.

commit c166bc0
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 18:08:39 2018 +0100

    Fixes makefile filter (see previous commit)
    Removed deprecated modules from merge
    Fixed container_environ fixture
    Added +x to entrypoint.sh otherwise container does not have execution permissions

commit 05d5423
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 17:42:45 2018 +0100

    Fixes bad merge in test_reverser_proxy.
    Disabled web/server pylint from makefile since test_package has a linter test and single-file pylint checks
    produces false positives with relative imports

commit 63df90c
Merge: bd48b1e e71ed59
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 17:29:43 2018 +0100

    Merge branch 'master' into is309/webserver-uses-apihub

commit e71ed59
Merge: 5320cef 2fbeba2
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 17:27:33 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit bd48b1e
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 17:25:09 2018 +0100

    Fixes login tests. Added new variable in config

commit f85a452
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 17:19:29 2018 +0100

    Replaced main.disable_services by "enabled" flags in every service

commit 376f652
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 16:40:25 2018 +0100

    Fixes enveloped schemas to have null error.
    Modifies the test_conventions... to adapt to new criteria

commit c034676
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 16:31:26 2018 +0100

    Added custom variables (OSPARC_SIMCORE_REPO_ROOTDIR) for the config file
    Fixes config folder key

commit f1a0a42
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 13:35:42 2018 +0100

    Fixed linter error

commit 5161825
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 12:01:37 2018 +0100

    fixes on schema my.yaml test failures

commit 0fbcd81
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 11:38:58 2018 +0100

    Removed deprecated tests

commit c774d67
Merge: 1df73de 359a12c
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 11:35:52 2018 +0100

    Merge branch 'is309/webserver-uses-apihub' of github.com:pcrespov/osparc-simcore into is309/webserver-uses-apihub

commit 359a12c
Merge: 6b35214 ff41cf4
Author: Pedro Crespo <[email protected]>
Date:   Tue Nov 13 11:32:42 2018 +0100

    Merge branch 'master' into is309/webserver-uses-apihub

commit 1df73de
Merge: fec4dd3 6b35214
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 16:49:09 2018 +0100

    Merge branch 'is309/webserver-uses-apihub' of github.com:pcrespov/osparc-simcore into is309/webserver-uses-apihub

commit 6b35214
Author: Pedro Crespo-Valero <[email protected]>
Date:   Mon Nov 12 16:01:00 2018 +0100

    complement previous commit

commit 352fc5e
Author: Pedro Crespo-Valero <[email protected]>
Date:   Mon Nov 12 15:58:35 2018 +0100

    Upgrade requests to version 2.20.0 to solve vulnerability alarm

commit 19c7596
Merge: 5746dd5 f51d68b
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 15:56:31 2018 +0100

    Merge branch 'master' into is309/webserver-uses-apihub

commit 5746dd5
Author: Pedro Crespo-Valero <[email protected]>
Date:   Mon Nov 12 15:46:56 2018 +0100

    Removed resouces_keys

commit fec4dd3
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 15:25:04 2018 +0100

    Fixes login ests

commit 26845b1
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 15:13:57 2018 +0100

    Removed application_keys.py

commit b3d7d9c
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 15:13:57 2018 +0100

    Removed application_keys.py

commit 984c9b8
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 15:09:59 2018 +0100

    Cleanup rest. test_rest running

commit 4597c24
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 12:16:07 2018 +0100

    Gathered all director subsystem's  modules under a folder

commit a815a3e
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 11:58:46 2018 +0100

    Renamed comp_backend as computation
    Minor

commit 0ff6c01
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 11:57:24 2018 +0100

    Split sockets subsystem into setup and handlers
    Added s3 subsystem

commit 5098583
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 10:55:20 2018 +0100

    Adapted configurations to latest schema.
    Added environs to docker-compose and removed from .env-devel hosts and ports (should be hard-coded in docker-compose)

commit cff674b
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 12 10:54:54 2018 +0100

    Synced some requirements

commit de4f9dc
Author: Pedro Crespo <[email protected]>
Date:   Sat Nov 10 21:21:28 2018 +0100

    - every subsystem defines its own config section: computation, db, email, rest, director . All have their own *_config.py
    - created directory submodule and its config
    - added tests for config sections
    - renamed settings as application_config
    - renamed computational_backend as computation

commit 4bde625
Author: Pedro Crespo <[email protected]>
Date:   Fri Nov 9 20:10:34 2018 +0100

    WIP

commit 5320cef
Merge: cf6c63a 062f221
Author: Pedro Crespo <[email protected]>
Date:   Fri Nov 9 17:17:34 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit cf6c63a
Merge: 4a8c093 63e662c
Author: Pedro Crespo <[email protected]>
Date:   Fri Nov 9 11:09:51 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit 4a8c093
Merge: 0fd2975 1a3ed38
Author: Pedro Crespo <[email protected]>
Date:   Wed Nov 7 11:57:52 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit 0fd2975
Merge: d423dcc 8d7d44f
Author: Pedro Crespo <[email protected]>
Date:   Mon Nov 5 10:24:49 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit d423dcc
Merge: 6ca6aad dfdd594
Author: Pedro Crespo <[email protected]>
Date:   Fri Nov 2 14:06:23 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit 6ca6aad
Merge: e733846 cae4fb1
Author: Pedro Crespo <[email protected]>
Date:   Wed Oct 31 14:33:37 2018 +0100

    Merge remote-tracking branch 'upstream/master'

commit e733846
Merge: 9e40d37 77d6ac1
Author: Pedro Crespo <[email protected]>
Date:   Wed Oct 31 13:36:25 2018 +0100

    Merge branch 'master' of github.com:pcrespov/osparc-simcore

commit 9e40d37
Merge: 65f510c 0b60f46
Author: Pedro Crespo <[email protected]>
Date:   Tue Oct 30 18:53:50 2018 +0100

    Merge remote-tracking branch 'upstream/master'

Pull

* updated storage configuration

* move the webserver-storage api to apihub

* Create bucket if not yet there

* fixes bad merge

* added director handlers
added oas specs

* linked director handlers
creating director test

* useless test

* added director client session

* added version to director schema
fixed typo

* bad merge

* removed registry_api

* removed userid from webserver api

* changed call to startDynamic stopDynamic to go through the rest API instead of websockets

* refactoring

* fixed version of path
use isfunction instead of ismethod to find handlers

* Feature/fe dsm (#15)

* Pylint

* Add reference osparc.itis stack template

* fixed call to API by explicitely using query parameters

* the server side does not like the format field...

* refatoring

* call forwarding now also forwards the query parameters

* removed logging

* added workbench json and its converted openapi counterpart
added route to start pipeline

* implementation of new route

* comment out code to fix cyclic import

* fix bad merge

* eslint

* pylint

* bad merge

* disable old start_pipeline

* bypass validation

* regenerated client sdk

* store now contains the id instead of the store name

* added async sleep when waiting for service start
fixed looking for service dependencies

* fix bad merge

* bad merge

* pass the user id to the sidecar

* fix test

* use env variable instead of hard-coded host/port

* add storage endpoint env to director so it can pass it on
add storage endpoint to sidecar so it can use it

* renamed to test_package

* mapping of routes working

* pylint
ignapas referenced this issue in ignapas/osparc-simcore Mar 11, 2019
@oetiker
Merge pull request #15 from odeimaiz/feature/fix-client-serving
91db142 
Test frontend is served
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dyollb dyollb

@odeimaiz odeimaiz

@sanderegg sanderegg

Labels
None yet
Projects
None yet
Milestone
Aquaria
Development

No branches or pull requests
6 participants
@dyollb @drniiken @odeimaiz @mguidon @esraneufeld @sanderegg