Generate catalog with tomlschema #50

sverhoeven · 2022-03-09T16:18:57Z

Refs #47

The catalogs have been regenerated with newer haddock3 yaml files.
The diff shows both the application of tomlschema to fold mol_1 and mol_2 into mol and documented/remove/added/moved parameters,

Fixes #35

Also make accept comma seperated for <input type=file accept/>

…talog-with-tomlschema

netlify · 2022-03-09T16:19:04Z

✅ Deploy Preview for wonderful-noether-53a9e8 ready!

Name	Link
🔨 Latest commit	`8e7aba8`
🔍 Latest deploy log	https://app.netlify.com/sites/wonderful-noether-53a9e8/deploys/624c2c0f23fef100088011d6
😎 Deploy Preview	https://deploy-preview-50--wonderful-noether-53a9e8.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Also removed dead code

…-tomlschema

sverhoeven · 2022-03-24T14:58:12Z

Noticed that hisd and hise parameter in topoaa node in easy catalog are written as hisd = [123] instead of hisd_1 = 123. So still some work todo.

Peter9192 · 2022-03-24T16:27:18Z

Tried to run the code but didn't manage to install haddock correctly. The instructions about linking the CNS executable are unclear. How am I supposed to obtain this executable?

I also wondered whether you are planning to write some small tests for this script.

Peter9192 · 2022-03-24T16:28:37Z

Found it: http://cns-online.org/v1.3/, continuing

Peter9192 · 2022-03-24T16:32:18Z

Please print out this page with your Institution's letterhead, sign and email a scanned PDF to [email protected] .

Okay I'm gonna call it a day 😞

sverhoeven · 2022-03-25T07:07:49Z

hisd and hise are now indexed as well.

bvreede · 2022-03-30T08:48:01Z

public/catalog/haddock3.expert.yaml

-        description: Defines a molecule as a shape.
-        $comment: Defines a molecule as a shape, which is a collection of beads.
-        type: boolean
+      mol_fix_origin:


Should this category have a title, too?

When selecting expert > rigidbody > molecule, the parameter 'mol_fix_origin' appears, and lacks a descriptive title.

The name seems to come from here: https://github.com/haddocking/haddock3/blob/e324b9cbd4c9c76ea1df6ce874e8ba5317486218/src/haddock/modules/sampling/rigidbody/defaults.yaml#L4

Would it make sense to inherit title and description from the array items, or are they not always the same for all items?

Good of you to notice.

I need to get those titles/descriptions from haddock3 so I created haddocking/haddock3#370

Peter9192

It seems to work well. @bvreede and I had a good time trying to understand all the regexes and how you exclude all options starting from the most complex ones to the simpler ones.

There's a few quirks that I guess are hard to iron out completely, and given that there's no tests I assume you don't want to make a very big effort to make this script perfect. So please go ahead and merge after looking at our few small questions :-)

Peter9192 · 2022-03-30T11:22:51Z

util/generate_haddock3_catalog.py

+    array_of_array_of_scalar = r'(\w+)_1_1'
+    array_of_array_of_object = r'(\w+)_(\w+)_1_1'
+    skip = r'\w+?(_\d\d?)(_\d\d?)?'
+    must_be_array_of_scalar = {'mol_fix_origin_1', 'mol_shape_1'}


it seems this is hardcoded because the regex can't distinguish between

mol_shape_1 -> mol_shape:[1]

mol_shape_1 -> mol:[{shape}]

So basically any key that consists of multiple name segments separated by underscores is confusing. Is it discouraged to create new names like that in Haddock? Should there be a warning if a new parameter like this is encountered?

Yep, parameters starting with mol need get special treatment. See https://github.com/haddocking/haddock3/blob/main/src/haddock/gear/expandable_parameters.py#L2-L72 how haddock3 code treats these parameters.

Peter9192 · 2022-03-30T11:44:17Z

util/generate_haddock3_catalog.py

+            if 'group' in v:
+                # Move group from nested prop to outer array
+                new_config[p]['group'] = v['group']
+                del v['group']


why must the group be deleted?

So as a side-effect of calling this function, the incoming config is modified?

The array will be in that group and the scalar value is a child of that array. So the scalar is already in the group, by keeping a group on the scalar it would be rendered in its own collapsible subform.

You would then have a collapsible subforms inside an array itemwhich was already inside a collapsible subform felt a bit to much folding going on.

Also the grouper currently only works on parameters that are not nested.

util/generate_haddock3_catalog.py

Peter9192 · 2022-03-30T12:40:44Z

util/generate_haddock3_catalog.py

 import sys
 from yaml import dump, load, Loader

 from haddock.modules import modules_category
-from haddock import config_expert_levels
+from haddock import config_expert_levels, _hidden_level


Not sure what _hidden_level is

Hidden level is the level beyond guru and used for experimental or very obscure parameters

Peter9192 · 2022-03-30T12:41:46Z

util/generate_haddock3_catalog.py

 def process_global(level):
    package = 'haddock.modules'
    module = importlib.import_module(package)
    with open(module.modules_defaults_path) as f:
        optional_global_parameters = load(f, Loader=Loader)
-    config = REQUIRED_GLOBAL_PARAMETERS | optional_global_parameters
+    gmodule = importlib.import_module('haddock.gear.parameters')


much cleaner 👍

Co-authored-by: Peter Kalverla <[email protected]>

sverhoeven

Thanks for reviewing, I know these first reviews can be hard to do.

sverhoeven · 2022-04-05T11:45:49Z

util/generate_haddock3_catalog.py

 import sys
 from yaml import dump, load, Loader

 from haddock.modules import modules_category
-from haddock import config_expert_levels
+from haddock import config_expert_levels, _hidden_level


Hidden level is the level beyond guru and used for experimental or very obscure parameters

sverhoeven · 2022-04-05T11:46:15Z

util/generate_haddock3_catalog.py

 def process_global(level):
    package = 'haddock.modules'
    module = importlib.import_module(package)
    with open(module.modules_defaults_path) as f:
        optional_global_parameters = load(f, Loader=Loader)
-    config = REQUIRED_GLOBAL_PARAMETERS | optional_global_parameters
+    gmodule = importlib.import_module('haddock.gear.parameters')


sverhoeven added 5 commits February 17, 2022 15:35

Fetch mandatory global parameters from haddock3

f13641b

When explevel is hidden then skip that parameter

a661f58

Fixes #35

Recover ui for global molecules parameter

f4b501c

Also make accept comma seperated for <input type=file accept/>

Merge remote-tracking branch 'origin/skip-hidden-35' into generate-ca…

92a5a00

…talog-with-tomlschema

Handle topoaa.mol1 when generating catalog + started on X_1 parameters

74b5c7b

sverhoeven and others added 5 commits March 11, 2022 08:49

Collapse *_1 before producing schemas

c0e7ae9

Use collapsed_config

3fa3a3a

No longer dying, but still incorrect array of objects

8983cba

Skip parameters with index > 1

f918f79

Also removed dead code

Expandable are not required

96804c2

sverhoeven mentioned this pull request Mar 14, 2022

Table widget #53

Merged

sverhoeven marked this pull request as ready for review March 16, 2022 11:56

sverhoeven added 3 commits March 22, 2022 12:37

Merge remote-tracking branch 'origin/main' into generate-catalog-with…

90bba42

…-tomlschema

Add kitchensink back to catalog

724886e

Added docstring

e26ba5e

sverhoeven requested a review from Peter9192 March 22, 2022 14:43

Store tomlschema of array inside array of objects

8a739d4

bvreede reviewed Mar 30, 2022

View reviewed changes

Peter9192 approved these changes Mar 30, 2022

View reviewed changes

Apply suggestions from code review

8e7aba8

Co-authored-by: Peter Kalverla <[email protected]>

sverhoeven commented Apr 5, 2022

View reviewed changes

sverhoeven merged commit 067fb67 into main Apr 5, 2022

sverhoeven deleted the generate-catalog-with-tomlschema branch April 5, 2022 11:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate catalog with tomlschema #50

Generate catalog with tomlschema #50

sverhoeven commented Mar 9, 2022 •

edited

Loading

netlify bot commented Mar 9, 2022 •

edited

Loading

sverhoeven commented Mar 24, 2022 •

edited

Loading

Peter9192 commented Mar 24, 2022

Peter9192 commented Mar 24, 2022

Peter9192 commented Mar 24, 2022

sverhoeven commented Mar 25, 2022

bvreede Mar 30, 2022

Peter9192 Mar 30, 2022 •

edited

Loading

sverhoeven Apr 4, 2022

Peter9192 left a comment

Peter9192 Mar 30, 2022

sverhoeven Apr 4, 2022

Peter9192 Mar 30, 2022

Peter9192 Mar 30, 2022

sverhoeven Apr 4, 2022

Peter9192 Mar 30, 2022

sverhoeven Apr 5, 2022

Peter9192 Mar 30, 2022

sverhoeven Apr 5, 2022

sverhoeven left a comment

sverhoeven Apr 5, 2022

sverhoeven Apr 5, 2022

Generate catalog with tomlschema #50

Generate catalog with tomlschema #50

Conversation

sverhoeven commented Mar 9, 2022 • edited Loading

netlify bot commented Mar 9, 2022 • edited Loading

✅ Deploy Preview for wonderful-noether-53a9e8 ready!

sverhoeven commented Mar 24, 2022 • edited Loading

Peter9192 commented Mar 24, 2022

Peter9192 commented Mar 24, 2022

Peter9192 commented Mar 24, 2022

sverhoeven commented Mar 25, 2022

Choose a reason for hiding this comment

Peter9192 Mar 30, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Peter9192 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sverhoeven left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sverhoeven commented Mar 9, 2022 •

edited

Loading

netlify bot commented Mar 9, 2022 •

edited

Loading

sverhoeven commented Mar 24, 2022 •

edited

Loading

Peter9192 Mar 30, 2022 •

edited

Loading