Add dynamic loading of optimizer #34

aramoto99 · 2022-08-10T05:37:02Z

Changed the way optimizers are specified in the Config file.This change eliminates the need for reinstallation when the optimzier code is changed.

grid -> aiaccel.optimizer.GridSearchOptimizer
random -> aiaccel.optimizer.RandomSearchOptimizer
sobol -> aiaccel.optimizer.SobolSearchOptimizer
nelder-mead -> aiaccel.optimizer.NelderMeadSearchOptimizer
tpe -> aiaccel.optimizer.TpeSearchoptimizer
(e.g.)

aiaccel/examples/benchmark/config.yaml

Line 16 in 55fcab8

search_algorithm: "aiaccel.optimizer.NelderMeadOptimizer"

2. Optimizers not included in the aiaccel package can also be specified.In that case, must be added path.

export PYTHONPATH="/hoge:$PYTHONPATH"

…/optimizer

github-actions · 2022-08-10T05:50:21Z

Unit Test Results

238 tests ±0 238 ✔️ ±0 40s ⏱️ +7s
    1 suites ±0     0 💤 ±0
    1 files ±0     0 ❌ ±0

Results for commit 55fcab8. ± Comparison against base commit 8f6df87.

This pull request removes 23 and adds 23 tests. Note that renamed tests count towards both.

tests.unit.optimizer_test.test_grid_search.TestGridSearchOptimizer ‑ test_generate_parameter
tests.unit.optimizer_test.test_grid_search.TestGridSearchOptimizer ‑ test_get_parameter_index
tests.unit.optimizer_test.test_grid_search.TestGridSearchOptimizer ‑ test_pre_process
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_check_result
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_generate_parameter
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_generate_parameter2
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_get_ready_parameters
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_pre_process
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_set_maximize
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadSearchOptimizer ‑ test_set_minimize
…

tests.unit.optimizer_test.test_grid_search.TestGridOptimizer ‑ test_generate_parameter
tests.unit.optimizer_test.test_grid_search.TestGridOptimizer ‑ test_get_parameter_index
tests.unit.optimizer_test.test_grid_search.TestGridOptimizer ‑ test_pre_process
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_check_result
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_generate_parameter
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_generate_parameter2
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_get_ready_parameters
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_pre_process
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_set_maximize
tests.unit.optimizer_test.test_nelder_mead_search.TestNelderMeadOptimizer ‑ test_set_minimize
…

♻️ This comment has been updated with latest results.

yoshipon · 2022-08-12T00:48:28Z

Thank you!
How about specifying the class name in the configuration to remain the original class name.
For example, aiaccel.optimizer.grid.GridOptimizer

yoshipon · 2022-08-12T01:20:14Z

aiaccel/optimizer/optimizer.py

+        """
+        return self.opt.Optimizer
+
+    @singledispatchmethod


Is it really necessary to construct a generic function?

…ure/optimizer

yoshipon · 2022-08-21T11:43:55Z

Thank you for updating the PR.
It would be better to consider a simpler way to specify the standard optimizer (e.g., NelderMeadOptimizer).

tkokada · 2022-08-22T02:08:18Z

@aramoto99 What kinds of rules are defined for any user optimizers? I think you find them using following codes.

aiaccel/aiaccel/start.py

Lines 512 to 519 in 6b3963d

    
           def get_pyfile_path(self, module_name: str) -> str: 
        
               path = ( 
        
                   module_name 
        
                   .replace(".", "/") 
        
                   .replace("//", "../") 
        
                   .replace(" ", "") 
        
               ) + ".py" 
        
               return path

yoshipon

Thank you, LGTM!

* Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix TPE resumtion test (#29) * Fix tpe resumption test * fix tpe resumption test * Fix tep resumption test * add retry (#38) * Add dynamic loading of optimizer (#34) * (WIP) Add dynamic loading of optimizer by importlib. * (WIP) Add dynamic loading of optimizer by importlib. * Add dynamic loading of optimizer by importlib * Add dynamic loading of optimizer by importlib * Fix test code * fix test * Fix resumtioin test * Add dynamic loading of optimizer * Fix dynamic loading of optimizer * Function names and comments were rolled back. * Changed the standard optimizer specification method * changed the specification method for importing the standard optimizer. * change deserialize (#40) Co-authored-by: aramoto99 <[email protected]> * Merge branch 'main' of github:aistairc/aiaccel into feature/database * Fix codestyle * (WIP) Change TPE resume * Fix resume test * Feature/fix resume tpe (#41) * fix resume tpe * fix resume tpe * add if len(running_trials) > 0 * fix about trial number * Fix TPE resumption Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]>

* Data management function by database is added. By adding the `--fs` option argument to the startup command, data management functions by the conventional file system can be used. Extensive refactoring was performed around the addition of this functionality. * Update setup.py * Delete tests/test_data/work/state/ as it is not used in develop/database. * Develop/database fix test (#18) * fix for tests * delete space * pInfoDict -> p_info * Merge main into debelop/database (#31) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix resumption test for DB Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * Feature/database (#36) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * Fixed a problem in which Snapshot could not be saved during parallel execution. * Add function of delete any data in DB/FS. * Fix code style Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> * Request to merge cumulative changes . (#37) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * The report generation function can now be called by command. * Fix data inconsistency when excute resume. (#39) * Fix resume * Fixed problem with sirialize not working in fs mode * Remove unused package imports * Fixed a problem that the resume function did not work in fs mode. * The name workspace/state is the name of the old specification, so it was changed to the appropriate name workspace/snapshot. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> * Add dynamic loading of optimizer, And fix resumtion test. (#42) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix TPE resumtion test (#29) * Fix tpe resumption test * fix tpe resumption test * Fix tep resumption test * add retry (#38) * Add dynamic loading of optimizer (#34) * (WIP) Add dynamic loading of optimizer by importlib. * (WIP) Add dynamic loading of optimizer by importlib. * Add dynamic loading of optimizer by importlib * Add dynamic loading of optimizer by importlib * Fix test code * fix test * Fix resumtioin test * Add dynamic loading of optimizer * Fix dynamic loading of optimizer * Function names and comments were rolled back. * Changed the standard optimizer specification method * changed the specification method for importing the standard optimizer. * change deserialize (#40) Co-authored-by: aramoto99 <[email protected]> * Merge branch 'main' of github:aistairc/aiaccel into feature/database * Fix codestyle * (WIP) Change TPE resume * Fix resume test * Feature/fix resume tpe (#41) * fix resume tpe * fix resume tpe * add if len(running_trials) > 0 * fix about trial number * Fix TPE resumption Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * Fix conflicts with main branch (#43) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix TPE resumtion test (#29) * Fix tpe resumption test * fix tpe resumption test * Fix tep resumption test * add retry (#38) * Add dynamic loading of optimizer (#34) * (WIP) Add dynamic loading of optimizer by importlib. * (WIP) Add dynamic loading of optimizer by importlib. * Add dynamic loading of optimizer by importlib * Add dynamic loading of optimizer by importlib * Fix test code * fix test * Fix resumtioin test * Add dynamic loading of optimizer * Fix dynamic loading of optimizer * Function names and comments were rolled back. * Changed the standard optimizer specification method * changed the specification method for importing the standard optimizer. * change deserialize (#40) Co-authored-by: aramoto99 <[email protected]> * Merge branch 'main' of github:aistairc/aiaccel into feature/database * Fix codestyle * (WIP) Change TPE resume * Fix resume test * Feature/fix resume tpe (#41) * fix resume tpe * fix resume tpe * add if len(running_trials) > 0 * fix about trial number * Fix TPE resumption Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * Fix codestyle Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]>

* Data management function by database is added. By adding the `--fs` option argument to the startup command, data management functions by the conventional file system can be used. Extensive refactoring was performed around the addition of this functionality. * Update setup.py * Delete tests/test_data/work/state/ as it is not used in develop/database. * Develop/database fix test (#18) * fix for tests * delete space * pInfoDict -> p_info * Merge main into debelop/database (#31) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix resumption test for DB Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * Feature/database (#36) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * Fixed a problem in which Snapshot could not be saved during parallel execution. * Add function of delete any data in DB/FS. * Fix code style Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> * Request to merge cumulative changes . (#37) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * The report generation function can now be called by command. * Fix data inconsistency when excute resume. (#39) * Fix resume * Fixed problem with sirialize not working in fs mode * Remove unused package imports * Fixed a problem that the resume function did not work in fs mode. * The name workspace/state is the name of the old specification, so it was changed to the appropriate name workspace/snapshot. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. * Delete unnecessary directories in Workspace. Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> * Add dynamic loading of optimizer, And fix resumtion test. (#42) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix TPE resumtion test (#29) * Fix tpe resumption test * fix tpe resumption test * Fix tep resumption test * add retry (#38) * Add dynamic loading of optimizer (#34) * (WIP) Add dynamic loading of optimizer by importlib. * (WIP) Add dynamic loading of optimizer by importlib. * Add dynamic loading of optimizer by importlib * Add dynamic loading of optimizer by importlib * Fix test code * fix test * Fix resumtioin test * Add dynamic loading of optimizer * Fix dynamic loading of optimizer * Function names and comments were rolled back. * Changed the standard optimizer specification method * changed the specification method for importing the standard optimizer. * change deserialize (#40) Co-authored-by: aramoto99 <[email protected]> * Merge branch 'main' of github:aistairc/aiaccel into feature/database * Fix codestyle * (WIP) Change TPE resume * Fix resume test * Feature/fix resume tpe (#41) * fix resume tpe * fix resume tpe * add if len(running_trials) > 0 * fix about trial number * Fix TPE resumption Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * Fix conflicts with main branch (#43) * Fix #19 (#20) * Fix #21 (#22) * fix README (#23) * fix codestyle test (#26) * fix for pycodestyle * add flake8==4.0.1 * Feature/update readme (#25) * update README * add bash Co-authored-by: Yoshiaki Bando <[email protected]> * Fix TPE resumtion test (#29) * Fix tpe resumption test * fix tpe resumption test * Fix tep resumption test * add retry (#38) * Add dynamic loading of optimizer (#34) * (WIP) Add dynamic loading of optimizer by importlib. * (WIP) Add dynamic loading of optimizer by importlib. * Add dynamic loading of optimizer by importlib * Add dynamic loading of optimizer by importlib * Fix test code * fix test * Fix resumtioin test * Add dynamic loading of optimizer * Fix dynamic loading of optimizer * Function names and comments were rolled back. * Changed the standard optimizer specification method * changed the specification method for importing the standard optimizer. * change deserialize (#40) Co-authored-by: aramoto99 <[email protected]> * Merge branch 'main' of github:aistairc/aiaccel into feature/database * Fix codestyle * (WIP) Change TPE resume * Fix resume test * Feature/fix resume tpe (#41) * fix resume tpe * fix resume tpe * add if len(running_trials) > 0 * fix about trial number * Fix TPE resumption Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: KanaiYuma-aist <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]> * change serialize timing Co-authored-by: aramoto99 <[email protected]> Co-authored-by: aramoto99 <[email protected]> Co-authored-by: Takashi OKADA <[email protected]> Co-authored-by: Yoshiaki Bando <[email protected]>

aramoto99 and others added 9 commits August 2, 2022 17:43

(WIP) Add dynamic loading of optimizer by importlib.

fd51991

(WIP) Add dynamic loading of optimizer by importlib.

ad3ac34

Add dynamic loading of optimizer by importlib

d26a395

Add dynamic loading of optimizer by importlib

7e6e827

Merge branch 'main' of github:aistairc/aiaccel into feature/optimizer

f400a0e

Fix test code

c4e248e

fix test

fa5ee20

Merge branch 'main' of github_aramoto99:aistairc/aiaccel into feature…

6bf650b

…/optimizer

Fix resumtioin test

01bf954

aramoto99 requested a review from aist-airc-sirt August 10, 2022 05:37

aramoto99 requested review from tkokada and yoshipon August 11, 2022 23:59

yoshipon requested changes Aug 12, 2022

View reviewed changes

aramoto99 added 5 commits August 16, 2022 17:25

Add dynamic loading of optimizer

aa6d71f

Merge branch 'main' of github:aistairc/aiaccel into feature/optimizer

cb213f2

Merge branch 'feature/optimizer' of github:aistairc/aiaccel into feat…

c3f8fc2

…ure/optimizer

Fix dynamic loading of optimizer

616ea56

Function names and comments were rolled back.

de815e7

aramoto99 requested a review from yoshipon August 17, 2022 06:55

Merge branch 'main' of github:aistairc/aiaccel into feature/optimizer

6b3963d

Changed the standard optimizer specification method

67a3172

changed the specification method for importing the standard optimizer.

55fcab8

yoshipon approved these changes Aug 22, 2022

View reviewed changes

yoshipon merged commit 917fd7d into main Aug 22, 2022

yoshipon deleted the feature/optimizer branch August 22, 2022 09:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add dynamic loading of optimizer #34

Add dynamic loading of optimizer #34

aramoto99 commented Aug 10, 2022 •

edited

Loading

github-actions bot commented Aug 10, 2022 •

edited

Loading

yoshipon commented Aug 12, 2022

yoshipon Aug 12, 2022

yoshipon commented Aug 21, 2022

tkokada commented Aug 22, 2022

yoshipon left a comment

Add dynamic loading of optimizer #34

Add dynamic loading of optimizer #34

Conversation

aramoto99 commented Aug 10, 2022 • edited Loading

github-actions bot commented Aug 10, 2022 • edited Loading

Unit Test Results

yoshipon commented Aug 12, 2022

yoshipon Aug 12, 2022

Choose a reason for hiding this comment

yoshipon commented Aug 21, 2022

tkokada commented Aug 22, 2022

yoshipon left a comment

Choose a reason for hiding this comment

aramoto99 commented Aug 10, 2022 •

edited

Loading

github-actions bot commented Aug 10, 2022 •

edited

Loading