Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching to Pyomo Config System and adding Python API #107

Merged
merged 18 commits into from
Aug 23, 2021

Conversation

bknueven
Copy link
Collaborator

@bknueven bknueven commented Jun 29, 2021

Changes:

  • New Prescient class which will run a prescient simulation in the current python environment using either a user-specified *.txt file or by setting options as kwargs to the simulate method (closes Create a better Python API #105).
  • Deprecates the--traceback and --profile options.
  • Switching all options handling to use the Pyomo Config system; necessitates bumping the minimum Pyomo version to 5.7.2 6.1.2.
  • Extra functions for handling "global" module-level variables (we should probably re-think this at some point)
  • Rewrites the plugin system to use Prescient instance-specific variables instead of module-level variables (thanks @darrylmelander)
  • Tests for the new functionality; better CBC options in tests
  • Using Pyomo fileutils' import_file, removing dependency on pyutilib
  • Better conversion/validation for several options for a better/smoother python interface

@bknueven
Copy link
Collaborator Author

@jsiirola any comments on Prescient's use of the Pyomo config system would be welcome.

prescient/simulator/config.py Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/prescient.py Outdated Show resolved Hide resolved
prescient/simulator/config.py Outdated Show resolved Hide resolved
prescient/simulator/prescient.py Outdated Show resolved Hide resolved
prescient/simulator/prescient.py Outdated Show resolved Hide resolved
prescient/simulator/prescient.py Outdated Show resolved Hide resolved
@bknueven
Copy link
Collaborator Author

bknueven commented Jul 6, 2021

Myself or @darrylmelander have addressed most of @jsiirola's comments except the question of the single pass / double pass parser. I have a few remaining (minor) questions about the user interface for both plugins and the Prescient object.

@bknueven bknueven linked an issue Aug 16, 2021 that may be closed by this pull request
@bknueven
Copy link
Collaborator Author

This needs to wait on resolution to grid-parity-exchange/Egret#243

bknueven and others added 16 commits August 23, 2021 08:07
…el obligated to use any of this commit if you aren't happy with the direction I went with it.

* Each Prescient() instance has its own independent PrescientConfig() instance. It starts with standard options and default values, independent of any options set in a previous Prescient run.

* Plugin registration is triggered by setting the config.plugin value (or values). Registration is handled by the config instance itself, making sure plugin registration gets handled first when setting multiple options in a single function call, whether via a dictionary or via a list of strings. Plugin registration has to come first so that any new options added by the plugin are recognized by the config object.

* Plugins are required to supply a function called "register plugins", which receives a PluginRegistrationContext and a PrescientConfig object as arguments. The PluginRegistrationContext supplies functions the plugin can call to register for callbacks. New command line options can be added to the supplied config object.

* Plugin registration is local to a particular Prescient instance, rather than part of the global state. The plugins active in a particular run are those in the run's Prescient.config.plugin list. There is no need to call reset_plugins() between runs because plugins and plugin registration aren't shared.
* You give each plugin an alias. You use a line of code like `config.plugin.my_alias = 'my_package.plugin'`. This assigns the alias "my_alias" to the plugin, and imports the specified python module. The module can be specified as a python module name, as an actual module object, or as the path to a python file.

* Setting up the plugin alias as described above will give the plugin a chance to supply any configuration settings it uses. The plugin-specific configuration is stored in config.plugin.my_alias.

* You can set up a plugin on the command line (or a config file) using --plugin=alias:module, where alias is the alias that determines where the plugin's configuration is stored, and module is either the module name or the module file path. The plugin may register new command line options, which are made available immediately. Plugins may choose to incorporate the alias into their option names.

* You can also set up a plugin using a dictionary. The alias is put into the dictionary with the key "module".  Other plugin configuation values can be put in the same dictionary.

* Plugins must supply a register_plugins method that registers whatever callbacks it uses. This method called just before the simulation starts.

Here are a couple examples of how you might set up a plugin:

  import my_plugins.pricing
  config.plugin.pricing = my_plugins.pricing
  config.plugin.pricing.option1 = 100
  config.plugin.pricing.option2 = 'fast'

  config.plugin.stats_writer = '../plugins/stats.py'
  config.plugin.stats_writer.write_hourly = True

  options = {'plugin':
                {'ruc_monitor':
                  { 'module': 'my_plugins.ruc',
                    'verbose':True
                  }
                }
             }
  config.set_value(options)
@bknueven bknueven merged commit 00fd061 into grid-parity-exchange:main Aug 23, 2021
@bknueven bknueven deleted the pyomo_config branch August 23, 2021 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a better Python API Refactor Options
3 participants