Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature parameterize obj #144

Merged
merged 14 commits into from
Jun 28, 2018
Merged

Feature parameterize obj #144

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
47bd5d5
Added *args and **kwargs to single
bradahoward Jun 22, 2018
9a4c85a
added *args and **kwargs to binary and base_discrete
bradahoward Jun 22, 2018
1c9a065
update usages of optimzer
bradahoward Jun 22, 2018
971de8b
Added tests
bradahoward Jun 22, 2018
685f12d
addition of example notebook for argument passing
bradahoward Jun 22, 2018
35344e1
minor fixes and some clean up
bradahoward Jun 22, 2018
05ded93
Added logging calls for the arguments being passed
bradahoward Jun 25, 2018
31abd82
cleanup based on flake8 output
bradahoward Jun 25, 2018
86f843d
Revert changes to search and environment
bradahoward Jun 26, 2018
aa71a24
changes based on comments from PR *144
bradahoward Jun 26, 2018
227295e
More fixes for PR #144
bradahoward Jun 26, 2018
f3bb96b
Update of the ipython notebook example
bradahoward Jun 26, 2018
c9f3c42
Address Second round of comments in PR *!44
bradahoward Jun 27, 2018
3695eca
Merge branch 'development' into feature_parameterize_obj
ljvmiranda921 Jun 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
236 changes: 236 additions & 0 deletions examples/basic_optimization_with_arguments.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Basic Optimization with Arguments"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, we will run a basic optimization using an objective function that needs parameterization. We will use the ``single.GBestPSO`` and a version of the rosenbrock function to demonstrate"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:53:06) \n",
"[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]\n"
]
}
],
"source": [
"import sys\n",
"# change directory to access pyswarms\n",
"sys.path.append('../')\n",
"\n",
"print(\"Running Python {}\".format(sys.version))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [],
"source": [
"# import modules\n",
"import numpy as np\n",
"\n",
"# create a parameterized version of the classic Rosenbrock unconstrained optimzation function\n",
"def rosenbrock_with_args(x, a, b, c=0):\n",
"\n",
" f = (a - x[:, 0]) ** 2 + b * (x[:, 1] - x[:, 0] ** 2) ** 2 + c\n",
" return f"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Using Arguments"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Arguments can either be passed in using a tuple or a dictionary, using the ``kwargs={}`` paradigm. First lets optimize the Rosenbrock function using keyword arguments. Note in the definition of the Rosenbrock function above, there were two arguments that need to be passed other than the design variables, and one optional keyword argument, ``a``, ``b``, and ``c``, respectively"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:pyswarms.single.global_best:Arguments Passed to Objective Function: {'c': 0, 'b': 100, 'a': 1}\n",
"INFO:pyswarms.single.global_best:Iteration 1/1000, cost: 1022.9667801907804\n",
"INFO:pyswarms.single.global_best:Iteration 101/1000, cost: 0.0011172801146408992\n",
"INFO:pyswarms.single.global_best:Iteration 201/1000, cost: 7.845605970774126e-07\n",
"INFO:pyswarms.single.global_best:Iteration 301/1000, cost: 1.313503109901238e-09\n",
"INFO:pyswarms.single.global_best:Iteration 401/1000, cost: 5.187079604907219e-10\n",
"INFO:pyswarms.single.global_best:Iteration 501/1000, cost: 1.0115283486088853e-10\n",
"INFO:pyswarms.single.global_best:Iteration 601/1000, cost: 2.329870757208421e-13\n",
"INFO:pyswarms.single.global_best:Iteration 701/1000, cost: 4.826176894160183e-15\n",
"INFO:pyswarms.single.global_best:Iteration 801/1000, cost: 3.125715456651088e-17\n",
"INFO:pyswarms.single.global_best:Iteration 901/1000, cost: 1.4236768129666014e-19\n",
"INFO:pyswarms.single.global_best:================================\n",
"Optimization finished!\n",
"Final cost: 0.0000\n",
"Best value: [0.99999999996210465, 0.9999999999218413]\n",
"\n"
]
}
],
"source": [
"from pyswarms.single.global_best import GlobalBestPSO\n",
"\n",
"# instatiate the optimizer\n",
"x_max = 10 * np.ones(2)\n",
"x_min = -1 * x_max\n",
"bounds = (x_min, x_max)\n",
"options = {'c1': 0.5, 'c2': 0.3, 'w': 0.9}\n",
"optimizer = GlobalBestPSO(n_particles=10, dimensions=2, options=options, bounds=bounds)\n",
"\n",
"# now run the optimization, pass a=1 and b=100 as a tuple assigned to args\n",
"\n",
"cost, pos = optimizer.optimize(rosenbrock_with_args, 1000, print_step=100, verbose=3, a=1, b=100, c=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is also possible to pass a dictionary of key word arguments by using ``**`` decorator when passing the dict"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:pyswarms.single.global_best:Arguments Passed to Objective Function: {'c': 0, 'b': 100.0, 'a': 1.0}\n",
"INFO:pyswarms.single.global_best:Iteration 1/1000, cost: 1.996797703363527e-21\n",
"INFO:pyswarms.single.global_best:Iteration 101/1000, cost: 1.0061676299213387e-24\n",
"INFO:pyswarms.single.global_best:Iteration 201/1000, cost: 4.8140236741112245e-28\n",
"INFO:pyswarms.single.global_best:Iteration 301/1000, cost: 2.879342304056693e-29\n",
"INFO:pyswarms.single.global_best:Iteration 401/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 501/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 601/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 701/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 801/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 901/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:================================\n",
"Optimization finished!\n",
"Final cost: 0.0000\n",
"Best value: [1.0, 1.0]\n",
"\n"
]
}
],
"source": [
"kwargs={\"a\": 1.0, \"b\": 100.0, 'c':0}\n",
"cost, pos = optimizer.optimize(rosenbrock_with_args, 1000, print_step=100, verbose=3, **kwargs)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"Any key word arguments in the objective function can be left out as they will be passed the default as defined in the prototype. Note here, ``c`` is not passed into the function."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:pyswarms.single.global_best:Arguments Passed to Objective Function: {'b': 100, 'a': 1}\n",
"INFO:pyswarms.single.global_best:Iteration 1/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 101/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 201/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 301/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 401/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 501/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 601/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 701/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 801/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:Iteration 901/1000, cost: 0.0\n",
"INFO:pyswarms.single.global_best:================================\n",
"Optimization finished!\n",
"Final cost: 0.0000\n",
"Best value: [1.0, 1.0]\n",
"\n"
]
}
],
"source": [
"cost, pos = optimizer.optimize(rosenbrock_with_args, 1000, print_step=100, verbose=3, a=1, b=100)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python [conda env:anaconda3]",
"language": "python",
"name": "conda-env-anaconda3-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
4 changes: 3 additions & 1 deletion pyswarms/base/base_discrete.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ def _populate_history(self, hist):
self.pos_history.append(hist.position)
self.velocity_history.append(hist.velocity)

def optimize(self, objective_func, iters, print_step=1, verbose=1):
def optimize(self, objective_func, iters, print_step=1, verbose=1, **kwargs):
"""Optimizes the swarm for a number of iterations.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe *args and **kwargs will work here instead of args=() and kwargs={}?


Performs the optimization to evaluate the objective
Expand All @@ -210,6 +210,8 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
amount of steps for printing into console.
verbose : int (the default is 1)
verbosity setting.
kwargs : dict
arguments for objective function

Raises
------
Expand Down
4 changes: 3 additions & 1 deletion pyswarms/base/base_single.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ def _populate_history(self, hist):
self.pos_history.append(hist.position)
self.velocity_history.append(hist.velocity)

def optimize(self, objective_func, iters, print_step=1, verbose=1):
def optimize(self, objective_func, iters, print_step=1, verbose=1, **kwargs):
"""Optimizes the swarm for a number of iterations.

Performs the optimization to evaluate the objective
Expand All @@ -241,6 +241,8 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
amount of steps for printing into console.
verbose : int (the default is 1)
verbosity setting.
kwargs : dict
arguments for objective function

Raises
------
Expand Down
14 changes: 9 additions & 5 deletions pyswarms/discrete/binary.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ def __init__(
# Initialize the topology
self.top = Ring()

def optimize(self, objective_func, iters, print_step=1, verbose=1):
def optimize(self, objective_func, iters, print_step=1, verbose=1,**kwargs):
"""Optimizes the swarm for a number of iterations.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe *args and **kwargs will work here instead of args=() and kwargs={}?


Performs the optimization to evaluate the objective
Expand All @@ -165,17 +165,22 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
amount of steps for printing into console.
verbose : int (the default is 1)
verbosity setting.
kwargs : dict
arguments for objective function

Returns
-------
tuple
the local best cost and the local best position among the
swarm.
"""
cli_print("Arguments Passed to Objective Function: {}".format(kwargs),
verbose, 2, logger=self.logger)

for i in range(iters):
# Compute cost for current position and personal best
self.swarm.current_cost = objective_func(self.swarm.position)
self.swarm.pbest_cost = objective_func(self.swarm.pbest_pos)
self.swarm.current_cost = objective_func(self.swarm.position, **kwargs)
self.swarm.pbest_cost = objective_func(self.swarm.pbest_pos, **kwargs)
self.swarm.pbest_pos, self.swarm.pbest_cost = compute_pbest(
self.swarm
)
Expand All @@ -187,8 +192,7 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
# Print to console
if i % print_step == 0:
cli_print(
"Iteration %s/%s, cost: %s"
% (i + 1, iters, np.min(self.swarm.best_cost)),
"Iteration {}/{}, cost: {}".format(i + 1, iters, np.min(self.swarm.best_cost)),
verbose,
2,
logger=self.logger,
Expand Down
15 changes: 10 additions & 5 deletions pyswarms/single/global_best.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ def __init__(
# Initialize the topology
self.top = Star()

def optimize(self, objective_func, iters, print_step=1, verbose=1):
def optimize(self, objective_func, iters, print_step=1, verbose=1, **kwargs):
"""Optimizes the swarm for a number of iterations.

Performs the optimization to evaluate the objective
Expand All @@ -147,16 +147,22 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
amount of steps for printing into console.
verbose : int (default is 1)
verbosity setting.
kwargs : dict
arguments for the objective function

Returns
-------
tuple
the global best cost and the global best position.
"""

cli_print("Arguments Passed to Objective Function: {}".format(kwargs),
verbose, 2, logger=self.logger)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, keep this. I'd suggest using:

cli_print("Arguments passed to objective function: \nargs: %s \nkwargs: %s\n" % (*args, **kwargs),
                  verbose, 2, logger=self.logger)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that using ** outside of a function call throws an IndexError. The Python documentation around this uses them without the decorators in the outside of the function call.
https://docs.python.org/dev/tutorial/controlflow.html#more-on-defining-functions
Thoughts?

Copy link
Owner

@ljvmiranda921 ljvmiranda921 Jun 27, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops. Yes, you are correct, just remove ** and it should print. One last Python 3.X thing, perhaps a .format() is better?

name = 'ljvmiranda921'
print("Hi my name is: {}".format(name))

I'd probably update my cli_print soon to make everything uniform.

for i in range(iters):
# Compute cost for current position and personal best
self.swarm.current_cost = objective_func(self.swarm.position)
self.swarm.pbest_cost = objective_func(self.swarm.pbest_pos)
self.swarm.current_cost = objective_func(self.swarm.position, **kwargs)
self.swarm.pbest_cost = objective_func(self.swarm.pbest_pos, **kwargs)
self.swarm.pbest_pos, self.swarm.pbest_cost = compute_pbest(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, this is just the modification we need. That is, to provide flexibility in the objective_function by providing an optional *args or **kwargs argument. There's no need to put args = () or kwargs = {} in the main optimize() method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! I fixed the optimize prototypes. The only goofy thing there is the *args had to be listed before the other key word arguments i.e. def fun(foo, bar, *args, baz=3, **kwargs)

self.swarm
)
Expand All @@ -169,8 +175,7 @@ def optimize(self, objective_func, iters, print_step=1, verbose=1):
# Print to console
if i % print_step == 0:
cli_print(
"Iteration %s/%s, cost: %s"
% (i + 1, iters, self.swarm.best_cost),
"Iteration {}/{}, cost: {}".format(i + 1, iters, self.swarm.best_cost),
verbose,
2,
logger=self.logger,
Expand Down
Loading