Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement basic wrapper to ROOT Minimizer function #13

Closed
beniz opened this issue Apr 10, 2014 · 12 comments
Closed

Implement basic wrapper to ROOT Minimizer function #13

beniz opened this issue Apr 10, 2014 · 12 comments

Comments

@beniz
Copy link
Collaborator

beniz commented Apr 10, 2014

Wrapper to http://root.cern.ch/drupal/

@beniz
Copy link
Collaborator Author

beniz commented May 6, 2014

Work is well under way. A wrapper to libcmaes is being integrated straight into ROOT in order to provide the exact same capabilities today provided by package minuit2.

See pictures below for basic usage output on fitting Gaussians adapted from an existing test file for Minuit2 from ROOT.

CMA-ES:
cmaes_gaus_fit_root

Minuit2:
minuit2_gauss_fit_root

@beniz
Copy link
Collaborator Author

beniz commented May 6, 2014

In order to watch development on integration into ROOT, please watch the following repository:
https://github.com/beniz/root/tree/cmaes4root_master

@loshchil
Copy link

loshchil commented May 6, 2014

Is it correct that standard deviations are 0 for CMA-ES results?

@beniz
Copy link
Collaborator Author

beniz commented May 6, 2014

The errors were not yet connected to the wrapper. They appears to be significantly smaller than the ones returned by Minuit2:
cmaes_gaus_fit_root_errors

@beniz
Copy link
Collaborator Author

beniz commented May 19, 2014

More results from integration into ROOT, with a benchmark fitting 1-D histograms (taken from Minuit2's tutorial files within ROOT):

cmafitbench1

Obviously CMA-ES suffers from heavy computations. sep-CMA-ES does not improve on performances much because the problem is uni-dimensional.

@loshchil
Copy link

The time difference (factor 4) is (more) due to different number of calls to fitness function or internal time complexity per function evaluation?

@beniz
Copy link
Collaborator Author

beniz commented May 19, 2014

I don't know yet but it appears that CMA-ES does not fit the function very well most of the time, so more investigation is needed. On the other side, Minuit2 and Fumili do fit it properly on all trials I've run. I am not controlling the initial parameters yet, so it may be a first direction for investigations. I've ruled out some others.

@loshchil
Copy link

If it is 1-D, then it might be useful to plot objective function in the whole range, say with 10000 points uniformly distributed from xmin to xmax, if possible then to have CMA-ES points from one run depicted with a different color.
Anyway, specifically for 1-D problem we may have at least 2 good candidates: i) golden section method if the problem is unimodal (or one of multiple model-based line-search methods) , ii) STEP algorithm from "STEP: The easiest way to optimize a function" by S Swarzberg, 1994 or from http://www.mat.univie.ac.at/~huyer/LocAlgs.pdf or MATLAB implementation from http://hal.inria.fr/docs/00/82/03/74/PDF/w0306-loshchilov.pdf. I think that STEP is efficient for 1-D multi-modal functions.

@beniz
Copy link
Collaborator Author

beniz commented May 20, 2014

It's actually a 6-D fit over a 1-D histogram. I was able to reproduce outside of ROOT. I've added it as an example on a branch (for now, we may want to merge examples like this one in master later on).

See commit 7a191f9

The example fits a 3-D lorentz peak function + 3-D 'background' function over a set of 201 points (in file lorentzpeakbench.dat). These points were randomly generated with ROOT, and the fit uses a modified least-squared Chi2 function. This is a simplified example from the one plotted yesterday, and I simplified it within ROOT as well so that results exactly match with each others.

To compile this branch and run the example:

git checkout lorentzbench
./configure && make
cd examples
./lorentzpeakbench

The correct minimum, as always found by Minuit, is a Chi2 of 235.355. Re-run multiple times in order to see CMA-ES getting stuck elsewhere or finding the minimum.

I haven't played with CMA-ES parameters yet. This is using the out-of-the-box values, which is fair as this is what Minuit does as well (though the initial values may very certainly differ...).

EDIT: here is a successful run typical output:

best solution: best solution => f-value=235.355 / sigma=0.0026856 / iter=1191 / elaps=89ms / x=  2.57083     2.447   3.64058    20.097 0.0320058  0.999203

EDIT2: visual of typical failed output (from ROOT):
lorentzbenchfail
The reported Chi2 is 1282, which indicates failure, besides the curve.

@beniz
Copy link
Collaborator Author

beniz commented May 20, 2014

I've added parameter control from the command line. Obviously, increasing lambda yields much better success rate, e.g.

./lorentzpeakbench -lambda 1000

appears to succeed most of the time.

@beniz
Copy link
Collaborator Author

beniz commented May 20, 2014

Got better results after fixing the initialization of start point from ROOT, see below:
lorentzbenchgood
sep-CMA-ES fails very often, but CMA-ES is now faster and more accurate than in previous runs.

@beniz
Copy link
Collaborator Author

beniz commented Jun 30, 2014

I consider the basic wrapper is now complete.
https://github.com/beniz/libcmaes/wiki/using-CMA-ES-in-CERN's-ROOT

and new developments, bug fixes and updates are now here:
https://github.com/beniz/root/issues

@beniz beniz closed this as completed Jun 30, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants