Same code produces different results in VScode vs terminal

[fabrizioleone]

Hi,

Thanks for this great package.

I am using bboptimize to find the minimum of a non-linear function. My approach is very simple

res = bboptimize(x -> my_obj_fun(x, args); SearchRange = [(0.0, 2.0), (0.0, 2.0)], Method = :adaptive_de_rand_1_bin)

This code produces different results if I run it on VScode or if I call Julia in batch mode from the terminal of my computer. Since everything else is identical (package version, seeds, and objective function) I wonder if bboptimize searches differently depending on how it is called.

Is this a known issue? Is there any way to make sure bboptimize gives the same results in both cases?

Thank you.

[robertfeldt]

Try to set the same random seed on both platforms otherwise the results might differ between runs since these algorithms are stochastic.

[fabrizioleone]

Hi, thanks. I did set the same seeds already. So I don’t think that’s the issue. On 15 Aug 2023, at 10:58, Robert Feldt ***@***.***> wrote:  Try to set the same random seed on both platforms otherwise the results might differ between runs since these algorithms are stochastic. — Reply to this email directly, view it on GitHub<#220 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKTA7G5O3IGPUZUKO22BIALXVM2ZPANCNFSM6AAAAAA2RFJJDY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[robertfeldt]

Hmm, then I don't understand it. If you're on the same machine, same Julia version, and everything else is the same there shouldn't be a difference. Are you sure VSCode is invoking the same Julia binary as when you run from the terminal? Maybe include a call to versioninfo()right before the call to bboptimize? What do you mean "searches differently depending on how it is called"? There is no difference in the code, you mean "from where" it is called, right?

[JobJob]

Maybe init_rng!(params) should be called in bbsetup? Populations get initialized there (which use the PRNG), i.e. seems it should be called before optimizer = optimizer_func(problem, params) here

[robertfeldt]

@JobJob but why would we need to do this within BlackBoxOptim when the user can do it themselves if they need a particular RNG "behavior"? Please clarify, maybe I'm missing something.

[JobJob]

Just if you wanted the same seed to produce the same result, for reproducibility. This is what I assumed would happen if I called bbsetup(f; ..., RandomizeRngSeed=false, RngSeed=42) that I'd get the same result every time, but I don't because of the setup that happens (initial population creation) before the RNG is seeded in init_rng!

[JobJob]

I guess my thinking is: what's the point of providing a RngSeed if it doesn't lead to reproducible results?

[robertfeldt]

Yes, for sure, you have a point. I'm considering skipping the fiddling with the RNG inside of the package though. Hmm, let me revisit the design around this, not sure why we introduced that in the first place... ;)

[JobJob]

Yeah I think that makes sense to remove all the RNG seeding stuff completely.

…

________________________________ From: Robert Feldt ***@***.***> Sent: Thursday, December 21, 2023 12:11:11 AM To: robertfeldt/BlackBoxOptim.jl ***@***.***> Cc: Joel Mason ***@***.***>; Mention ***@***.***> Subject: Re: [robertfeldt/BlackBoxOptim.jl] Same code produces different results in VScode vs terminal (Issue #220) Yes, for sure, you have a point. I'm considering skipping the fiddling with the RNG inside of the package though. Hmm, let me revisit the design around this, not sure why we introduced that in the first place... ;) — Reply to this email directly, view it on GitHub<#220 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAGQRJ2VM52FZM2YHNGIXU3YKNO77AVCNFSM6AAAAAA2RFJJD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGIYTCNZQGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[robertfeldt]

Ok, now changed on master:

f3b8a2b