RFC: Make MersenneTwister equivalent in 32- and 64-bit architectures

[jperla]

As mentioned in issue #5999, reproduce the solution in 67f6fee for the MersenneTwister.

Note that users on 64-bit architectures will see their randomness change with no change in their seed after this commit.

[JeffBezanson]

Like @StefanKarpinski said before, this test doesn't seem to make sense. Where does the .01 value come from? Why would it be ok to get a value different by .001 but not by .1?

[jperla]

@JeffBezanson I misread that, thanks. Updated

[andreasnoack]

I think it is sufficient with the last line only if you use default value, i.e. MersenneTwister(seed=uint(0)).

[StefanKarpinski]

In fact, you don't even need the uint because the whole point of this change is that only the value of the seed should matter, not it's type.

[jperla]

Julia is indeed quite powerful. Updated.

[StefanKarpinski]

Isn't that only the case if they were invoking the MersenneTwister constructor directly? I suspect that is not a terribly common case. Although arguably, we should insist on explicitly choosing an RNG whenever setting an RNG seed. Otherwise we're locked into using MersenneTwister exactly this way forever.

[jperla]

@StefanKarpinski Okay so you're suggesting I put up another diff that requires srand to receive a concrete RNG rather than just an integer seed?

If the explicit MersenneTwister use is not common today, is this diff good to go then?

[StefanKarpinski]

That's a different design issue, but yes, that's the idea. The principle being that there are two basic ways to use random numbers:

1. need reproducibility
2. don't need reproducibility

In the former case, the system should be free to choose and RNG for you and generate a seed with good entropy. In the latter case, the user should provide enough information to unambiguously specify the random stream – which entails not only a seed value, but also an RNG to use. As it is, we can't change our default RNG without breaking programs that need reproducibility. If people are required to do that, then we will have future-proofed programs that need reproducibility against changes in the default system RNG.

[JeffBezanson]

If you can set a different RNG globally, then we risk rand() having to do a dynamic dispatch in order to use the current RNG.

[StefanKarpinski]

Ah, that's a fair point. It's the default output stream problem all over again and we know how that turned out. I guess that means that we're permanently stuck with MT as the default RNG and anyone who wants to use anything else will have to pass the RNG object around explicitly.

[ivarne]

Why are we stuck with MT as default? If LLVM decides to give access to the random instruction on some CPUs, wouldn't it be expected that we enable usage. Lots of benchmarks rely on fast random numbers, and for most applications we do not care if NSA or FSB or China knows something about the numbers. A compile time (make variable) should at least be possible without dynamic dispatch.

[StefanKarpinski]

Because if someone uses srand, then they expect a particular sequence of reproducibly pseudo-random values, which depends on both the seed and the RNG. What Jeff is saying is that we cannot allow changing default RNGs without a big performance hit. Thus, if we change the default RNG, we'll break any code that uses srand and expects a particular sequence of values. Since we also can't change default RNG based on whether someone called srand or not, we're stuck.

[ivarne]

Sorry, I thought the idea was to deprecate srand() and define an abstract RNG. Implementations of RNG will probably be seeded in the constructor. The random family of functions can then have a optional first argument to specify what RNG to use, but provide a sensible automatically seeded default like the IO functions do now.

[StefanKarpinski]

That approach would leave only two modes of operation:

1. non-reproducible: use whatever the default RNG is automatically seeded with real randomness; you call rand et al. without explicitly passing an RNG.
2. reproducible: explicitly pick an RNG and seed it upon construction; you have to pass the RNG explicitly to rand et al. whenever you're generating reproducible pseudo-random values.

Although, in some sense this is the "correct" API, I'm not sure that this is acceptable from a UX standpoint. People want to be able to just call rand without passing around an explicit RNG yet be able to get predictable pseudo-random values by calling srand first.

[pao]

Although, in some sense this is the "correct" API, I'm not sure that this is acceptable from a UX standpoint.

As a point of reference, MATLAB provides the RandStream API, which is (2), and provides independent instances of RNG state; they also implemented a New and Improved global RNG setting API with the rng() function, which is roughly equivalent to srand() and sets the global RNG state. MATLAB does not implement (1); on startup the global RNG state is always consistently seeded (MT with seed 0).

[JeffBezanson]

Selecting an RNG at build time might be a viable option for both fast and reproducible numbers.

[ivarne]

If julia is going to be used for cryptography in web applications, we should not have an API that force us to never change RNG implementation. The suggested api is also very similar to the IO API, so I would consider the approach to be Julian.

I can see that it might be a hassle to pass a RNG around in a random reproducible walk simulation, but it is fairly easy to do

const rng = MT(5)
my_random() = random(rng)

I'm often better off when I explicitly use a global, than when that API hides it for me.

[StefanKarpinski]

Right, but the issue is that peoples programs will break if they use the wrong build option. The only way to make that future proof is to require them to pass in the type of RNG they want to use and raise an error (or warning?) if it's not what their Julia has as its built-in default.

[ViralBShah]

The thing is that most users do not know anything about RNGs or MersenneTwister and just want to do rand. For crypto applications and such, we may need a separate package that meets those needs.

Do see: https://github.com/SamChill/RdRand.jl

[StefanKarpinski]

This argument has nothing to do with crypto, IMO. It has to do with reproducibility of pseudo-random sequences. It's entirely plausible that someone will figure out something really wrong with MT in the next five years and yet we could be stuck with it as a default. I suppose that could be handled by major language version changes (e.g. v1 => v2) and sufficient documentation.

[ViralBShah]

Yes - that is what I was going to say as well - we should handle it along with a major number change. The tradeoff is burdening every user with the details vs. having reproducibility across major releases.

Even now, we can document that if you want guaranteed reproducibility with srand, specify the RNG type, without which there are no guarantees.

[andreasnoack]

I don't know what is the right solution here, but maybe it can be useful to have MATLAB's unfortunate state in mind. In order to have backward reproducibility rand('seed',123) changes the RNG to the bad legacy version. In consequence many users end up with a worse RNG than necessary.

[pao]

Interaction between the old MATLAB API and new API:

>> rand('seed', 123)
>> rng(0)
Error using rng (line 96)
The current random number generator is the legacy generator.  This is because you have
executed a command such as rand('state',0), which activates MATLAB's legacy random number
behavior.  You may not use RNG to reseed the legacy random number generator.

Use rng('default') to reinitialize the random number generator to its startup configuration,
or call RNG using a specific generator type, such as rng(seed,'twister').

[JeffBezanson]

We could put const default_rng = MT() in Base now, and if this changes in the future people can get their old random sequences by editing this line and rebuilding. I think that's reasonable to ask, given that you're not then forced to use an old version, or rewrite your code, or have bad performance.

[JeffBezanson]

In the mean time this PR seems totally reasonable; really just cleaning up a couple silly definitions. Everybody ok with merging it?