POC: Automatically parallelize map(f, arrays...)

[tkf]

Thanks to @Keno's work on purity modeling #43852, we can check the data-race-freedom (DRF) of user-defined function by asserting that it is :effect_free as defined in #43852 (presumably). This already covers a nontrivial interesting subset of DRF functions. (Edit: The direction of this PR seems to be OK but calling an :effect_free method a "DRF function" is wrong: #43910 (comment)) This PR is a POC implementation of automatic parallelization of map(f, arrays...) by detecting that f and getindex are effect-free. For example, map(sin, xs) is parallelized for "well-behaving" array types like xs::Array and xs::UnitRange.

A major obstacle for automatic parallelization is (apart from that we don't have JuliaFolds in Base) that, since map on arrays is used everywhere, we can't add overhead to the workloads that are not "big enough." So, this PR currently uses a rather strict bound for parallelism (the length of the array has to be larger than 2^18 ATM). However, it means that the automatic parallelization kicks in very rarely. For this reason, I'm not entirely sure if this is the right direction. Maybe it's much better to let the user declare that a certain call is "throughput oriented" so that we can increase the parallelism a bit more while still doing the automatic DRF check (so, the user doesn't have to be nervous about introducing data races).

@Keno Is it correct to assume that :effect_free implies DRF (given that the caller does not "touch" (mutate) the returned object)?

[oscardssmith]

To get a better tradeoff, could we @time the first function application and base the decision on that?

[Keno]

@Keno Is it correct to assume that :effect_free implies DRF (given that the caller does not "touch" (mutate) the returned object)?

Well, not in general, a data race is not an effect. You are guaranteed that it won't have the write side of a data race, but it could have the read side. However, if you are only concerned about data races among the different execution of the functions (which seems like you might be for this case, since any other data race would have been there before), I do think you can conclude that the different function executions do not race with each other.

[tkf]

To get a better tradeoff, could we @time the first function application and base the decision on that?

Ooooh, this is interesting. That sounds like a very good idea!

(We can perhaps even do something clever like exponentially increase base case size a la galloping search to get a good measurement or sth like that.)

You are guaranteed that it won't have the write side of a data race, but it could have the read side.

Thanks! Yes, I should've phrased it carefully. I think a more accurate way to say what I wanted to say is that parallel map(f, arrays...) does not introduce new data races if f and getindex are :effect_free. (If the user program was DRF, it stays DRF.)

[giordano]

Was this closed intentionally or it's just an accident of merging #43852?

[vchuravy]

Accidental due to the deletion of the base branch.

@tkf looks like we can't reopen. Mind opening a new PR?