From 374f864c10c67c2f88097e4a104e2ff2a56d71a7 Mon Sep 17 00:00:00 2001 From: quix Date: Sat, 13 Sep 2008 07:50:43 -0400 Subject: [PATCH] new option --rand, randomize task prerequisites. Suggested by Thomas Sawyer. --- README | 36 ++++++++++++++++++++++++++++++++++++ lib/rake.rb | 18 +++++++++++++++--- 2 files changed, 51 insertions(+), 3 deletions(-) diff --git a/README b/README index 191d84354..aa3521c33 100644 --- a/README +++ b/README @@ -75,6 +75,42 @@ other tasks is rather contrary to this notion, throwing a monkey wrench into the system. An exception will be raised when this is attempted in multi-threaded mode. +=== Migrating to -j + +First of all, do you want to bother with -j? If you are +satisfied with your build time, then there is really no reason to use +it. + +If on the other hand your build takes twenty minutes to complete, you +may be interested in investing some time getting the full dependency +tree correct in order to take advantage of multiple CPUs or cores. + +Though is no way for Drake to fathom what you mean by a +correct dependency, there is a tool available which helps you get +closer to saying what you mean. + + % drake --rand[=SEED] + +This will randomize the order in which prerequisites are invoked. +When given the optional SEED integer, it will call srand(SEED) to +produce the same permutation each time. + +Though this may produce an error due to an unspecified dependency, at +least it will be an error which is exactly the same on each run. In +addition, you'll have the major debugging advantage of using a single +thread. + +The randomize option will also work in multi-threaded mode. After +all, once -jN is running smoothly there is still no +guarantee that you have it right. However with each successful +execution of -jN --rand, the probability of correctness gets +closer to 1 (though asymptotically so). + +(The only way to prove correctness is to test all +such permutations, which for any non-trivial project would be +prohibitively large, especially those which meaningfully benefit from +-j.) + == Links * Download: http://rubyforge.org/frs/?group_id=6530 diff --git a/lib/rake.rb b/lib/rake.rb index fab5b8ebb..65d38e367 100755 --- a/lib/rake.rb +++ b/lib/rake.rb @@ -595,6 +595,11 @@ def invoke_with_call_chain(task_args, invocation_chain) # :nodoc: end return if @already_invoked @already_invoked = true + + if application.options.randomize + @prerequisites = @prerequisites.sort_by { rand } + end + prereqs = if application.num_threads == 1 invoke_prerequisites(task_args, new_chain) @@ -602,6 +607,7 @@ def invoke_with_call_chain(task_args, invocation_chain) # :nodoc: else invoke_prerequisites_parallel(task_args, new_chain) end + if needed? if application.num_threads == 1 execute(task_args) @@ -2276,9 +2282,15 @@ def standard_rake_options ['--threads', '-j N', "Specifies the number of threads to run simultaneously.", lambda { |value| self.num_threads = value.to_i } ], - #['--fork', '-k', "When --threads=N given, run each thread in a separate process.", - # lambda { options.fork = true } - #], + ['--rand[=SEED]', "Randomize task prerequisite orders", + lambda { |value| + MultiTask.class_eval { remove_method(:invoke_prerequisites) } + options.randomize = true + if value + srand(value.to_i) + end + } + ], ['--libdir', '-I LIBDIR', "Include LIBDIR in the search path for required modules.", lambda { |value| $:.push(value) } ],