Ruby Forever

Parallele map with Ruby

Posted on 2 August 2011 Updated on 28 February 2013

With my Intel Core i7 , i want use all 8 cpu …

But manuel Threading can be painfull. So here is a little tool for doing same traitement on a (big) collections of data, using all mys cpu power.

Source Collection can by created dynamicaly. So traitments can begin before the end of collection generation.

A schema :

(thread is oval, closure is rectangle, ==> is ruby queue)

So here is a exemple ( recursive grep )  :

https://gist.github.com/1142278

?

rfind() generate a list of ruby (.rb) file
selectLine() grep one file in memory
reducer() format the result in html

I don’t give the code here, see( http://snippets.dzone.com/posts/show/13263 )

Benchmark

 Here the results, with different ruby implementation and with shell tool :

• Ruby 1.9.2 : 3.6 secs
• IronRuby :  0.90 secs
  
• JRuby:  0.97 secs
  
• find -exec grep : 41 secs
  
• grep -r :    1.6 secs
  

CONCLUSION :

with 8 cores and IronRuby, whe are a little better than 1 core with C program (grep)