Microgem for compiling assets in parallel

parallel_assets_compiler.gemspec

# -*- encoding: utf-8 -*-

Gem::Specification.new do |s|
  s.name        = 'parallel_assets_compiler'
  s.version     = '0.2.0'
  s.platform    = Gem::Platform::RUBY
  s.author      = 'Jørgen Orehøj Erichsen'
  s.email       = '[email protected]'
  s.summary     = 'Compile assets in parallel'
  s.description = 'Compile assets in parallel to speed up deployment'

  s.files         = ['parallel_assets_compiler.rb']
  s.require_path  = '.'

  s.add_dependency('parallel')
  s.add_dependency('actionpack')
end

parallel_assets_compiler.rb

# Compile assets in parallel to speed up deployment
#
# Works by monkey patching actionpack from Rails 3.2.5
#
# Use it by adding this to your Gemfile and run bundle install
#
# gem 'parallel_assets_compiler', :git => 'git://gist.github.com/2873091.git'
#
# Inspired by
# https://github.com/steel/sprockets/commit/6327afc3341e34efd1dfdcfad08e3b9d85f7fd4a
# https://github.com/hornairs/sprockets-rails-parallel

require 'sprockets/static_compiler'
require 'parallel'

module Sprockets
  class StaticCompiler

    def compile
      # Collect paths
      logical_paths = []
      env.each_logical_path do |logical_path| 
        if File.basename(logical_path)[/[^\.]+/, 0] == 'index'
          logical_path.sub!(/\/index\./, '.')
        end
      	logical_paths << logical_path if compile_path?(logical_path)
      end

      # Compute!
      results = Parallel.map(logical_paths) do |logical_path|
        if asset = env.find_asset(logical_path)
          [logical_path, write_asset(asset)]
        end
      end

      write_manifest(Hash[results]) if @manifest
    end

  end
end

Nice dude.@grosser did a great job on the parallel gem. I didn't know about this or the other two works you reference in the comment above.

One small issue here: when I did a test run of this, my manifest.yml was a empty hash. Since the Parallel.map block is executed within each worker fork, it'll be using the work process' memory as opposed to the parent process, so the mutations to the manifest are effectively ignored. Parallel.map is a great piece of awesome since it'll handle marshalling the return value of the block back to the parent process via pipes. See my gist for what I did: https://gist.github.com/3221581

Duh, you're right about the empty manifest file - thx :-)
I've changed my version based on your suggestions and bumped the version to 0.2.0.

Sweet. Nice shortcut using Hash[] - hadn't seen that before.

Cut my assets compilation time from 1m13secs to 53 secs on a Mac i5 Dual SSD machine.

And from 95.5 secs to 57.3 secs on Heroku. That's awesome. Thanks @joerichsen for the awesome gem and @brendtd for the correction

Undumpable Exception -- #<NoMethodError: undefined method `options=' for true:TrueClass>
/Users/kain/.rvm/gems/ruby-1.9.3-p194/gems/parallel-0.5.18/lib/parallel.rb:173:in `work_in_processes'
/Users/kain/.rvm/gems/ruby-1.9.3-p194/gems/parallel-0.5.18/lib/parallel.rb:55:in `map'

same issue Undumpable Exception -- #<NoMethodError: undefined method `options=' for true:TrueClass>

I get this w/ sprockets 2.2.2:

undefined method `compile_path?' for #<Sprockets::StaticCompiler:0x007fa14f4fc048>
/Users/user/bundle/ruby/1.9.1/bundler/gems/2873091-2e1e123a0036/parallel_assets_compiler.rb:26:in `block in compile'

Looks like that method may have been removed from actionpack? I tried this one from turbo-sprockets-rails3:

      private
      def compile_path?(logical_path)
        paths.each do |path|
          case path
          when Regexp
            return true if path.match(logical_path)
          when Proc
            return true if path.call(logical_path)
          else
            return true if File.fnmatch(path.to_s, logical_path)
          end
        end
        false
      end

It worked fine on my dev box, but on my build agent I got the Undumpable Exception :(

Make it a real gem!

Doesn't work with compile_path? error.

Make it a real gem!