Installation fails when the extension is cached

[haines]

Our CI builds are failing because the generated lib/mimemagic/path.rb file is missing, but only when the mimemagic extension has been cached by Bundler. A fresh install always succeeds.

Here's a minimal reproduction:

$ docker run --rm -it -w /mimemagic ruby:alpine sh

# apk add build-base shared-mime-info

# export BUNDLE_GLOBAL_GEM_CACHE="true"

# export BUNDLE_PATH="vendor/bundle"

# cat >Gemfile <<EOF
source "https://rubygems.org"
gem "mimemagic", "0.3.7"
gem "rake"
EOF

# bundle install

# ls vendor/bundle/ruby/3.0.0/gems/mimemagic-0.3.7/lib/mimemagic
overlay.rb  path.rb     tables.rb   version.rb

# rm -r vendor

# bundle install

# ls vendor/bundle/ruby/3.0.0/gems/mimemagic-0.3.7/lib/mimemagic
overlay.rb  tables.rb   version.rb

# rm -r vendor

# rm -r ~/.bundle/cache/extensions/x86_64-linux-musl/ruby/3.0.0/rubygems.org.443.*/mimemagic-0.3.7

# bundle install

# ls vendor/bundle/ruby/3.0.0/gems/mimemagic-0.3.7/lib/mimemagic
overlay.rb  path.rb     tables.rb   version.rb

As you can see, path.rb is

• created on the fresh install
• not created after reinstalling with the extension cached
• created again after deleting the extension cache and reinstalling

Bundler only caches the gem's extension directory, meaning that writing to lib in the extension build is problematic:
https://github.com/rubygems/rubygems/blob/dcf7c3f5def6078f5e9c48a8e345135cdb7f1bb5/bundler/lib/bundler/rubygems_gem_installer.rb#L78

[ziggythehamster]

I'm not 100% sure you can drop a .rb file in ext/ and require it the same way, but that would be one solution.

Another would be this stupid hack, if it were to write the file out and then compile it:

#include "ruby.h"

VALUE cMimeMagic;

void Init_MimeMagic() {
  cMimeMagic = rb_define_class("MimeMagic", rb_cObject);
  rb_define_const(cMimeMagic, "DATABASE_PATH", rb_str_new_cstr("plug the path in here"));
}

Probably would be a good idea to ensure that the path doesn't contain the double quote character or anything that would be a C escape before interpolating it in :).

[haines]

I'm not 100% sure you can drop a .rb file in ext/ and require it the same way, but that would be one solution.

You can - as long as you add ext to the require_paths in the gemspec. I've implemented that in #132 🎉

Update: spoke too soon... although it works from the require standpoint, it's still not included in the cache - the extension directory that bundler is caching is actually the one defined here, where the compiled .so/.bundle/whatever objects would be placed when compiling a C extension (e.g. extensions/x86_64-darwin-19/2.7.0-static/mimemagic-0.3.9)

[haines]

Ok, take two - I worked out where Rubygems expects the compiled files to be written (it passes the destination path as the RUBYARCHDIR environment variable to Rake), and #136 writes to that directory instead.

Note that this directory is automatically added to the load path, so with this approach we don't have to modify require_paths in the gemspec.