rbenv-bundler not caching bundle installation paths

[cespare]

rbenv-bundler isn't working for me at all. I'm 99% sure everything is installed correctly (rbenv bundler shows the usage message, for instance).

I traced the code and I can see that the $cached_dirs_file referred to in etc/rbenv.d/rehash/bundler.bash is created but remains empty. It looks like the only way for entries to be put in that file (i.e. the only place that things are added to $cached_dirs in this script) is the following (lines 51-53 of etc/rbenv.d/rehash/bundler.bash):

if { get_bundle_path "$RBENV_DIR" > /dev/null; } then
    cached_dirs="${cached_dirs}${RBENV_DIR}"$'\n'
fi

However, the get_bundle_path function fails unless ${1}/.bundle/config exists. Hence, it doesn't seem like rbenv-bundler will do anything unless $RBENV_DIR/.bundle/config exists. I don't have such a file. (On my machine, that would be ~/.rbenv/.bundle/config. I don't have a .bundle directory in my rbenv installation folder. Should I?)

Edit: Apologies for not including code; I couldn't figure out the intended behavior or I would have tried to fix it and make a pull request.

[carsomyr]

What happens when you do your usual Bundler thing in a project and then rbenv rehash? Does the $cached_dirs_file change then?

[cespare]

That's the scenario I was trying to describe. $cached_dirs_file (which is ~/.rbenv/plugins/bundler/share/rbenv/bundler/paths on my box) stays empty after running bundle install and rbenv rehash.

[carsomyr]

What does your Bundler-controlled project's .bundle/config file look like?

[cespare]

It doesn't have a .bundle/config. I'm not aware of a scenario in which the .bundle directory is necessarily created.

[carsomyr]

It's created when you type bundle install --path ${your_path}. Please sanity check that when you type bundle install that the gems are being installed into your system Ruby?

[cespare]

But I don't use --path, I install globally. (I run precisely the command bundle install). According to #7, rbenv-bundler should still work.

I verified that all the versions of gems that are installed when I type bundle install show up in gem list.

[carsomyr]

Did you upgrade Bundler itself, like in #7?

[cespare]

Oh yes, I meant to put that in the original bug description but I forgot. I'm using ruby 1.9.2-p290 and bundler 1.1.3, which I believe is the latest. I reinstalled bundler today and that was the version I got.

[carsomyr]

Ok, after thinking about it for a while, why should the cached paths file be modified? rbenv-bundler knows where to look already (excerpted from includes.sh):

shopt -s -- nullglob \
    && bundled_executables=("$bundle_path"/ruby/*/bin/"$binary_name") \
    ; shopt -u -- nullglob

Could you give an example where you install a system-level gem and it's not picked up by rbenv-bundler?

[cespare]

Upon more investigation, this is actually the same issue as #7. However, updating to the very latest version of bundler doesn't fix the issue. I tried both the latest stable (1.1.3) and also --pre (1.1.rc.8).

Here's a console transcript so I won't be ambiguous in my description of what I'm seeing:

$ gem list | cut -d" " -f1 | xargs gem uninstall -aIx
<...uninstall lots of gems...>
$ gem list

*** LOCAL GEMS ***


$ gem install bundler --pre
Fetching: bundler-1.1.rc.8.gem (100%)
Successfully installed bundler-1.1.rc.8
1 gem installed
$ gem install rake
Fetching: rake-0.9.2.2.gem (100%)
Successfully installed rake-0.9.2.2
1 gem installed
$ cat Gemfile
source :rubygems
gem "rake", "~> 0.8.0"
$ bundle install
Fetching gem metadata from http://rubygems.org/..
Installing rake (0.8.7)
Using bundler (1.1.rc.8)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
$ gem list

*** LOCAL GEMS ***

bundler (1.1.rc.8)
rake (0.9.2.2, 0.8.7)
$ rbenv rehash
$ rake -V
rake, version 0.9.2.2
$ bundle exec rake -V
rake, version 0.8.7
$ bundle show rake
/home/caleb/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/rake-0.8.7
$ rbenv which rake
/home/caleb/.rbenv/versions/1.9.2-p290/bin/rake

[carsomyr]

Alright, here's my theory.

Issue #7 still doesn't apply here. The reason why you have been seeing this problem is that the current implementation of rbenv-bundler searches for a Bundler path. If not available, it simply quits and yields control back to the vanilla rbenv-which command, which in turn picks up something like

/home/caleb/.rbenv/versions/1.9.2-p290/bin/rake

and not the version specified in the Gemfile, which is

/home/caleb/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/rake-0.8.7

Since there's no Gemfile processing, how can it? I believe the correct behavior is to consult the Gemfile versions for gems. Would you care to submit a patch? The specification is that, if the Bundler path doesn't exist, execute bundle show <command> to figure out which version is truly the desired one. It's hacky, but I think it covers all cases.

[cespare]

Sure, I'll write up a patch to do this tomorrow.

What is the use case for bundler where rbenv-bundler does work? (i.e. how does this "Bundler path" get created?) In any case, wouldn't bundle show always work? What I mean is, instead of checking the bundle path, couldn't we always just run bundle show and delegate back to rbenv if bundler doesn't find the gem in the current bundle?

[carsomyr]

Oh boy, now that I think of the issues, I don't know how deep this rabbit hole goes anymore.

My proposal above is to separate rbenv-bundler's behavior into two cases:

• With --path specified. Gems are installed in a designated Bundler path. Finding their executables isn't an ambiguous business anymore, since Bundler and only Bundler uses said path. I originally designed rbenv-bundler around this case, and your bug should, in theory, not manifest itself here.
• Without --path specified. The system Ruby paths are used, which introduces ambiguity like what you observed above. As you can see, we'll have to invoke bundler show <command> here for disambiguation.

As you can see, the above proposal is quite hacky and unsatisfactory. A more permanent solution might be to do lots of Bundler gem inspection on rbenv rehash and record the locations of gem executables. Note that bundler show <command> every time might be too slow and a deal killer for many; however, if we cache the gem information for each visited project, it might all work out. That itself introduces the problem of when the cache should be invalidated whenever the user upgrades or uninstalls gems.

[developish]

I would love to see rbenv-bundle work without --path specified. I think I'm using rbenv in nearly the same way as @cespare is, so I'll test your fix on my system right away and let you know how it goes.

Great work @carsomyr, I think that rbenv and bundler together (not gemsets) is on the path to the "simple" Ruby ecosystem of yesteryear.

[carsomyr]

@developish, looking forward to a patch!

[developish]

Only if @cespare doesn't get to it first!

[carsomyr]

Alright, I committed a fix in the rehash branch. Could you gentlemen try it out with your use cases and report back?

[cespare]

Doesn't seem to work for me. Should the paths and manifests files have entries in them after I run bundle install and/or rbenv rehash?

[carsomyr]

You are supposed to see ~/.rbenv/plugins/bundler/share/rbenv/bundler/manifest as well as mangled pathnames where the /s are replaced with _s.

[carsomyr]

Also, make sure you visit a Bundler-controlled directory and run rbenv rehash.

[developish]

I just checked out the rehash branch and ran rbenv rehash and got this output.

cat: /Users/brandon/.rbenv/plugins/bundler/share/rbenv/bundler/manifest: No such file or directory
cat: /Users/brandon/.rbenv/plugins/bundler/share/rbenv/bundler/manifest: No such file or directory
cat: /Users/brandon/.rbenv/plugins/bundler/share/rbenv/bundler/manifest: No such file or directory
cat: /Users/brandon/.rbenv/plugins/bundler/share/rbenv/bundler/manifest: No such file or directory
/Users/brandon/.rbenv/plugins/bundler/etc/rbenv.d/bundler/rehash.rb:80:in `rehash': The output directory does not exist (RuntimeError)
    from /Users/brandon/.rbenv/plugins/bundler/etc/rbenv.d/bundler/rehash.rb:175:in `<main>'

[carsomyr]

Try running rbenv rehash in a Bundler-controlled directory; and yes, you found a bug.

[developish]

The initialization process is fixed! And it works (mostly)!

Nice touch with the manifest files, I expected to have to rbenv rehash anytime I changed into a directory that used this, but coming back and having it just work is fantastic.

It works with some of my existing projects, but one of my newer ones doesn't, and also newly created projects.

To test this:

rails new testapp
cd testapp
bundle
rbenv rehash
grep 'testapp' ~/.rbenv/plugins/bundler/share/rbenv/bundler/manifest

should return the entries in the manifest file, but returns empty.

[carsomyr]

@developish, you'll have to type rbenv rehash from within new projects to register them with rbenv-bundler. As for the testapp issue, why should grep output anything? If you don't specify the --path, everything should be installed to the rbenv Ruby's gem directory. That would mean the string testapp doesn't appear. Am I misunderstanding?

[carsomyr]

Alright guys, speak up about problems soon, because I'm fast-forwarding master tomorrow.

[cespare]

This just doesn't work for me. I blew away the ~/.rbenv/plugins/bundler/ folder and re-cloned, checked out the rehash branch, and opened a new terminal. Then I went into a bundler-controlled project directory and re-ran bundle install and rbenv rehash. I don't get the error that I had before (same thing that @developish was getting, which was fixed), but I'm still not getting the behavior I was expecting:

$ rake -V
rake, version 0.9.2.2
$ bundle exec rake -V
rake, version 0.8.7

In ~/.rbenv/plugins/bundler/share/rbenv/bundler/, the only file is manifest, which is empty.

I'll try to have another go at debugging this myself when I get a chance, but I'm pretty busy at the moment and getting this working isn't high on my own priority list.

[carsomyr]

Alright, I think I figured out the problem: rbenv-bundler uses the incorrect assumption that the .rbenv-version file and the Gemfile sit in the same directory. Try changing into the Gemfile's directory and typing rbenv local <version>, and then try the rake test?

[cespare]

@carsomyr Nice, that is indeed the case! It works as expected after setting a local rbenv version. I assume it's not a difficult fix to look for the Gemfile and .rbenv-version files in the proper locations?

(Good job guessing at the problem btw.)

[carsomyr]

Yeah, and it was a silly assumption to make, in retrospect.

[carsomyr]

@cespare, @developish, this issue should be closed for good. Try testing the plugin again, this time with .rbenv-version files omitted or in parent directories relative to Gemfiles.