New external command: brew-tap #6086

Sharpie · 2011-06-19T19:21:05Z

brew-tap is an external command that brings multi-repo support to Homebrew by making it easy to add alternate repositories, update them, and install the formulae they contain. Currently, the tool is set up to use repositories in the fork network of Adam's homebrew-alt:

https://github.com/adamv/homebrew-alt/network

See the manpage for full usage details:

https://raw.github.com/Sharpie/homebrew/brew-tap/Library/Contributions/manpages/brew-tap.1.md

When creating brew-tap I tried very hard to integrate with existing Homebrew code and style. However, one small change was required to the core file formula.rb in order to enable dependency resolution among formulae in alternate repositories by adding an :alt type to depends_on. This dependency type is ignored by brew, but used by brew-tap. See the HDF-EOS2 formula for an example of :alt usage:

https://github.com/Sharpie/homebrew-science/blob/for-science/scientific/hdf-eos2.rb#L24

I have been using this subcommand for a couple of months and I think it would be a great addition to Homebrew!

mxcl · 2011-07-07T08:43:02Z

This is cool. I'll review. Adam let me know if you have concerns, etc.

Sharpie · 2011-07-07T16:49:48Z

Thanks for taking the time to look at my code Max!

Sharpie · 2011-07-09T22:41:57Z

One thing that just crossed my mind is that I should include some method of flagging kegs as being installed by brew tap install such that brew doctor can pick up on it. That way if someone uses brew-tap to install things we don't provide in the main repo and then those things interfear with later installs, we can see the presence of non-standard software in bug reports.

@adamv, @MikeMcQuaid thoughts?

adamv · 2011-07-10T00:06:45Z

This can be worked into the mechanism Homebrew will use to save options from normal installs, the metadata file. I'm working on it in a branch, but need to rebase it against master; the install-name fixes are causing me a slight conflict due to a change in indent level.

mxcl · 2011-07-10T10:37:19Z

In the spirit of homebrew this kind of metadata should be a new folder and symlinks. I mean the tappedbrews not the options. The options would have to be in some metadata store.

MikeMcQuaid · 2011-07-10T15:17:15Z

I agree with @mxcl. I'm wary of making brew-tap too good an external command as it might limit the likelihood of us making a "proper" integrated solution (or it could be start to be the base for one).

Sharpie · 2011-07-10T15:49:22Z

@mxcl:

I agree that that folders and symlinks would be the best system. However, currently brew tap install just expands formula names into paths and recalls brew install. Getting things to install into separate folders would require re-implementing some of the installation routines with custom versions. These custom versions would have to be re-worked every time the functionality of brew install changes in order to keep things consistent. Perhaps this sort of system would be more appropriate if brewing from alternate repositories moves into the Homebrew core?

Currently I was thinking of something quick and dirty, like touching an INSTALLED-BY-BREW-TAP file inside the keg after brew install finishes execution. Then brew doctor could easily get a list of kegs installed through brew-tap via something like:

Dir["#{HOMEBREW_PREFIX}/**/INSTALLED-BY-BREW-TAP"]

@MikeMcQuaid:

From the beginning of this project I have been hoping that brew-tap could eventually form the basis of multirepo support in the Homebrew core---or at least a good case study for figuring out what works and what doesn't in this situation.

mxcl · 2011-07-30T10:32:29Z

Still integrating this, haven't forgotten, I promise. Max x

Sharpie · 2011-08-04T19:20:17Z

@mxcl

Sounds good, let me know if you have any questions or concerns.

Sharpie · 2011-09-17T01:21:06Z

I'm taking a shot at integrating brew-tap into the core code instead of having it as an external command. Pull request #7643 is tracking the changes.

samueljohn · 2011-09-21T12:23:14Z

Notes from my attempt to install brew tap:

sudo gem install httparty is needed.

   brew tap add samueljohn/homebrew-alt
   Error: No repository named samueljohn/homebrew-alt on the menu!

But I have forked homebrew-alt: https://github.com/samueljohn/homebrew-alt

There I am stuck right now.

Sharpie · 2011-09-22T21:10:43Z

@samueljohn

We hit up the GitHub API to retrieve the Homebrew-alt fork network---this in turn is used to determine the list of available repositories. The weird thing is that you show up if the V2 API is used to retrieve the forks, but not if the V3 API is used---so there must be some GitHub synching issue going on.

I'll send them a support request.

samueljohn · 2011-09-23T12:53:11Z

@Sharpie
Thanks. So I'll just wait. Right now it's still not found.

ashgti · 2011-10-07T20:24:30Z

Is it possible to have a repository that is outside of the Homebrew-alt fork network as a taproom?

Sharpie · 2011-10-07T21:08:14Z

@ashgti

There is no explicit support for it, but you should be able to add one manually:

cd /usr/local/Library/Taproom
git clone <repository> <name>

Then you should be able to install formulae:

brew tap install <name>/<formula>

I would like to make this easier in the future but it is beyond the scope of my current efforts.

docwhat · 2012-01-17T23:53:16Z

Is there anything I can do to help with this effort? I'd like to have homebrew-alt be a first class citizen... :-)

samueljohn · 2012-01-26T20:59:27Z

I am trying brew tap. Works good!

One issue, perhaps:

brew tap search scipy
No formula found for "/usr/local/Library/Taproom/samueljohn-homebrew-alt/other/scipy.rb". Searching open pull requests...

When in fact there is my scipy formula and brew tap install scipy works. And brew tap which also works:

brew tap which scipy
/usr/local/Library/Taproom/samueljohn-homebrew-alt/other/scipy.rb

I added my homebrew-alt for via brew tap add samueljohn. That works like charm.

Sharpie · 2012-01-26T21:08:18Z

@docwhat

Is there anything I can do to help with this effort? I'd like to have homebrew-alt be a first class citizen... :-)

This pull request is essentially complete (but bug reports are welcomed and needed!) and is inteded as a way for people to add multi-repo support to an existing Homebrew installation without much fuss. First-class support is being developed in #7643.

@samueljohn

At this point, there is no implementation for searching. brew tap search just re-calls brew search which has no clue what to do with external formulae.

samueljohn · 2012-01-31T18:51:37Z

@Sharpie, I have another strange issue.
To reproduce/verify: brew tap add samueljohn

Then brew tap install wxpython works but brew tap install numpy does not :-(
The output is: This command requires a formula argument
Both can be found with brew tap which numpy or brew tap which wxpython. Note, they are in the same directory.

What am I doing wrong?

samueljohn · 2012-01-31T18:53:29Z

More:

  brew -v tap install numpy
  Homebrew 0.8.1
  brew install -v
  This command requires a formula argument
  Kernel.exit

wjwwood · 2012-02-24T19:40:17Z

@samueljohn I get the same thing (This command requires a formula argument) if I try to install a formula that is already installed. You can reproduce on mine with:

brew tap add ros-deps
brew tap install ros/fuerte/swig-wx
....Installation stuff...
brew tap install ros/fuerte/swig-wx
This command requires a formula argument

wjwwood · 2012-02-24T19:49:46Z

Also, is there a way to depend on formula in tap?

For example, say formula catkin and swig-wx are in a tap called ros-deps. How can I make catkin depend on swig-wx from ros-deps?

Because:

depends_on 'swig-wx'

Gives me:

Error: No available formula for swig-wx

Is there a different syntax for depending on tap formulae? Something like:

depends_on 'ros-deps/swig-wx'
OR
depends_on 'ros-deps-swig-wx'
OR
depends_on 'swig-wx' => :tap

UPDATE:

I figure this out myself:

depends_on 'ros-deps/swig-wx' => :alt

Works. Sorry for the noise.

wjwwood · 2012-02-29T01:43:38Z

This pull request doesn't seem to apply cleanly on a fresh vm.

To reproduce:

I setup a fresh vm
apple updates
install clt-4.3 (not xcode)
Install homebrew: /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
brew update
brew pull 6086

I get this: https://gist.github.com/1936861

Relavent parts:

error: patch failed: Library/Homebrew/formula.rb:644
error: Library/Homebrew/formula.rb: patch does not apply
Patch failed at 0020 Add new external dependency type `:alt`
When you have resolved this problem run "git am --resolved".
If you would prefer to skip this patch, instead run "git am --skip".
To restore the original branch and stop patching run "git am --abort".
Error: Failure while executing: git am --signoff --whitespace=fix /Users/william/Library/Caches/Homebrew/6086.patch

I think this commit keeps formula.rb from merging automagically: c4027c4#Library/Homebrew/formula.rb

Sharpie · 2012-02-29T03:22:00Z

@wjwwood

Thanks for the report and tracing the problem down---it looks like a simple matter of updating the cross-repo deps code to be compatible with a recent change that added more external deps.

I've been traveling all weekend and am coming down with the flu tonight, but I will try to clear this up tomorrow.

`brew tap` is planned to be a subcommand for "tapping" (checking out) repositories from other "breweries" (GitHub users other than mxcl) and then managing syncs and installs. Initial commit contains an implementation of the Brewery class which is a simple abstraction of a GitHub repository. Brewery instances can query GitHub for information such as branch lists and fork networks through memoized API calls.

New class named Taproom. Instances of taproom represent folders where repositories ("Breweries") are checked out. The list of available repositories is generated from the fork network of a "founding brewery". Adamv's homebrew-alt repository is being used as the prototype founder. The "founding brewery" concept will probably need to be refactored away someday in the name of generalization, but right now it serves as a useful boundary by restricting the list of possible repositories to a well-known quantity. Also added support for what will eventually be the dispatch method of `brew-tap` subcommands. Currently implemented: - list: Prints out a "menu" that lists the available "breweries" - update: Refreshes the menu.

Taproom objects can now report on the breweries they have available and which are on tap (checked out). A Taproom can also place a brewery on tap by cloning it into the Taproom path. Library/Taproom was chosen as the default taproom path which required adding an entry to the master .gitignore file. This could probably be avoided by using HOMEBREW_PREFIX/Taproom but Library/Taproom seems more logical as it is next to the master Formula directory.

Two alternate repositories may provide different formulae for the same software. `brew-tap` will now accepts forumla names of the following form: repo/[subdirectories]/formula Case-sensitive partial matching is also used. For example, the following formula specification: adam/gcc Would cause the formula search to use the following glob: TAPROOM_PATH/adam*/**/gcc.rb Specifying subdirectories also works: adam/dupe/gcc => TAPROOM_PATH/adam*/dup*/**/gcc.rb Both of these globs resolve to the following path: TAPROOM_PATH/adamv-homebrew-alt/duplicates/gcc.rb

Partial matching of brewery names is now case sensitive. This was done for consistancy as the partial matching used for formula names is case sensitive and there is no easy way to make it insensitive.

brew-tap can now loop over each tapped repository and perform an operation similar to brew update. A modified version of the RefreshBrew class is used, so the update operation is summarized in a similar way.

The menu is now separated into tapped and untapped repositories.

Various stylistic and organizational tweaks. Trimmed some unused material.

Made usage more concise and clarified examples in manpage.

The GitHub API now returns `master_branch` in repository summaries. This allows the use of a better, simpler method for discovering the name of this branch.

Small tweak to Homebrew core to allow for a new `:alt` external dependency type. Dependencies flagged as `:alt` will be ignored by Homebrew core commands and so will not raise errors.

`brew-tap` now scans alternate formulae for dependencies of the form: depends_on 'repo/formula' => :alt If `repo` has not been added to the taproom, it is cloned. The original formula along with it's dependencies are then formed into a correctly sorted list for `brew install`

Now that the newest GitHub API offers repository access, brew-tap should use it instead of the V2 API.

The method by which `RefreshBrew` detects changes was improved by: Homebrew/legacy-homebrew@cae8f9e This patch upgrades the version of `RefreshBrew` used by `brew-tap` to be compatible.

Normally `brew-tap` expands all formula arguments into paths to formula files and then recalls `brew`. This is inappropriate for `brew uninstall` so we pass formula names unchanged.

The mechanics of how `brew update` obtains a diff changed which requires a small update to `brew-tap update`.

Unlike version 2, v3 of the GitHub API splits long lists of responses into pages. So, we may need multiple requests in order to retrieve the whole fork network. The code used to locate the "last page" is a bit hacky---but I can't think of anything better at the moment.

Just had my University firewall hose `git:` connections.

Sharpie · 2012-02-29T18:10:42Z

Allright, this should apply cleanly on master again. I did a rebase followed by a force-push, which is something that I have been trying to avoid in this pull request---but there was no other obvious way to straighten this out.

pboling · 2012-03-08T05:27:19Z

+1

adamv · 2012-03-16T21:39:01Z

@mxcl pushed his brew-tap, so closing this

Sharpie mentioned this pull request Oct 12, 2011

Core Change: Multi-repository support #7643

Closed

Sharpie mentioned this pull request Dec 3, 2011

Updating cairo, pango and gtk+ to have --quartz options and --universal. #7658

Closed

Sharpie mentioned this pull request Dec 13, 2011

Third party formulae #9111

Closed

Sharpie added 3 commits February 29, 2012 09:51

Sharpie added 24 commits February 29, 2012 09:51

Implement repository removal

8338370

Make repo matching case-sensitive

845c24d

Partial matching of brewery names is now case sensitive. This was done for consistancy as the partial matching used for formula names is case sensitive and there is no easy way to make it insensitive.

Implemented brew-tap update

4439030

brew-tap can now loop over each tapped repository and perform an operation similar to brew update. A modified version of the RefreshBrew class is used, so the update operation is summarized in a similar way.

Update brew tap list

a9fd16d

The menu is now separated into tapped and untapped repositories.

Rearrange and clean up

fff38d5

Various stylistic and organizational tweaks. Trimmed some unused material.

Added a usage message for brew-tap

d909199

Added manpage for brew-tap

1b01b7a

Updates to brew-tap usage and manpage

92a3a7e

Made usage more concise and clarified examples in manpage.

Fixes, tweaks and more clean up

72ac8e5

Update Brewery master branch discovery

5731a48

The GitHub API now returns `master_branch` in repository summaries. This allows the use of a better, simpler method for discovering the name of this branch.

Added brew tap which command

6c7dc85

Fixes and tweaks

fdf80ca

Add new external dependency type :alt

7342962

Small tweak to Homebrew core to allow for a new `:alt` external dependency type. Dependencies flagged as `:alt` will be ignored by Homebrew core commands and so will not raise errors.

Guard against nil values in external_dependencies

4d2db7c

Update to version 3 of the GitHub API

be8fb50

Now that the newest GitHub API offers repository access, brew-tap should use it instead of the V2 API.

Update some comments

0d48eec

Update brew-tap update

58e4e0c

The method by which `RefreshBrew` detects changes was improved by: Homebrew/legacy-homebrew@cae8f9e This patch upgrades the version of `RefreshBrew` used by `brew-tap` to be compatible.

Handle brew-tap uninstall

9061360

Normally `brew-tap` expands all formula arguments into paths to formula files and then recalls `brew`. This is inappropriate for `brew uninstall` so we pass formula names unchanged.

Update brew-tap manpage

870f5d3

brew-tap update: Fix to restore compatibility

f6c979b

The mechanics of how `brew update` obtains a diff changed which requires a small update to `brew-tap update`.

brew-tap: Use https: instead of git: for checkout

3151a1b

Just had my University firewall hose `git:` connections.

adamv closed this Mar 16, 2012

emiltin mentioned this pull request Jul 2, 2013

problems compiling on mac os x 10.7.1 Project-OSRM/osrm-backend#13

Closed

Homebrew locked and limited conversation to collaborators Feb 16, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New external command: brew-tap #6086

New external command: brew-tap #6086

Sharpie commented Jun 19, 2011

mxcl commented Jul 7, 2011

Sharpie commented Jul 7, 2011

Sharpie commented Jul 9, 2011

adamv commented Jul 10, 2011

mxcl commented Jul 10, 2011

MikeMcQuaid commented Jul 10, 2011

Sharpie commented Jul 10, 2011

mxcl commented Jul 30, 2011

Sharpie commented Aug 4, 2011

Sharpie commented Sep 17, 2011

samueljohn commented Sep 21, 2011

Sharpie commented Sep 22, 2011

samueljohn commented Sep 23, 2011

ashgti commented Oct 7, 2011

Sharpie commented Oct 7, 2011

docwhat commented Jan 17, 2012

samueljohn commented Jan 26, 2012

Sharpie commented Jan 26, 2012

samueljohn commented Jan 31, 2012

samueljohn commented Jan 31, 2012

wjwwood commented Feb 24, 2012

wjwwood commented Feb 24, 2012

wjwwood commented Feb 29, 2012

Sharpie commented Feb 29, 2012

Sharpie commented Feb 29, 2012

pboling commented Mar 8, 2012

adamv commented Mar 16, 2012

New external command: brew-tap #6086

New external command: brew-tap #6086

Conversation

Sharpie commented Jun 19, 2011

mxcl commented Jul 7, 2011

Sharpie commented Jul 7, 2011

Sharpie commented Jul 9, 2011

adamv commented Jul 10, 2011

mxcl commented Jul 10, 2011

MikeMcQuaid commented Jul 10, 2011

Sharpie commented Jul 10, 2011

mxcl commented Jul 30, 2011

Sharpie commented Aug 4, 2011

Sharpie commented Sep 17, 2011

samueljohn commented Sep 21, 2011

Sharpie commented Sep 22, 2011

samueljohn commented Sep 23, 2011

ashgti commented Oct 7, 2011

Sharpie commented Oct 7, 2011

docwhat commented Jan 17, 2012

samueljohn commented Jan 26, 2012

Sharpie commented Jan 26, 2012

samueljohn commented Jan 31, 2012

samueljohn commented Jan 31, 2012

wjwwood commented Feb 24, 2012

wjwwood commented Feb 24, 2012

wjwwood commented Feb 29, 2012

Sharpie commented Feb 29, 2012

Sharpie commented Feb 29, 2012

pboling commented Mar 8, 2012

adamv commented Mar 16, 2012