Add ability to specify the url path to code in dependencies

[ilya-stromberg]

Please, add ability to specify the url path to code in dependencies.
Today we can specify the local path:

  "dependencies": {
      "mysql-native": {"version": "~master", "path": "path/to/package"}
  },

Please add url support, for example like this:

 {"version": "~master", "https": "https://github.com/rejectedsoftware/mysql-native.git"}

or this:

 {"version": "~master", "url": "https://github.com/rejectedsoftware/mysql-native.git"}

It can help if I fork project but don't add it to the dub database.

[s-ludwig]

See also #50

I'm still not sure if this really is a good idea in general, although it is easy to implement. The approach I'm using for this is to check out all my non-public packages to a specific workspace folder and then do "dub add-path path/to/workspace" ("dub add-local ..." for each individual package as mentioned in the other ticket will work as well). It will then automatically find all packages there and no URLs or paths have to be hard-coded (which can be sub-optimal since for example URLs may vary depending on authentication requirements).

[ilya-stromberg]

I don't talk about private repository, but this addition also can be useful.
Live example: I want to use vibe.d and hibernated. Is it possible? In current situation - no, it isn't.
hibernated required ddbc, ddbc required "mysql-native": "==0.0.9", but current version of vibe.d required "mysql-native": ">=0.0.10". Also, hibernated has got a few compiler errors - it's easy to fix, but I have to do that. The latest update was 2 months ago and it looks like it haven't got support.
So, what should I do? Fork it, fix dependencies in ddbc, fix errors in hibernated, specify to use local the local code and push all code to github. maybe I will make a pull request for the changes or register new dub project in public package registry, but now I want to compile my code. Also, register new dub project every time for 2 changed lines is terrible idea, so I don't want to do it.
Now I must to install this software to the production server (different from my computer for work). I have to clone changed ddbc and hibernated, and only after that I can use dub. Now assume that there are 2-5 developers and 2-3 servers. It's easy to make a mistake, isn't it?
The dub's philosophy is to keep things as simple as possible, so let's do it.

[s-ludwig]

OK, convinced. In that situation it does indeed provide some solid benefits. My doubts mostly come from providing too many possibilities to achieve similar things, possibly resulting in the wrong tool being used for a specific task and generally increasing the system's complexity. But it seems to be useful enough for such public forks to justify that.

[ilya-stromberg]

Please, add more documentation and examples. Current http://code.dlang.org/package-format page only lists available options, and it's difficult to understand what exactly I need to use I my situation. Examples should help to decide what should I do.
Also, please add link http://code.dlang.org/ to the D home page http://dlang.org/ - I think it can help to provide more dub projects and simplify search of D code.

[ilya-stromberg]

It's better to add git like identifier, not url:

{"version": "~master", "git": "https://github.com/rejectedsoftware/mysql-native.git"}

In future we can easy add "svn" or "hg" identifiers for Subversion or Mercurial.

[jcrapuchettes]

👍

[jacob-carlborg]

Is there any status on this?

[timotheecour]

@s-ludwig I would really like to have this feature; any help would be greatly appreciated; eg, where would code need to be changed for this?
Thanks!

[s-ludwig]

Off the top of my head:

• Dependency needs to get a new url property of type URL - including update of fromJson and toJson
• The (de)serialization logic for Dependency in dub/recipe/sdl.d needs to be updated as well
• It would probably make sense to lower the Dependency.path property to Dependency.url (i.e. as "file://"~path) to be able to unify the high-level code
• dependencyToJson and dependencyFromJson of SelectedVersions need to be updated as well. Path based dependencies should continue to be output as "path": "..." for backwards compatibility
• Package manager needs to get overloads that operate on URLs in addition to Paths and needs a chaching/cloning logic for non-"file://" URLs

There are probably a few places where currently the .path property is used instead of .url that also need to be changed (e.g. Project.collectDependencies). But my hope (and I might overlook something) is that PackageManager is the only place where the real change needs to happen.

[timotheecour]

is anyone planning to work on this?

[Geod24]

I made a slightly different proposal at #1840 .
The idea is to support git submodule without having the URL added explicitly to the manifest file.
The reason not to add the URL explicitly is that we don't want packages on the registry using URL dependencies (feel free to argue on the issue, that's why it's there).

I've been using git submodules + path-based dependencies for a few years now, for the same reason mentioned here: the ability to override packages as necessary, or to even switch to a different provider for a library (we use a fork of vibe.d for example).

The proposition is however more tied to git and does not support SVN or Mercurial, as asked by #104 (comment) , but there is currently no support for either tool.

[Geod24]

Done, we can now use repository.