Reverse Dependencies indexed on PackageName

[ysangkok]

This is #723, but rebased and rewritten to use PackageNames in its Graph instead of PackageIdentifiers.

It also includes tests and a benchmark. I have changed various page names to have more consistent naming, but the core UI is the same. These are the new paths, they all serve HTML. Note that I don't have the full Hackage package set mirrored, so the screenshots do not reflect exactly how it would look like if deployed.

/package/:package/reverse

/package/:package/reverse/old

/package/:package/reverse/flat

/package/:package/reverse/verbose

This page can be really expensive to generate (like tens of seconds for a popular package like containers). That's why I limited it to only show the oldest 10 versions, and I was thinking that if the CDN is configured to aggressively cache this for like a week, this could still be useful.

/packages/reverse

screenshot

(screenshot doesn't show it, but there is a version number in the vertical middle of the left table field in the lower table)

Given that this runs on all packages, it is also expensive to generate for the full package set of 16k packages. But since it only deals with counts, and doesn't actually look up the packages, it is way faster than the /verbose endpoint. This could also be aggressively cached.

There is also a graph page which I haven't included here since it is identical to the previous PR (implemented by revJSON).

Looking forward to any kind of feedback, since I am not really sure where to prioritize at this point.

EDIT: Added strike-through on sections referring to the slow /verbose endpoint as the code is now much faster.

[andreasabel]

Thanks for making a move on this @ysangkok !

Looking at the screenshots, I would like that all quantitative information is communicated via tables, rather than its loud-reading, simply because one can then get the information at one glance. E.g.

Packages depending on containers

total	on latest version	on older versions
42	38	4

A legend could be given as a tool-tip (hovering) if the table isn't self-explanatory.

[ysangkok]

Good suggestion, @andreasabel , thank you. I have implemented your suggestions in commit b168fb6.

If anyone would like new screenshots, please let me know.

[andreasabel]

If anyone would like new screenshots, please let me know.

If it is not too much work, it would make sense to update the screenshots in the OP.

[ysangkok]

@andreasabel The screenshots have been updated as requested.

I pushed a commit that massively speeds up the code. Now the /verbose list of reverse dependencies performs acceptably:

$ time curl -s "http://localhost:8080/package/containers/reverse/verbose" -o /dev/null
0.03s user 0.01s system 28% cpu 0.119 total

[gbaz]

I pushed a commit that massively speeds up the code

The diff is pretty large (perhaps because of the rebase?) -- can you explain what the speed up comes from?

[ysangkok]

@gbaz Yes, it just comes from storing the list of dependencies for each package id in memory. That way, each cabal file is parsed only once. Here is the line that adds that map: 7f41c8b#diff-f1e6c87a39d7bc21d74de6224c837864dc413584c9fc37e6a2f6daa25db82834R70

[gbaz]

Happy new year @ysangkok ! Do you feel this is ready for merge, or is there anything you meant to get to but didn't?