nix: refactor with flakes

[berberman]

Flakes are the new feature of nix package manager that solves pain points of pinning nix dependencies and makes nix parts more composable. Recently, pre-commit-hooks.nix adds flake supports, but now we use it via builtins.fetchTarball, which is not good for maintenance. Currently we use niv to manage nix dependencies, but the generated files and inconsistency are annoying. So I think it's time to migrate to flakes. New commands:

• nix flake update - update all nix dependencies
• nix develop - start a HLS development environment, like the legacy nix-shell
• nix build - build HLS without cabal-install
• nix run - run HLS executable
• ...

If a downstream package using flakes wants to include HLS to its dev shell, it can directly add "github:haskell/haskell-language-server" to the inputs, and the latest git version of HLS will be available.

I keep only the support of the default GHC version in nixpkgs, because I don't think people will use other GHC versions to develop HLS with nix - not using binary caches is painful. I also add flake-compat to make legacy shell.nix still available.

Now running nix flake check or nix flake show will cause the evaluation of each system's expression, so I got :

error: a 'x86_64-darwin' with features {} is required to build '/nix/store/jxm5zc2s1w6qzfw28j0fdijq2q66d8py-cabal2nix-haskell-language-server.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}

on my NixOS machine, which will not affect the development, but I don't know where the problem is, and I'd appreciate if anyone could help...

UPDATE: I think the working principle of callCabal2nix might have some problems with flakes, and other Haskell projects using flakes seem to have the similar issue as well.

[berberman]

Hmm, I found in order to make nix build and nix run work, we need process jailbreaking, tests, etc., which would duplicate @maralorn's work on nixpkgs...

[maralorn]

I want to remark that flakes is still an experimental feature. Users will need to enable experimental features to be able to use this.

Besides that I can‘t follow your concerns in the mention. Why does the switch to flakes change your needs to override/fix stuff?

[berberman]

Besides that I can‘t follow your concerns in the mention. Why does the switch to flakes change your needs to override/fix stuff?

Because I want to make use of the output schema defaultPackage, i.e. a drv that build HLS executable without the help of cabal-install like in nixpkgs, providing a "git" version of HLS that can be used developing with nix.

[maralorn]

Ah, okay, I understand. Yes, this is a general issue, where package maintenance in nixpkgs duplicates efforts for every project that has it’s own default.nix. I am kind of hoping that once we have nix-in-nix it might emerge a new pattern where nixpkgs can delegate the package description to the upstream package. But I am not sure that it’s a) a good idea and b) will ever happen.

[pepeiborra]

How difficult is it to support customising the GHC version? Even if it is not convenient, sometimes it is necessary in order to fix compatibility issues

[berberman]

The structure of flake (eliding x86_64-darwin):

├───defaultPackage
│   └───x86_64-linux: package 'haskell-language-server-1.1.0.1'
├───devShell
│   └───x86_64-linux: development environment 'ghc-shell-for-packages-0'
├───overlay: Nixpkgs overlay
└───packages
    └───x86_64-linux
        ├───haskell-language-server: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-8104: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-8104-dev: package 'ghc-shell-for-packages-0'
        ├───haskell-language-server-884: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-884-dev: package 'ghc-shell-for-packages-0'
        ├───haskell-language-server-901: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-901-dev: package 'ghc-shell-for-packages-0'
        └───haskell-language-server-dev: package 'ghc-shell-for-packages-0'

Building (installation)

Derivations that build HLS are exported to packages.

• nix build - create HLS executable binary, without cabal-install
• nix build .#haskell-language-server - the same as nix build
• nix build .#haskell-language-server-8104 - for GHC 8.10.4
• nix build .#haskell-language-server-884 - for GHC 8.8.4
• nix build .#haskell-language-server-901 - this should fail because we don't support GHC 9 yet

Developing

• nix develop - enter the development shell that includes all dependencis needed to develop HLS
• nix develop .#haskell-language-server-dev - the same as nix develop
• nix develop .#haskell-language-server-8104-dev - for GHC 8.10.4
• nix develop .#haskell-language-server-884-dev - for GHC 8.8.4
• nix develop .#haskell-language-server-901-dev - for GHC 9.0.1

Notice

IIRC, hydra builds only the default GHC version of Haskell packages set, so you may have to build many Haskell dependencies manually if you don't use the default version. (We can do this in GitHub actions, and push it into cachix, but I don't know whether it is worth)

Besides cloning this repo and using the commands shown above, a Haskell project based on flakes can add this flake as input, obtaining the latest unreleased version of HLS in developing. This could be an alternative installation method (#122 discussed about nix installations).

[maralorn]

We currently build 884, 8104 and 901 on hydra, also we build (or rather try to build in the case of ghc9) hls for all of those. So actually you should get great caching for these builds.

[pepeiborra]

The structure of flake (eliding x86_64-darwin):

├───defaultPackage
│   └───x86_64-linux: package 'haskell-language-server-1.1.0.1'
├───devShell
│   └───x86_64-linux: development environment 'ghc-shell-for-packages-0'
├───overlay: Nixpkgs overlay
└───packages
    └───x86_64-linux
        ├───haskell-language-server: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-8104: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-8104-dev: package 'ghc-shell-for-packages-0'
        ├───haskell-language-server-884: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-884-dev: package 'ghc-shell-for-packages-0'
        ├───haskell-language-server-901: package 'haskell-language-server-1.1.0.1'
        ├───haskell-language-server-901-dev: package 'ghc-shell-for-packages-0'
        └───haskell-language-server-dev: package 'ghc-shell-for-packages-0'

Building (installation)

Derivations that build HLS are exported to packages.

• nix build - create HLS executable binary, without cabal-install
• nix build .#haskell-language-server - the same as nix build
• nix build .#haskell-language-server-8104 - for GHC 8.10.4
• nix build .#haskell-language-server-884 - for GHC 8.8.4
• nix build .#haskell-language-server-901 - this should fail because we don't support GHC 9 yet

Developing

• nix develop - enter the development shell that includes all dependencis needed to develop HLS
• nix develop .#haskell-language-server-dev - the same as nix develop
• nix develop .#haskell-language-server-8104-dev - for GHC 8.10.4
• nix develop .#haskell-language-server-884-dev - for GHC 8.8.4
• nix develop .#haskell-language-server-901-dev - for GHC 9.0.1

Notice

IIRC, hydra builds only the default GHC version of Haskell packages set, so you may have to build many Haskell dependencies manually if you don't use the default version. (We can do this in GitHub actions, and push it into cachix, but I don't know whether it is worth)

Besides cloning this repo and using the commands shown above, a Haskell project based on flakes can add this flake as input, obtaining the latest unreleased version of HLS in developing. This could be an alternative installation method (#122 discussed about nix installations).

Could you record these instructions in the default.nix file?

[Ailrun]

LGTM :)

[berberman]

Branch protection rules need to be updated

[Ailrun]

Done