gh-136: Implementation of variable depth

[mwiet]

This merge request implements variable depth's effect on the galaxy density in the angular and line-of-sight directions into GLASS. This closes #136.

This code allows for functionality similar to SALMO. The implementation adds two novel features:

• The interpolation between the variable depth tracer variable and the change in galaxy density can be interpolated with any function (previously, in SALMO only linear interpolation was possible).
• The effect of variable depth along the line-of-sight and in the angular direction can be modelled as fully independent (previously, in SALMO the selection of galaxy shapes and galaxy redshifts was always assumed to be the same). This allows for the application of a separate selection function for the galaxy shapes and the galaxy redshifts.

The glass.observations.angular_variable_depth_mask implements the basic structure and functionality for the object, but only outputs the variable depth masks in the angular direction as they have been input.

The angular_los_variable_depth_mask expends upon this functionality by implementing the aformentioned features. Hence, when sampling galaxies using glass.points.positions_from_delta, one can multiply the given vis visibility mask by the output of angular_los_variable_depth_mask for a given tomographic bin and redshift shell index.

Tests with KiDS-1000-like mock

Testing this implementation with KiDS-1000 data (see here) gives results consistent with previous mocks on the same data:

• Overall redshift distributions are consistent with inputs:
  

• Local redshift distributions are shifted due to variable depth (e.g. North vs. South field):
  

• Between 2% and 10% (on average) of additional power is added to the measured pseudo-Cls from the realised galaxy fields:
  

Performance

The inclusion of the angular_los_variable_depth_mask when sampling the KiDS-1000-like galaxies added an additional 14 seconds to the total runtime on my local machine (from 7m 14s to 7m 28s, i.e. a 3.2% increase). In this time, angular_los_variable_depth_mask was evaluated 165 times (n_bins x n_shells = 5 x 33).

Potential extensions

Note that this implementation does not include the variability in the intrinsic shapes of galaxies which may be caused by variable depth. Additional features will be necessary for this in glass.shapes to allow the input sigma to vary per pixel.

[Saransh-cpp]

Thanks for the amazing work on this, @mwiet! This PR will require some design changes and adhering to good practices (tests, PEP8 conventions, ...), but I think we can discuss those once the functionality is ready and approved.

[paddyroddy]

pre-commit.ci autofix

[paddyroddy]

Sorry quite a lot of changes, but we can iterate.

As a minimum we need all CI to pass. Have a look at pre-commit. That is currently failing. You can run pre-commit run -a locally.

Also as @Saransh-cpp says we really need tests to go with this work.

[mwiet]

I have fixed most of the typing, naming convention and documentation issues. The only outstanding issue flagged by pre-commit is glass/observations.py:432:9: PLR0913 Too many arguments in function definition (10 > 5). Given the setup of the overall feature, this may be difficult to change unless we create an intermediate class or the inputs are merged into some larger object. Let me know what you think.

In terms of tests, I will try to add some useful test functions to tests/test_observations.py as soon as possible.

[paddyroddy]

Thanks so much for doing that! Appreciate there were few things in there, but that has been a lot of development velocity in the last six months.

The only outstanding issue flagged by pre-commit is glass/observations.py:432:9: PLR0913 Too many arguments in function definition (10 > 5).

Ah, don't worry about that. Simple functions are something to strive for, but it is not always possible. You can suppress that error using # noqa: PLR0912. See an explanation here https://docs.astral.sh/ruff/linter/#error-suppression if it's not clear.

[paddyroddy]

Not sure if we want this public, was more for making sure the docs CI passed

[ntessore]

Thanks for the fantastic work @mwiet! From your side, I think we are all set, you can tick the box on the to-do list :)

For us, I believe we should align this more closely with the overall roadmap for sampling galaxies in GLASS. I see a couple of tasks before merging:

We do not need the two classes. The base class checks for a valid index before accessing an array of maps; numpy does that for us, we can therefore simply use the stack of matrices. The work that the other class does in __getitem__() should be a standalone function that operates not on a stack of inputs, but on individual inputs (so the selection with index happens before the function is called).

And, importantly, a lot of the "variable depth functionality" is actually happening in the notebook, not in the library. Creating the stack of depth maps from a tracer, for example, is something that we need to provide. Think of it this way: how would a user go about creating the inputs for variable depth? Everything they would need to copy and paste verbatim from the example notebook should be provided by GLASS, I believe.

[paddyroddy]

The resolution was initially 1024. I have had to drop it to 64 as otherwise this notebook takes a really long time. 128 would be suitable, but the pre-allocation array step becomes too big in CI (based on GH runners), so went with 64. Still gives similar results (by eye).

[paddyroddy]

This section was initially

for i in range(nbins):
    for j in range(len(zbins)):

and displayed every single hp.mollview. This took a long time and made the notebook huge, so I have just made it one.

[paddyroddy]

This section was very slow in the initial implementation. By pre-allocating the catalogue, it is significantly quicker.

[paddyroddy]

I started to add tests yesterday before @ntessore's review. It feels pointless continuing these given the proposed changes.

[paddyroddy]

Some notes @mwiet sent me, which I think would be good to have here so they don't get lost.

Hi Paddy, Sorry about the delay! Here's an example notebook for a KiDS-1000-like survey with mock n(z), mock variable depth and a stage-IV-like mask. The survey properties are not all physical, but they give an idea of how they should be formatted.
Also, if you actually want to sample the galaxy catalogue. With the current galaxy density, it might take a really long time, so I would recommend just multiplying tomo_nz by 0.01.

Hi Paddy, If you multiply it by 0.01 just before it is integrated to get the n_gal for a given tomographic bin/shell in the sampling process, it will drastically reduce the number of galaxies it simulates.

[ntessore]

This actually closes #136, not 413.