Improve support for multiple TODs

[ziotom78]

Many modules in the framework directly write into Observation.tod, but in simulations it's often the case that we want to save components of the TOD file separately. This PR introduces the "components" keyword to a number of modules:

• Madam output
• Dipole
• Map scanning
• Noise
• make_bin_map

[marcobortolami]

Hi. @nraffuzz and I are finalising the test for this PR. Just a comment that could end up in a small modification of litebirdsim that we can include in this PR. Here from line 115 to line 125, maybe the indentation is not correct: shouldn't these lines be indented out of the for loop?
Sorry, I mistakenly clicked on "close with comment" and closed the PR .

[marcobortolami]

Also, here we believe that make_bin_map does not have the possibility to make maps from multiple tods, for example summing tod+tod_fg+tod_white_noise. Do you think that this function should be modified in order to have this feature implemented?

[ziotom78]

Here from line 115 to line 125, maybe the indentation is not correct: shouldn't these lines be indented out of the for loop?

Mmm, I see your point, it might be faster to run the if just once before the for loop.

[ziotom78]

Also, here we believe that make_bin_map does not have the possibility to make maps from multiple tods, for example summing tod+tod_fg+tod_white_noise. Do you think that this function should be modified in order to have this feature implemented?

Done, now make_bin_maps supports a components keyword that accepts a list of strings.

[marcobortolami]

First of all, thank you very much.
Then, we think that here there is not the possibility to save more than 1 map, right? Even if more than 1 tod component is saved, the output files are named as destriped.fits for example. The attribute output_file_prefix of the Class DestriperParameters is not used.
To be more clear, we would like to have, e.g., the following components inside a single obs: cmb, fg, 1/f with f_knee=30mHz, 1/f with f_knee=100mHz. Then, we would like to produce destriped maps of

• cmb + fg + 1/f with f_knee=30mHz
• cmb + fg + 1/f with f_knee=100mHz
• 1/f with f_knee=30mHz only
• 1/f with f_knee=100mHz only

We were thinking to set different output_file_prefix values and, if it is possible, to choose the components while using madam, maybe having different versions of the madam.sim and madam.par files. What do you think?

[ziotom78]

We were thinking to set different output_file_prefix values and, if it is possible, to choose the components while using madam, maybe having different versions of the madam.sim and madam.par files. What do you think?

I see! That code was something I wrote following my personal style: when running the same code multiple times, I prefer to change the name of the output directory while using the file names (like destriped.fits) over and over again, as it helps me to remember that those files had the same meaning.

If you feel that we should provide the user with the ability to pick their own file names, I can add keywords for each of these. But it's possible also to use the parameter madam_subfolder_name, which is analogous to Madam's output_file_prefix: it's the name of the subdirectory inside the output path of the simulation.

So, you could run Madam more than once with the following commands:

save_simulation_for_madam(…, components=["wn"], madam_subfolder_name="wn")
save_simulation_for_madam(…, components=["wn", "oof_30mHz"], madam_subfolder_name="wn+1f_30")
save_simulation_for_madam(…, components=["wn", "oof_100mHz"], madam_subfolder_name="wn+1f_100")
# …and so on

and the three destriped maps would be saved in

• wn/destriped.fits
• wn+1f_30/destriped.fits
• wn+1f_100/destriped.fits

Does it make sense? Or would you prefer to modify destriped.fits for each run?

[marcobortolami]

A small comment before proceeding with the discussion. Is there a motivation for which the keyword added to make_bin_map is called components and not component like in the previous commits of this PR? If not, we think it would be better to have the same keyword.

[ziotom78]

Is there a motivation for which the keyword added to make_bin_map is called components and not component like in the previous commits of this PR? If not, we think it would be better to have the same keyword.

Yes, here the type is List[str], because you can bin multiple components in the same map, while in code that writes something into a TOD (e.g. the dipole) the type is just str, because it's being written in just one place.

[marcobortolami]

We tested the version of LB sim for this PR and everything works fine: all the modified modules (make_bin_map, save_simulation_for_madam, ...) work as expected.
@nraffuzz and I think that the PR can be merged.