orthogonalize AP/PD src/sink edges

[jordandekraker]

This approach continues with the distance transform logic introduced by #371, and closes #373.

We want to ensure that all edge vertices are assigned to an AP or PD source or sink (4 possible labels), without overlapping or missing any vertices (I will call this orthogonal edge conditions). Otherwise we can end up with "corners" that are entire missing from the unfolded space, or other funny distortions in the unfolding.

Here, we introduce a new script to fix this up:

• we move the application of signed distance transforms out of the laplace-beltrami solver rule and into its own rule, where we jointly solve AP and PD edges
• instead of iteratively increasing distances to add vertices, we treat the distances as probabilities for one of these 4 labels and take the maxprob
• in some cases we will still not have min_terminal_vertices>5. In these cases, we gradually increase the distance scaling factor from that label until min_terminal_vertices is reached.
• we get to keep parallelization of laplace-betrami. AP and PD edges do have to be processed jointly though, to ensure they are orthogonal. We also get to toss out a bunch of complicated functions

we also now apply largest connected component to finding boundary vertices to prevent issues with holes

TODO:

• we could still ensure conncomp in addition to min_terminal_vertices, but i don't think its necessary
• parameter to adjust distance scaling factor stepsize?

[akhanf]

nice, will give it a try!

[jordandekraker]

sorry still WIP - some formatting bugs to fix! hopefully push an update in a few hours

[jordandekraker]

should be working now, and I also managed to speed up the laplace-beltrami solver by replacing a for loop with a vectorized approach (I noticed from the logger that just setting the boundary conditions was taking a while)

[akhanf]

might need a bit more tweaks for robustness to ensure the counts are at least 1 vertex, as this fails now for the ds002168 dataset that I had working correctly before.. Haven't had a chance to look at that fix yet, but made some cosmetic changes just now..

[akhanf]

OK just had a chance to look at this again for my failed test case (dentate for ds002168)

Looks like just needed more iterations than the capped 10.. I'll do a bit more clean-up then I thnk should be ready to merge.

[jordandekraker]

OK one thing I also just pushed: we can make the distance scaling additive rather than multiplicative in order to match what we were doing before with iterations of dilation
Either solution should be fine, but yeah we probably just need a few more iterations. I can't seem to wetrun test because there seems to be an issue with my nnunet conda env

[akhanf]

Actually I still have an issue now with the resulting labels being multiple connected components, so may have to add in connected components for robustness..
Yes I actually had a nnunet issue when trying to run on the weekend too, and reusing my previous runs to avoid recomputing -- not sure if something changed there, could post an issue for @Dhananjhay to look into?

[Dhananjhay]

Yes, I'm aware of the issue and I've been trying to debug it. We are using nunnet's official conda package and it's pinned in the env configuration file, i.e., hippunfold/workflow/envs/nnunet.yaml so I doubt it's something we changed. I wanted to cross check the issue by running the pipeline with --use-singularity flag but weirdly enough, I get this error everytime:

CreateCondaEnvironmentException:
Conda must be version 24.7.1 or later, found version 4.10.3. Please update conda to the latest version. Note that you can also install conda into the snakemake environment without modifying your main conda installation.

I'll open an issue.

[akhanf]

OK this cleans things up a bit more and now is working well on my test dataset -- it needed the additive approach (I didn't see your commit until I was done, but looks like we did the same thing)..

I'll merge this in to move things along..