BIDS filter not working

[bilas003]

What happened?

BIDS-filter is not working to filter out the set of files with the latest release of the build. The job ran for 4 days and then timed out . Can anyone help?

What command did you use?

Used a slurm job to run the pipeline,see the attached image.

What version of NiBabies are you using?

nibabies-24.0.0.sif

Relevant log output

Time -out

Add any additional information or context about the problem here.

[mgxd]

Can you share the output log and the contents of the filter file you are using? Also looks like your line break on --derivatives is not properly spaced.

FYI - The latest release is 24.1.0, and we just cut a release candidate (25.0.0rc1) of a new minor series slated to come out later this week.

[bilas003]

BIDS -filter json

{
"bold":{"datatype":"func","suffix":"bold","session":"V02"},
"sbref":{"datatype": "func", "suffix": "sbref","session":"V02"},
"fmap": {"datatype": "fmap","session":"V02"},
"t2w": {"datatype": "anat", "suffix": "T2w" , "run":"2", "session": "V02","SeriesDescription":"anat-t2w","ProtocolName":"anat-t2w"}

}

I corrected the derivatives flag and ran again.

[mgxd]

Were any T2w images found? I don't think --bids-filter-file will filter metadata, only bids entities directly in the filename.

[erikglee]

@mgxd, nibabies uses pybids to find files based on the bids filter file, right? @bilas003 had tried running some independent tests with pybids that involved a filter file with fields that reference the JSON metadata. In her example, it looked like pybids came up with the right files based on the filter settings.

[erikglee]

Hi @bilas003, @mgxd just informed me that another good option to test would be trying to put null (without quotations) for the t2w "acq" entity in the BIDS Filter File. This should be another strategy to avoid the localizer images (since they have "acq-mrsLoc" in their name) during processing, without Nibabies needing to search through the JSON meta-data.

Could you try running another test where you replace the t2w filter file content to be:
"t2w": {"datatype": "anat", "suffix": "T2w" , "run":"2", "session": "V02", "acq": null}

[bilas003]

Hi @mgxd @erikglee

Added the acq tag as null to bids-filter.

The new error received is
ValueError: 'acq' is not a recognized entity. If you're sure you want to impose this constraint, set invalid_filters='allow'.

[mgxd]

You'll have to use the full entity name - acquisition

[bilas003]

The job has been running for 3 days now. The error log shows FileNotFound (see screenshot)

[mgxd]

There are a few moving parts here, can you clarify the following?

1. What does your bids directory tree look like for the subject you are running?
2. What does your filter file look like now?

Additionally, some suggestions:

• --skip-bids-validation should not be used when debugging querying a BIDS dataset
• If you are just starting analysis, try using 25.0.1 as it's the most up to date version
• --participant-label <id> can be used in your command to narrow down processing for a single subject, simplifying debugging
• --age-months is deprecated and shouldn't be used

[bilas003]

Hi @mgxd

Here is the tree structure

filter like is

I followed your suggestions and made changes with latest version of sif , the job fails with below err