Preprocessing
Executing fMRIPrep (on Curnagl)¶
Preparations¶
-
Prepare a FreeSurfer license file, for example at
$HOME/.freesurfer.txt
: -
Ensure the dataset is up-to-date:
- Checkout the correct tag corresponding to the intended processing:
Executing anatomical workflow first with --anat-only
¶
Compute nodes DO NOT have access to the NAS
Therefore, make sure data have been installed and fetched onto the <workdir>/data/hcph-dataset/
directory.
-
Create a SLURM sbatch file, for example at
$HOME/fmriprep-anatonly.sbatch
:# Copyright 2024 The Axon Lab <theaxonlab@gmail.com> # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ########################################################################### # # General SLURM settings #SBATCH --account <someaccount> #SBATCH --mail-type ALL #SBATCH --mail-user <email>@unil.ch # # Job settings #SBATCH --job-name fmriprep #SBATCH --partition cpu #SBATCH --cpus-per-task 10 #SBATCH --mem 10G #SBATCH --time 24:00:00 #SBATCH --export NONE #SBATCH --chdir /scratch/oesteban # # Logging #SBATCH --output /users/%u/logs/%x-%A-%a.out #SBATCH --error /users/%u/logs/%x-%A-%a.err ml singularityce gcc mkdir -p <workdir>/data/derivatives mkdir -p /scratch/oesteban/fmriprep export SINGULARITYENV_FS_LICENSE=$HOME/.freesurfer.txt singularity exec --cleanenv \ -B <workdir>/data/hcph-dataset:/data/datasets/hcph/ \ -B <workdir>/data/derivatives/:/out \ -B <workdir>/data/hcph-fmriprep/:/derivatives \ -B /scratch/oesteban/fmriprep:/tmp \ docker://nipreps/fmriprep:24.1.1 \ fmriprep /data/datasets/hcph/ /out/fmriprep-24.1.1 participant \ --participant-label 001 \ --bids-database-dir /data/datasets/hcph/.bids-index/ \ --nprocs 4 --omp-nthreads ${SLURM_CPUS_PER_TASK} \ -w /tmp/ -vv --skip-bids-validation --anat-only
-
Submit the anatomical workflow:
Executing functional workflow¶
-
Create a SLURM sbatch file, for example at
$HOME/fmriprep.sbatch
:#!/bin/bash # Copyright 2024 The Axon Lab <theaxonlab@gmail.com> # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ########################################################################### # # General SLURM settings #SBATCH --mail-type=ALL #SBATCH --mail-user=<email>@unil.ch # # Job settings #SBATCH --job-name=fmriprep #SBATCH --partition=cpu #SBATCH --cpus-per-task=10 #SBATCH --mem=128G #SBATCH --time=24:00:00 #SBATCH --export=NONE #SBATCH --chdir=/scratch/%u # # Logging #SBATCH --output=/users/%u/logs/%x-%A-%a.out #SBATCH --error=/users/%u/logs/%x-%A-%a.err ml singularityce gcc WORKDIR=<workdir> SCRATCH=<scratchdir>/fmriprep/ OUTDIR=$WORKDIR/data/derivatives/fmriprep-generalization/ mkdir -p $OUTDIR mkdir -p $SCRATCH export SINGULARITYENV_FS_LICENSE=$HOME/freesurfer.txt singularity exec --cleanenv \ -B $WORKDIR/workspace/fmriprep/fmriprep:/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep \ -B $WORKDIR/data/hcph-dataset:/data/datasets/hcph/ \ -B $OUTDIR:/out \ -B $SCRATCH:/tmp \ -B ${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/:${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/ \ docker://nipreps/fmriprep:24.1.1 \ fmriprep /data/datasets/hcph/ /out participant \ --participant-label 001 \ --fs-subjects-dir /out/sourcedata/freesurfer \ --nprocs 4 --omp-nthreads ${SLURM_CPUS_PER_TASK} \ -w /tmp/ -vv --skip-bids-validation echo "Completed with return code: $?"
-
Submit the functional workflow:
If running all sessions in a single job is too heavy, you can follow the following steps to run fMRIPrep on each session in parallel
-
To avoid conflicts when multiple fMRIPrep instances write to the same file simultaneously, we recommend pre-downloading the template with TemplateFlow.
- If not yet installed, install TemplateFlow using PyPi
- Use the TemplateFlow Python client to download the template, which is saved by default to
$HOME/.cache/templateflow
. In the SLURM sbatch file, we will mount that path into the container. To download the template, open a Python session by typingpython
and then run the following commands:
-
Once the anatomical workflow ran successfully, submit a job array with one scanning session each with the
--bids-filter-file
argument selecting the corresponding session, and point the--fs-subjects-dir
argument to the folder where FreeSurfer results were stored.
#!/bin/bash
# Copyright 2023 The Axon Lab <theaxonlab@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# We support and encourage derived works from this project, please read
# about our expectations at
#
# https://www.nipreps.org/community/licensing/
#
# STATEMENT OF CHANGES: This file is derived from work of the Nipreps
# developers and has been adapted to run smoothly on our particular
# dataset.
#
# ORIGINAL WORK'S ATTRIBUTION NOTICE:
#
# Copyright 2021 The NiPreps Developers <nipreps@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
DATADIR=<workdir>/data/hcph-dataset
SUB="sub-001"
pushd $DATADIR/inputs/$SUB > /dev/null
ALL_SES=(`ls -d ses-*`)
popd > /dev/null
#Remove session with no fMRI
SES=()
for S in "${ALL_SES[@]}"; do
if [ -d "$DATADIR/inputs/$SUB/$S/func" ]; then
SES+=("$S")
else
echo "Session '$S' has no functional scan."
fi
done
#remove one since we are starting at 0
JOBS=`expr ${\#SES[@]} - 1`
sbatch --array=0-$JOBS ss-fmriprep.sh $DATADIR ${SES[@]}
#!/bin/bash
# Copyright 2023 The Axon Lab <theaxonlab@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# We support and encourage derived works from this project, please read
# about our expectations at
#
# https://www.nipreps.org/community/licensing/
#
# STATEMENT OF CHANGES: This file is derived from work of the Nipreps
# developers and has been adapted to run smoothly on our particular
# dataset.
#
# ORIGINAL WORK'S ATTRIBUTION NOTICE:
#
# Copyright 2021 The NiPreps Developers <nipreps@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#SBATCH --partition=russpold
#SBATCH --mem=55GB
#SBATCH --cpus-per-task=16
#SBATCH --time=20:00:00
#SBATCH --job-name=fmriprep
#SBATCH --error="slurm-%A_%a.err"
# Submit a Single Session through fMRIPrep
ARGS=($@)
DATADIR=$1
STUDY=`basename $DATADIR`
if [[ -n $SLURM_ARRAY_TASK_ID ]]; then
SES=${ARGS[`expr ${SLURM_ARRAY_TASK_ID} + 1`]}
else
SES=$2
fi
echo "Processing: $SES"
IMG="/oak/stanford/groups/russpold/users/cprovins/singularity_images/fmriprep-23.1.4.simg"
WORKDIR="${L_SCRATCH}/fmriprep/${STUDY}/${SES}"
mkdir -p ${WORKDIR}
OUTDIR="${DATADIR}/derivatives"
mkdir -p $OUTDIR
PATCHES=""
BINDINGS="-B $DATADIR/inputs:/data:ro \
-B ${WORKDIR}:/work \
-B ${OUTDIR}:/out \
-B $DATADIR/code/license.txt:/opt/freesurfer/license.txt \
-B ${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/:${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/
$PATCHES"
FMRIPREP_CMD="/data /out/fmriprep-23.1.4 participant \
-w /work \
--bids-filter-file /work/filter_file_$SES.json \
--skip_bids_validation \
--fs-subjects-dir /out/fmriprep-23.1.4/sourcedata/freesurfer \
--anat-derivatives /out/fmriprep-23.1.4 \
--nprocs 4 --mem 45G --omp-nthreads 8 -vv"
#Create json file to filter one session only
echo '{"bold": {"datatype": "func", "session": "'${SES#*-}'", "suffix": "bold"}}' > ${WORKDIR}/filter_file_${SES}.json
SING_CMD="singularity run -e $BINDINGS $IMG $FMRIPREP_CMD"
echo $SING_CMD
$SING_CMD
echo "Completed with return code: $?"
How to proceed if some fMRIPrep derivatives are missing¶
If some derivatives are missing, it is a sign that fMRIPrep encountered an error.
- Check the "Errors" section of the visual report.
- Check the
log/
folder corresponding to the fMRIPrep run, carefully ensuring no errors were missed out on the reports. - Search for associated keywords in the issues on fMRIPrep's GitHub repository if the solution remains unclear after the first assessment; it is likely someone else experienced the same problem before you and reported it. The solution might be documented in the issue. Don't forget to check closed issues!
- Search for the issue on NeuroStars, if the solution remains unclear.
- If the solution remains elusive, open an issue in fMRIPrep's GitHub repository to report the problem. Your description of the problem needs to be as complete and detailed as possible to help the maintainers identify the problem efficiently.
- Re-run fMRIPrep on that particular subject and session after implementing a solution.
If the error remains despite all efforts, the session MAY be excluded
Visualizing fMRIPrep's individual reports¶
Anatomical preprocessing assessment¶
- Open the fMRIPrep anatomical report on a Web Browser.
- Assess the "Summary" section and apply the QA/QC criteria.
- Assess the "Anatomical conformation" section and apply the QA/QC criteria.
- Assess the mosaic showing the calculated brain mask and brain tissue segmentation, and apply the QA/QC criteria.
- Visualize the spatial normalization flickering mosaic, and apply the QA/QC criteria. Flickering between the subject and the template space is active while your mouse pointer hovers the mosaic area.
- Assess the surface reconstruction mosaic, and apply the exclusion criteria.
- Visualize the first section entitled Summary and apply the QA/QC criteria.
Assessment of fMRI Preprocessing¶
- Open each fMRIPrep functional report on a Web Browser.
-
Go through the section of each fMRI run and proceed as follows:
IMPORTANT — QCT and BHT are assessed first as proxies for the RSfMRI run's quality.
We employ the QCT (mainly) and the BHT as proxies for the quality of the RSfMRI run. Screening the reports in the prescribed order (QCT — BHT — RSfMRI) helps identify issues in the QCT and BHT that may anticipate problems in the RSfMRI.
- Assess the textual summary and apply the corresponding QA/QC criteria.
- Visualize the T2☆ map mosaic, and apply the QA/QC criteria.
- Check the T2☆ gray-matter intensity histogram, and apply the QA/QC criteria.
- Visualize the co-registration flickering mosaic, apply the QA/QC criteria. Flickering between T1w and BOLD images is active while hovering your mouse on the mosaic area.
- Visualize the next mosaic displaying regions of interest (ROIs) used to estimate the nuisance regressors, and apply the QA/QC criteria.
- Visualize the carpet plot and nuisance signals panel,
- Apply the exclusion criteria
- If you are visualizing the carpet plot corresponding to a RSfMRI run, apply additional QA criteria.
- Visualize the confound correlation heatmap and use it to choose the regressors you will include in the nuisance regression model.
- Proceed as indicated above if errors are reported within the "Errors" section.
- Continue with the next fMRI run section