How to?

The two steps are: 1) determine whether the slopes of your effects differ, 2) test your main effects.

In order to do 1):

• Build the more complex (maximal) model.
• Check convergence with a large number of iterations.
• Build a reduced model by removing 1 element.
• Compare models in descending order: if there is a significant reduction in fit, accept the more complex model; if there is no decrease, continue descending.
• If further descending is needed, remove 1 element from the previously reduced model and repeat the process.
• Continue until a significant reduction in fit is found.
• Pro tip: the suggested reduction order is interactions > main effects.

In order to do 2), you can use a Chi-square likelihood ratio test. In R, this is what anova(reduced_model, full_model) does (not summary(), which only prints the coefficient estimates).

Pro tip: if you’re running a logistic (binomial) mixed model, you can get the effect size as an odds ratio by exponentiating the beta estimate — in R, exp() will do it for you. This only applies to logistic/binomial models; for a standard linear mixed model with a continuous outcome, the beta is already on the outcome’s scale and there’s no odds ratio to compute.

What does “number of groups” mean? The number of actual groups of participants? The number of levels in your variables? Also, what does “number of measurements” mean? The number of variables? The levels in them? Should I sum the levels of all variables? Should I multiply them? And, what is even worse, which measure of effect size should I use? d, eta, eta squared, F, the other weird Greek letter… Should I leave the one set by default in the software? I guess that must be the correct one, but then… how do I decide how much is a medium effect size?

If you want to dive deep and learn about all of that, you can go here and here. But if you only want to know which numbers to use, rest assured, I got you covered. You just need to know the size of the effect you would care about and which one is your key analysis. You can use the table below as an orientation on effect sizes.

Cohen’s d (definition and calculation)

• 0.2 = Small effect size
• 0.5 = Medium effect size
• 0.8 = Large effect size

Cohen’s f (definition and calculation)

• .10 = Small effect size
• .25 = Medium effect size
• .40 = Large effect size

From: Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Quick calculators by design

The original version of this post had interactive calculators embedded for each design below. Those widgets were tied to my old Google Sites page and didn’t survive the move, but the designs they covered — and where to get the numbers instead — are still useful:

• One-tailed t-test (matched samples)
• Two-tailed t-test (matched samples)
• One-way repeated-measures ANOVA (3 levels)
• Mixed design: 3 (within) by 3 (between)

For any of these, G*Power remains the standard free tool — pick the matching test family and design under “Test family → t tests / F tests” and plug in the effect size from the table above; for the mixed design specifically, use G*Power’s dedicated “ANOVA: Repeated measures, within-between interaction” test. If you prefer to stay in R, the pwr package covers the t-tests and the one-way ANOVA (pwr.t.test(), pwr.anova.test()), but doesn’t have a repeated-measures/mixed-design function — for that, use wp.rmanova() from the WebPower R package (or its free online calculators if you’d rather not install anything).

Why?

If you are here, you have probably tried to run fmriprep through Docker and have realized that you do not have admin rights in the system that you want to use. If you have not tried that, I suggest you do it (see here how to) and only use Singularity as a plan B.

Since I cannot find a reason why you might want to run Singularity locally (other than debugging your scripts), I will describe here how to run it on a High-Performance Cluster (HPC). The first thing to do is to take a deep breath, get a cup of your favorite drink, and equip yourself with a lot of patience: this can get messy.

Every HPC can be different and you will probably have to tweak things here and there to get it running the first time. The examples I give here are for an HPC using SLURM and with a rather closed setup, so you can think about this procedure as a brute-force approach: the bright side is that if you try all this, you will probably succeed on every system.

Also check the Tips for Singularity post, gathered after too many trial-and-error attempts of my own.

How?

Make sure Singularity is (or can be) installed on your HPC. This part might require some time and mano izquierda to talk your support people into doing it.

Once Singularity is up and running, you need to make sure that you have enough writing space for your user on the HPC. Usually some temporary storage drive is allowed for this — make sure you know where it is and how to access it. NOTE: storage limits can come in size or number of files (inodes), or both; try to discover your limits.

The next step is to download fmriprep’s image. While this is a rather trivial step in Docker, in Singularity it can be tricky. First, identify which version of fmriprep you would like to use; I recommend you always start with the latest one. Singularity does not like the “latest” tag when building, so you need to specify the exact version (e.g., poldracklab/fmriprep:20.1.1). You might want to build the image locally and transfer it to the HPC:

singularity build /my_images/fmriprep-<version>.simg docker://poldracklab/fmriprep:<version>

scp /my_images/fmriprep-<version> user@address:/path/to/writable/location/

Replacing <version> with the exact version name as in the example above, and user and address with your HPC login details.

1. Some HPC nodes do not have internet access, and fmriprep will try to go online to pull brain templates from TemplateFlow. My recommendation is that you download the templates you are going to need (or all of them, if you want to go with the sledgehammer approach) and transfer them to the HPC. Templates can be downloaded with the Python API using the syntax below (choose the templates that you need):

   from templateflow import api as tflow
   
   tflow.get('MNI152NLin2009cAsym')
   tflow.get('OASIS30ANTs')
   tflow.get('MNI152NLin6Asym')
   tflow.get('fsaverage')
   tflow.get('NKI')
   tflow.get('MNI152NLin2009cSym')
   tflow.get('WHS')
   tflow.get('fsLR')
   tflow.get('MNIPediatricAsym')
   tflow.get('MNI152Lin')
   tflow.get('MNIInfant')
   tflow.get('MNI152NLin6Sym')
   

2. Rather obvious step: you need to put your BIDS dataset on the HPC (duh!).

3. Once you have all the files needed on the HPC, you can start… preparing your call script. Singularity and fmriprep will try to write a bunch of temporary files in your home directory, so if you have limited space in home, you need to redirect them to your larger writable locations. See the lines after “Re-direct some (…)” in the script below.

4. You also need to overwrite fmriprep’s TEMPLATEFLOW_HOME with the path inside the container where you are going to mount your templates. See the lines after “Pass some variables into the container”.

5. Now you are ready to call fmriprep through Singularity. After the run command, you need to mount the BIDS folder, the output folder, the templates folder, and a writable folder for the home and working directory. Use the -B flag for that (see script below). Do not forget to also mount the path to your FreeSurfer license.

6. Remember about the nodes not having internet access? You need to tell fmriprep not to attempt it, since it will fail anyway. That is what the --notrack flag does.

7. Finally, it is important to pass the --home flag to Singularity so that it overwrites the default home directory. This one needs to be followed by /home/fmriprep.

8. And you are set!

Just a few more notes on the script below:

• I am passing a variable into the container named subject so I can pass it to the --participant-label flag for fmriprep. This is pretty useful if you want to create different temporary folders for different subjects, or if you want to use your job ID to index several subjects.

• Sometimes, after crashing, FreeSurfer does not delete all of its intermediate files, and that might cause problems when trying to re-run fmriprep on the same subjects. You might want to include a line at the beginning to search for those corrupted files and delete them if found; something like this will do:

  find ${FREESURFER_HOST_CACHE}/$subject/ -name "*IsRunning*" -type f -delete
  

• Everything prior to the line “Which subject do you want to run?” is set for our HPC at Goethe University in Frankfurt. Yours might be different in terms of available resources or requirements; adjust it accordingly.

#!/bin/bash
#SBATCH --array 01
#SBATCH --partition=general1
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=80
#SBATCH --mem-per-cpu=1024
#SBATCH --time=02:30:00
#SBATCH --no-requeue
#SBATCH --mail-type=ALL
# ------------------------------------------

##########################################################################
# fmriprep
##########################################################################
# This script runs a singularity image of fmriPrep on files on BIDS format.
# This script is optimized for Frankfurt's HLR cluster.
#
# Javier Ortiz-Tudela (ortiz-tudela@psych.uni-frankfurt.de)
##########################################################################

echo "Loading singularity..."
spack load singularity@3.5.2

# Which subject do you want to run?
subject=01  # swap for $SLURM_ARRAY_TASK_ID if you want each array task to run a different subject

# Re-direct some environmental variables to writable locations
export SINGULARITY_TMPDIR=/path/to/writable/location/fmriprep_temp/$subject
export SINGULARITY_CACHEDIR=/path/to/writable/location/.cache
export TEMP_DIR=/path/to/writable/location/fmriprep_temp/$subject
echo "creating temp directory in"
echo $TEMP_DIR
mkdir -p $TEMP_DIR

# Pass some variables into the container
export SINGULARITYENV_subject=$subject
export SINGULARITYENV_TEMPLATEFLOW_HOME=/templateflow

# Setup done, print a message and run the fmriprep call
echo -e "\n"
echo "Starting fmriprep."
echo "subject: $subject"
echo -e "\n"

singularity run \
    -B /path/to/writable/location:/home/fmriprep \
    -B /path/to/data/BIDS:/data:ro \
    -B /path/to/templates/templateflow:/templateflow \
    -B /path/to/output/folder:/output \
    -B /path/to/license:/lic \
    -B $TEMP_DIR:/working_dir \
    --home /home/fmriprep --cleanenv \
    /path/to/image/fmriprep-20.1.0.simg \
    /data /output participant \
    --notrack \
    --fs-license-file /lic/license.txt \
    --participant-label $subject \
    --work-dir /working_dir

Failed to set effective UID to 0

From this GitHub issue I got that the easiest way is to look for a singularity.conf file and change allow setuid=yes to allow setuid=no. That file should be located in your Singularity installation directory. If you used spack for the installation, the path would look something like this:

/path/to/spack/opt/spack/linux-scientific7-haswell/gcc-4.8.5/singularity-3.5.2-k2cgya4v4mjzvfay2n4jfmy7wbda5ucq/etc/singularity/singularity.conf

See also: how to run fmriprep on an HPC cluster with Singularity.

The easiest way to use fmriprep is through a Docker container (learn more about them here). You need to install it in your system in case it is not there yet (click here to learn how to). Bear in mind that, in order to use Docker, you need admin rights in your system. In case you cannot get those, you can use Singularity instead.

Below you will find an example shell script that I use to make the fmriprep call using Docker (original source here).

##########################################################################
# fmriprep
##########################################################################
# This script runs a docker image of fmriPrep on files on BIDS format.
#
# Javier Ortiz-Tudela (ortiz-tudela@psych.uni-frankfurt.de)
##########################################################################

### Input in the study info here #####

# Which subject are you preprocessing?
which_sub=999

# Where is your project located?
project_path=~/path/to/project/

### Now I will build some names for you ####

# Build BIDS and output folders paths
bids_path=$project_path/BIDS
out_path=$project_path/BIDS/derivatives

# Build license folder path
lic_path=$project_path/

#----START PREPROCESSING ------#
sudo docker run -ti --rm \
  -v $bids_path/:/data:ro \
  -v $out_path/:/output \
  -v $lic_path/:/lic \
  nipreps/fmriprep:latest \
  /data /output \
  participant \
  --participant_label $which_sub \
  --fs-license-file /lic/license.txt

Just place this snippet of code on your machine, adapt the paths to match yours, and select which options you want to turn on or off for fmriprep (check the full list here). Most of the things can be set to default, but there are a few options that you will probably want to fiddle with:

• Options for handling performance: fmriprep runs some heavy stuff on your images, so you might want to restrict how much of your machine’s power it can harvest. Find here the recommendations from their site. I would not recommend less than 10-12 GB of RAM for one subject. Use the number-of-threads argument to restrict the number of subprocesses run in parallel (too many things in parallel will max out your RAM and crash the entire thing!).
• --output-spaces: your preprocessed data will be resampled into whichever space you put here. You can specify the space and the resolution (this last option is only available for standard spaces). See here a more detailed guide on spaces. NOTE: If you want to use AROMA in fmriprep to clean your data, you can only use MNI152NLin6Asym; if you want to preprocess your data with fmriprep and then run AROMA manually, that is also possible.

Getting the whole thing running for the first time might sound daunting, but after you have done it once for one dataset (and computer), it will be super easy to run more people and to transfer that knowledge to new projects.