Pangenome inference with Minigraph-Cactus

Author:Gregg Thomas · Last updated: November 1, 2026

For aligning genome assemblies from the same species for population genomic analyses, the Cactus genome alignment software has implemented a workflow using minigraph . However, this can still be technically difficult to run. Here we have developed a Snakemake pipeline to facilitate running Minigraph-Cactus on a computing cluster. For more details on how Snakemake breaks up Minigraph-Cactus's steps, expand the box below.

The minigraph pipeline's rulegraph

Here is the rulegraph for the pipeline. The split rule breaks up the workflow by chromosome/scaffold for efficient submission of align steps, with align picking up where split left off.

A directed acyclic graph showing the rules for the pipeline.

This pipeline is suitable for aligning and constructing pangenomes from samples of the same species

For whole genome alignment between different species, see our Cactus whole genome alignment tutorial

Getting started

Requirements

You will need several things to be able to run this pipeline:

A computing cluster that uses SLURM , which most datasets will need. It should be possible to extend the pipelines to any job scheduler that Snakemake supports .
conda or mamba to install software. See Installing command line software if you don't have conda/mamba installed.
Singularity, installed on your system. The pipelines run Cactus with singularity exec, and conda can't install Singularity for you. Apptainer installs usually provide a singularity command too. Some clusters (such as the Harvard cluster) already have it installed.
The Harvard Informatics cactus-snakemake pipelines .

You don't need to install Snakemake or its SLURM plugin yourself. The pipelines come with a wrapper script, cactus_snakemake, that creates a conda environment with everything they need except Singularity.

Installing cactus-snakemake

The pipelines are available on GitHub. Navigate to the directory in which you want to download them, then clone the repository, move into it, and use the wrapper to create the conda environment:

git clone https://github.com/harvardinformatics/cactus-snakemake.git
cd cactus-snakemake
./cactus_snakemake setup    # create the conda environment from envs/environment.yml
./cactus_snakemake check    # confirm the environment and singularity are ready (read-only)

If you have SSH set up on GitHub, you can clone with git clone git@github.com:harvardinformatics/cactus-snakemake.git instead. If you don't have or don't wish to use git, you can download the latest release , unzip it, and run the same setup and check commands from inside the unzipped folder.

setup creates a conda environment named cactus-snakemake. check reports whether that environment exists and is up to date, and whether singularity is on your PATH. It never creates, updates, or deletes anything. You don't need to activate the environment yourself: the wrapper activates it when it runs a pipeline.

The commands in this tutorial are run from inside the cactus-snakemake folder. To run them from somewhere else, give the path to the wrapper instead of ./cactus_snakemake, for example /path/to/cactus-snakemake/cactus_snakemake. Run ./cactus_snakemake help to see every command and option.

Using a different environment name

To use a conda environment with a different name, give --env-name NAME before the command, for example ./cactus_snakemake --env-name my-cactus-env setup, or set the environment variable CACTUS_SMK_ENV_NAME. If you don't use the default name, you must give it every time you run the wrapper.

Important! cactus-snakemake v3.0.0 and later requires Cactus v3.0.0 or later.

Due to bug fixes in Cactus, v3.0.0+ of cactus-snakemake is only compatible with Cactus v3.0.0 or later. Don't worry, cactus-snakemake will always use the latest version of Cactus available unless you specify otherwise in your config file. However, if you do wish to use an older version of Cactus, you can use cactus-snakemake v2.1.0 .

Automatic partition selection

Snakemake's SLURM executor plugin can choose a partition for each job based on its resource requests, if you give it a partition config for your cluster with --slurm-partition-config <path>. You can then leave the partition: entries blank in your pipeline's config file.

If you are on the Harvard Cannon cluster, we maintain a partition config you can use. See Snakemake configuration for Cannon partition selection for its location and how to use it.

If you are not on the Harvard Cannon cluster, you can still directly specify the partitions for each rule in the config file (see below), or you may take it upon yourself to make a partition config for your own cluster.

Upgrading from cactus-snakemake v3

The way the pipelines are installed and run has changed. If you used cactus-snakemake v3:

Set up the environment with the wrapper. Run ./cactus_snakemake setup as shown above. You no longer install Snakemake yourself, and you run the pipelines with ./cactus_snakemake run instead of snakemake -s.
Prep is now a separate, required step. v3 ran cactus-prepare (or cactus-update-prepare) every time Snakemake was invoked. Now it runs only during prep, and every other run stops until prep has been done for its output directory. For an output directory from v3, run prep once with the same config and input files, then run the pipeline as before.
Two pipelines were renamed. cactus_update.smk is now cactus_add_genome.smk (the add-genome pipeline) and cactus_replace.smk is now cactus_replace_genome.smk (the replace-genome pipeline), with their config templates renamed to match. Config keys are unchanged.
skip_prepare is no longer used. If it's set, the pipelines log a warning and otherwise ignore it.

With that, you should be ready to set-up your data for the pipeline!

Inputs you need to prepare

To run this pipeline, you will need (corresponding config option given in parentheses):

The assembled genome FASTA files for each sample (specified in input_file).
A reference sample (reference).

You will use these to create the input file for Minigraph-Cactus.

Preparing the Minigraph-Cactus input file

The various Minigraph-Cactus commands depend on a single input file with information about the genomes to align. This file is a simple tab delimited file with two columns.

Each line contains in the first column one sample label and in the second column the path to the genome FASTA file for that sample.

Unlike regular Cactus, Minigraph-Cactus does not require the input sequences to be softmasked

However, the FASTA files should still not be hard masked, meaning masked bases are replaced with Ns.

For example, if one were running this pipeline on 5 samples named A, B, C, D, and E, the input file may look something like this:

A   seqdir/a.fa
B   seqdir/b.fa
C   seqdir/c.fa
D   seqdir/d.fa
E   seqdir/e.fa

There is much more information about the Minigraph-Cactus input file in the official documentation for Minigraph-Cactus , including how to run the pipeline with diploid assemblies. There is also an example input file for a small test dataset here or at tests/yeast-minigraph/yeastPangenome-local.txt. For more info, see section: Test dataset.

Reference sample

Minigraph-Cactus requires that you select one sample as a reference sample for a variety of reasons . Keep these in mind as you prepare your inputs. The reference sample will be indicated in the config file (outlined below). For instance, if I wanted my reference sample in the above file to be C, I would put the string C in the reference: line of the config file.

Preparing the Snakemake config file

Besides the sequence input, the pipeline needs some extra configuration to know where to look for files and write output. That is done in the config file for a given run. Create a starter config file from the pipeline's template with init:

./cactus_snakemake init minigraph -o my-config.yaml

This copies the pipeline's config template (config-templates/minigraph-config-template.yaml in your cactus-snakemake folder) to my-config.yaml. Without -o, it's written to ./minigraph-config.yaml. init won't overwrite an existing file unless you add --force. Fill in the blank fields before running prep.

See an example config file

The config for the Minigraph-Cactus test data can be found here or at tests/yeast-minigraph/yeast-minigraph-cfg.yaml in your cactus-snakemake folder.

The config file contains 3 sections: one for specifying the input and output options, one for specifying resources for the various rules (see below), and one for optional settings that most runs can leave blank. The first part should look something like this:

cactus_path: <path/to/cactus-singularity-image OR download OR version tag (e.g. v3.3.0)>

input_file: <path/to/cactus-input-file>

output_dir: <path/to/desired/output-directory>

reference: <Sample ID from input_file>

prefix: <string>

join_graph_type: <string. options: full, clip, filter>

join_filter: <integer>

tmp_dir: <path/to/tmp-dir/>

Simply replace the string surrounded by <> with the path or option desired. Below is a table summarizing these options, along with the optional setting at the bottom of the config file:

Option	Description
`cactus_path`	The Cactus Singularity image to use: a file path, a version tag (e.g. `v3.3.0`), or `download` for the latest release. See the box below the table.
`input_file`	Path to the input file containing the sample IDs and genome paths (described above).
`output_dir`	Directory where all the output will be written. It will be created if it doesn't exist.
`reference`	The sample ID from the input file to serve as the reference for pangenome creation.
`prefix`	A string that will be appended to all files created by the pipeline.
`join_graph_type`	Controls the level of filtering on the final files. One of "full", "clip", or "filter".
`join_filter`	Only applicable for `join_graph_type: filter`. The minimum number of haplotypes that must be present in a node for it to be retained.
`tmp_dir`	A temporary directory for Snakemake and Cactus to use. It will be created if it doesn't exist, but should be somewhere with lots of space.
`singularity_bind_paths`	Optional. A YAML list of extra Singularity bind paths to add to every container command, each in `src[:dest[:opts]]` format, e.g. `/nfs6:/nfs6` or `/scratch:/scratch:rw`. Only needed if some of your files are on a filesystem that isn't available inside the container by default; jobs failing to find files that do exist are the usual sign. Leave blank or set to `[]` to use the cluster defaults only.

For join_graph_type and join_filter, see the Cactus docs for more info.

Choosing the Cactus image with cactus_path

The pipelines run Cactus from a Singularity image, set with cactus_path. It can be:

download: prep downloads the latest Cactus release.
A version tag, such as v3.3.0 or 3.3.0: prep downloads that version. Always give all three parts of the version. An unquoted 3.3 is read by YAML as a number, and prep fails.
The path to a Cactus Singularity image you already have.

Cactus v3.0.0 or later is required.

When cactus_path is download or a version tag, prep downloads the image only once. It's saved as cactus_vX.Y.Z.sif in the directory you run prep from, and every later run uses that exact image.

In the Cactus alignment pipelines (all except Minigraph-Cactus), use_gpu decides which build of Cactus is used. With use_gpu: True, download or a version tag gets the GPU build of that version, saved as cactus_vX.Y.Z-gpu.sif. Don't add -gpu to the version tag yourself. If you give a file path with use_gpu: True, it must point to a GPU image. The Minigraph-Cactus pipeline doesn't use a GPU image.

Specifying resources for each rule

Below these options in the config file are further options for specifying resource usage for each rule that the pipeline will run, starting with a default: entry. For example:

rule_resources:
  default:
    partition: shared
    mem_mb: 10000
    cpus: 1
    time: 30

  minigraph:
    partition: shared
    mem_mb: 25000
    cpus: 8
    time: 30

The steps in the Minigraph-Cactus pipeline are not GPU compatible, so there are no GPU options in this pipeline.

Notes on resource allocation

Be sure to use partition names appropriate for your cluster. Several examples in this tutorial have partition names that are specific to the Harvard cluster, so be sure to change them.
If you run with -e slurm --slurm-partition-config <path to your cluster's partition config>, you can leave the partition: fields blank and one will be selected for each job based on its other resources. On the Harvard Cannon cluster, you can use our partition config .
The default: entry fills in any value that a rule's own entry leaves blank, and is used in full for rules that have no entry of their own. If neither a rule's entry nor default: gives a value, the run stops with a "Missing resource ... no default set" error. The one exception is partition, which may be left blank when running with -e slurm.
mem_mb is in MB and time is in minutes.

Scale resources on retry

Any resource except gpus and gres can be given as a list of per-attempt values instead of a single value. Attempt 1 uses the first value, attempt 2 uses the second, and later attempts reuse the last value. This way, failed jobs can be resubmitted with more memory or time without editing the config between restarts. For example, for the rule align, you could scale memory and runtime as follows:

rule_resources:
  align:
    mem_mb: [80000, 160000, 200000]
    time: [1660, 3320, 6640, 7200]

Retries only happen when you add --retries N to the run command, for example:

./cactus_snakemake run <pipeline> --configfile my-config.yaml -j 20 -e slurm --retries 3

gpus and gres can't be lists. A list there stops the run with an error.

You will have to determine the proper resource usage for your dataset. Generally, the larger the genomes, the more time and memory each job will need, and the more you will benefit from providing more CPUs.

Running the pipeline

With everything installed, the Minigraph-Cactus input file, and the config file set up, you are now ready to run the pipeline. This takes three steps: prep, a dry-run, and then the real run.

Prep

Prep is run once for each output directory, before the pipeline itself:

./cactus_snakemake prep minigraph --configfile my-config.yaml

For this pipeline, prep only resolves the Cactus image, downloading it if cactus_path is download or a version tag. Minigraph-Cactus has no prepare command.

Prep runs on the node you start it from, not as a cluster job. It records what it did in <output_dir>/prep-manifest.yaml. Every later run, including each job submitted to the cluster, reads that manifest instead of repeating prep, and stops with an error if:

prep hasn't been run for the output directory yet. The error gives the command to run.
the config or input files that prep depended on have changed since (for example the tree in input_file, cactus_path, or where a new genome goes), or a file prep wrote has been changed. The error lists what changed.

If a change is intended, run prep again with --force:

./cactus_snakemake prep <pipeline> --configfile my-config.yaml --force

Files already in the output directory were made from the earlier prep and are not deleted, so for anything that changes the tree, a new output directory is usually the safer choice.

Do a `--dryrun` first

Next, we want to make sure everything is set up properly by using the --dryrun option. This tells Snakemake to display what jobs it is going to run without actually submitting them. This is important to do before actually submitting the jobs so we can catch any setup errors beforehand.

./cactus_snakemake run minigraph --configfile my-config.yaml -j 20 -e slurm --dryrun

Command breakdown

Command-line option	Description
`./cactus_snakemake run`	Activates the pipelines' conda environment and runs Snakemake. Everything after the pipeline name is passed to Snakemake unchanged.
`<pipeline>`	The pipeline to run: `align`, `add-genome`, `replace-genome`, `add-outgroup`, or `minigraph`. This picks the workflow file (Snakefile), so don't give `-s` or `--snakefile`.
`--configfile my-config.yaml`	The path to your config file. See: Preparing the Snakemake config file.
`-j 20`	The maximum number of jobs that will be submitted to your cluster at one time. Always give it with `-e slurm`: recent Snakemake versions require it, even for a dry-run.
`-e slurm`	Use the SLURM executor plugin to submit jobs to the cluster. If you leave out `-e` and aren't in a SLURM allocation, the wrapper warns that everything will run on the current node.
`--dryrun`	Do not execute anything, just display what would be done.
`--slurm-partition-config <path>`	Optional. Lets the SLURM plugin choose a partition for each job, so the `partition:` entries in your config can be left blank. On the Harvard Cannon cluster, see our partition config.
`--retries N`	Optional. Resubmit failed jobs up to N times, using per-attempt resources if you've set them (see Specifying resources for each rule).

If this completes successfully, you should see a bunch of blue, yellow, and green text on the screen, ending with a summary of the jobs to be run. For the repo's test dataset (6 yeast genomes, with S288C as the reference), it looks like this:

Job stats:
job           count
----------  -------
copy_input        1
minigraph         1
graphmap          1
split             1
join              1
all               1
total             6

The order of the rows can differ between Snakemake versions.

The number of jobs will be higher than indicated by the dryrun!

Note that rule align does not appear in the list of jobs to run. The dry-run stops at rule split, which is a checkpoint in Snakemake terms, and Snakemake prints: "The run involves checkpoint jobs, which will result in alteration of the DAG of jobs (e.g. adding more jobs) after their completion." The align jobs are added once split has run, so they will be run! The total number of jobs will be the number reported during the dryrun + the number of chromosomes in your reference sample (1 align job per chromosome).

If you see any red text, that likely means an error has occurred that must be addressed before you run the pipeline.

Submitting the jobs

If you're satisfied that the --dryrun has completed successfully and you are ready to start submitting Cactus jobs to the cluster, you can do so by simply removing the --dryrun option from the command above:

./cactus_snakemake run minigraph --configfile my-config.yaml -j 20 -e slurm

This will start submitting jobs to SLURM. On your screen, you will see continuous updates regarding job status in blue text. In another terminal, you can also check on the status of your jobs by running squeue -u <your user id>.

Be sure you have a way to keep the main Snakemake process running.

Remember, that some of these steps can take a long time, or your jobs may be stuck in the queue for a while. That means the Snakemake job must have a persistent connection in order to keep running and submitting jobs. There are a few ways this can be accomplished, some better than ~~others~~:

~~Keep your computer on and connected to the cluster. This may be infeasible and you may still suffer from connection issues.~~
Run the Snakemake job in the background by using nohup <run command> &. This will run the command in the background and persist even if you disconnect. However, it makes it difficult to check on the status of your command.
Submit the ./cactus_snakemake run command itself as a SLURM job. This will require preparing and submitting a job script . This is a good solution.
Use a terminal multiplexer like GNU Screen or tmux . These programs allow you to open a sub-terminal within your currently connected terminal that remains even after you disconnect from the server. This is also a good solution.

Depending on the number of genomes, their sizes, and your wait in the queue, you will hopefully have your whole genome alignment within a few days!

Test dataset

Cactus provides a test dataset which we have set up to run in the tests/yeast-minigraph/ folder of the cactus-snakemake repo.

Here is a breakdown of the files so you can investigate them and prepare similar ones for your project:

File/directory	Description
`seq/`	This directory contains the input sequence files for the test dataset in FASTA format.
`yeast-minigraph-cfg.yaml`	This is the config file for this pipeline and has all of the options you would need to set up for your own project.
`yeastPangenome-local.txt`	This is the input file as required by Minigraph-Cactus. It has lines containing the location of the sequence files for each sample.

We recommend running this test dataset before setting up your own project. The config file doesn't need any edits: it downloads the latest Cactus image, uses its own temporary directory inside tests/yeast-minigraph/, and leaves the partitions blank. Its output is written to tests/yeast-minigraph/yeast-minigraph-out/.

The paths in the config file are relative to the dataset's folder, so the test is run from there. Change into tests/yeast-minigraph/, then run prep and a dry-run:

cd tests/yeast-minigraph/
../../cactus_snakemake prep minigraph --configfile yeast-minigraph-cfg.yaml
../../cactus_snakemake run minigraph --configfile yeast-minigraph-cfg.yaml -j 20 -e slurm --dryrun

Prep downloads the Cactus image once, into tests/yeast-minigraph/. If the dry-run completes without error, run the pipeline by removing the --dryrun option:

../../cactus_snakemake run minigraph --configfile yeast-minigraph-cfg.yaml -j 20 -e slurm

Since the partitions are blank, you can optionally add --slurm-partition-config <path> to these commands to have partitions selected automatically.

Pipeline outputs

The pipeline produces many output files including .hal, the Cactus alignment format, .gfa, the graphical fragment assembly format, .vcf, the variant call format, and .vg, the variation graph format.

For more information about all outputs, see Cactus's minigraph documentation section about output .

Questions/troubleshooting

The Minigraph-Cactus docs say to not use hardmasked genomes. How can I tell if my genome FASTA files are hardmasked?

How can I tell if my genome FASTA files are hardmasked?

Hardmasking means you replace low quality or low confidence bases in your assembly with N characters. Unfortunately, there are many reasons for Ns to appear in a genome fasta file, so it is difficult to tell if it is because it is hardmasked based on the presence of Ns alone. Hopefully the source of the file has left some documentation describing how it was prepared...

My jobs were running but my Snakemake process crashed because of connection issues/server maintenance! What do I do?

Snakemake crashes

As long as there wasn't an error with one of the jobs, Snakemake is designed to be able to resume and resubmit jobs pretty seamlessly. You just need to run the same ./cactus_snakemake run command you ran to begin with and it should pick up submitting jobs where it left off. You don't need to run prep again. You could also run a --dryrun first and it should tell you which jobs are left to be done.

My run stopped with an error about prep. What do I do?

Prep errors

Every run reads the record prep wrote, <output_dir>/prep-manifest.yaml, and stops if:

prep hasn't been run for this output directory. Run the prep command given in the error, then run the pipeline again.
the config or input files that prep depended on have changed since prep was run, or a file prep wrote has been changed. The error lists what changed. If you didn't mean to make those changes, undo them. If you did, run prep again with --force:
```
./cactus_snakemake prep <pipeline> --configfile my-config.yaml --force
```
Files already in the output directory were made from the earlier prep and are not deleted. If the change affects the tree, starting over in a new output directory is usually the safer choice.

See Prep for more information.

I want to run this on a cluster with a job scheduler other than SLURM! What do I do?

Clusters other than SLURM?

Generally, it should be relatively easy to install and use the appropriate Snakemake cluster executor .

If you need help or run into problems, please create an issue on the pipeline's github and we'll try and help - though it will likely be up to you to test on your own cluster, since we only have easy access to a cluster running SLURM.

I tried to run the pipeline and I ran into an error that I don't understand or can't resolve. What do I do?

Encountering errors

Please search for or create an issue on the pipeline's github that includes information about your input files, the command you ran, and the error that you are getting. The text of any log files would also be appreciated.

Additionally, if you are at Harvard, there are several ways to contact us to help you through your errors.

I have an idea to improve or add to the pipeline. What do I do?

Pipeline improvements

Great! Please create an issue on the pipeline's github describing your idea so we can discuss its implementation!

Can I run the pipelines without the cactus_snakemake wrapper?

Running without the wrapper

Yes, though we recommend the wrapper for most users. The wrapper only runs Snakemake for you, so the same steps work with Snakemake directly, from a conda environment that has the packages listed in envs/environment.yml in the cactus-snakemake folder.

Without the wrapper, you give the pipeline's workflow file (Snakefile) with -s:

Pipeline	Snakefile
`align`	`cactus.smk`
`add-genome`	`cactus_add_genome.smk`
`replace-genome`	`cactus_replace_genome.smk`
`add-outgroup`	`cactus_add_outgroup.smk`
`minigraph`	`cactus_minigraph.smk`

The prep, dry-run, and run steps are then, shown here for the align pipeline from inside the cactus-snakemake folder:

snakemake -s cactus.smk --configfile my-config.yaml --config prep=T
snakemake -s cactus.smk --configfile my-config.yaml -j 20 -e slurm --dryrun
snakemake -s cactus.smk --configfile my-config.yaml -j 20 -e slurm

As with the wrapper, the prep step is required once per output directory before the pipeline can run. These options can be given with --config:

Command line flag	Description
`--config prep=T`	Run the prep step and exit. Required once per output directory before the pipeline can run.
`--config prep=T prep_force=T`	Run prep again even though the config or input files have changed since the last prep.
`--config debug=T`	The same as prep, but display extra information about the pre-processing steps.
`--config display=T`	Print the current config settings and exit.
`--config info=T`	Display some information about the pipelines, including version and last commit date.
`--config version=T`	Display the version of the pipeline.