Skip to content

CLI tutorial

This tutorial covers common refgenie tasks on the command line: downloading assets, getting paths to them, managing aliases, and building your own. The examples use rCRSd, the small human mitochondrial genome, so downloads take seconds.

  • Python 3.11 or higher
  • An internet connection
Terminal window
pip install "refgenie>=1.0.0a3"
refgenie --version
Terminal window
refgenie init

This creates ~/.refgenie/ with a SQLite database, a genomes/ folder for assets, and a database config file. It also subscribes you to the public server at https://api.refgenie.org.

Terminal window
refgenie config get

This shows your genome folder, stage folder, database config path, and subscribed servers. See Configure for details.

A refgenie server hosts pre-built assets. You are already subscribed to https://api.refgenie.org. To add another server, such as one your lab runs:

Terminal window
refgenie subscribe -s https://my-lab-server.example.org

To remove it:

Terminal window
refgenie unsubscribe -s https://my-lab-server.example.org
Terminal window
refgenie listr
Refgenie assets. Source: https://api.refgenie.org
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Aliases ┃ Genome digest ┃ Asset group ┃ Asset ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ bowtie2_index │ 2.5.5 │
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ bwa_index │ 0.7.19 │
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ fasta │ default │
...

Each row is one asset. The asset group is the kind of asset (bowtie2_index), and the asset is a named build within the group (2.5.5). To list the assets for one genome, name it: refgenie listr -g rCRSd. It need not be on your machine yet.

Terminal window
refgenie pull rCRSd/fasta

Refgenie looks up the rCRSd alias on the server to find its genome digest, downloads the asset, and registers it in your local database. Pull several at once by listing more registry paths:

Terminal window
refgenie pull rCRSd/bowtie2_index rCRSd/bwa_index
Terminal window
refgenie list
Refgenie assets. Source: local
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Aliases ┃ Genome digest ┃ Asset group ┃ Asset ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ bowtie2_index │ 2.5.5 │
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ bwa_index │ 0.7.19 │
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ fasta │ default │
└─────────┴──────────────────────────────────┴───────────────┴─────────┘

seek returns the local path to an asset:

Terminal window
refgenie seek rCRSd/fasta
/home/user/.refgenie/genomes/alias/rCRSd/fasta/default/rCRSd.fa

Some assets contain several files. The fasta asset holds the FASTA file, its index, and chromosome sizes. Pick a file with a seek key:

Terminal window
refgenie seek rCRSd/fasta.fai # FASTA index
refgenie seek rCRSd/fasta.chrom_sizes # chromosome sizes

For aligner indexes, seek returns the index prefix that the aligner expects:

Terminal window
bowtie2 -x $(refgenie seek rCRSd/bowtie2_index) -U reads.fq

Use the bowtie2 version that built the index: refgenie seek rCRSd/bowtie2_index.version prints it (2.5.5); see the FAQ for installing it.

Using seek in scripts, instead of fixed paths, makes them work on any computer.

seekr returns the URL of an asset file on a server, without downloading it:

Terminal window
refgenie seekr rCRSd/fasta

Refgenie identifies each genome by a digest computed from its sequences. Aliases like rCRSd or hg38 are names that point to these digests. When you pull by alias, refgenie gets the matching digest from the server.

Terminal window
# List aliases
refgenie alias get
# Add aliases to a genome (comma-separate several)
refgenie alias set -a mito,human_mt -d jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP
# Remove an alias
refgenie alias remove -a human_mt

See Manage aliases.

For genomes or assets that no server has, build them yourself. Building needs recipes, which come from a data channel:

Terminal window
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yaml
refgenie data-channel sync refgenie --exists-ok

Register your genome from its FASTA file. This also builds its fasta asset:

Terminal window
refgenie genome init --fasta /path/to/my_genome.fa.gz --name my_genome --description "My custom genome"

Then build other assets from it (this one needs bowtie2 installed):

Terminal window
refgenie build my_genome/bowtie2_index

See Build assets, and the Building and serving tutorial for staging, serving, and pushing assets to the cloud.

CommandDescription
refgenie initInitialize configuration
refgenie config getView current configuration
refgenie subscribe -s URLAdd a server
refgenie unsubscribe -s URLRemove a server
refgenie listrList remote assets
refgenie pull GENOME/ASSETDownload an asset
refgenie listList local assets
refgenie seek GENOME/ASSETGet local asset path
refgenie seekr GENOME/ASSETGet remote asset URL
refgenie alias getView genome aliases
refgenie alias set -a ALIAS -d DIGESTSet a genome alias
refgenie genome init --fasta FILE --name NAMERegister a genome from a FASTA file
refgenie build GENOME/ASSETBuild an asset