Skip to content

Refgenie CLI

The installed command is refgenie. This document walks through a first session and then documents every command and flag.

Install from PyPI. Because 1.0 is still a pre-release, pip must be told to accept it:

Terminal window
pip install --pre refgenie

The base install provides the CLI and the Python API. refgenie dash, refgenie serve, refgenie-mcp, and snakemake-based bulk building each require an extra (dash, server, mcp, snakemake); see the README installation section for the extras table. Running a command without its extra prints a message naming the extra to install.

Check the installed version:

Terminal window
$ refgenie --version
refgenie 1.0.0a1

Refgenie stores asset metadata in a database — SQLite by default, PostgreSQL optionally. With no configuration, refgenie creates a refgenie SQLite file under ~/.refgenie (or $REFGENIE_HOME_PATH). Configuration is initialized automatically on first use, but running refgenie init explicitly is the supported starting point: it also creates the genome folder and the stage folder.

Terminal window
$ refgenie init
INFO Database configuration file created at /home/user/.refgenie/refgenie_db_config.yaml.
INFO Genome folder ready: /home/user/.refgenie/genomes
INFO Genome stage folder ready: /home/user/.refgenie/archives
INFO Initialized refgenie backend: 'sqlite:////home/user/.refgenie/refgenie'

No asset classes or recipes — not even fasta — are registered by the package. They come from a data channel, which you must register and sync before building anything.

  1. refgenie init — initialize the config, database, and folders.
  2. refgenie data-channel add ... then refgenie data-channel sync ... — register asset classes and recipes.
  3. refgenie genome init ... — register a genome. This auto-builds the fasta asset, but only once a fasta recipe is registered by step 2. Running genome init before syncing a data channel skips the auto-build and prints a message; --no-build skips the attempt entirely.
  4. refgenie build ... — build other assets (e.g. bwa_index) whose recipes are registered.

A data channel is an index of asset class and recipe definitions. The canonical channel is published at refgenie-registry.

Terminal window
$ refgenie data-channel add my-fav-channel https https://refgenie.github.io/refgenie-registry/index.yaml
INFO Added data channel: my-fav-channel

The three positional arguments are the channel name, its type, and the address of its index.yaml.

List registered channels:

Terminal window
$ refgenie data-channel list
Data Channels
┏━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Name ┃ Type ┃ Index Address ┃ Description ┃ Credentials set ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ my-fav-channel │ https │ https://refgenie.github.io… │ │ False │
└────────────────┴───────┴─────────────────────────────┴─────────────┴─────────────────┘
Terminal window
$ refgenie data-channel sync my-fav-channel --exists-ok
INFO Registered 'fasta' recipe
...
INFO Successfully synced from channel 'my-fav-channel'

--exists-ok skips items already present instead of erroring. Use --exists-overwrite to replace conflicting items.

Recipes contain shell commands, and building an asset runs them. Sync only channels you trust.

Inspect what arrived:

Terminal window
refgenie recipe list
refgenie asset-class list

Register a genome and build its fasta asset

Section titled “Register a genome and build its fasta asset”

genome init computes the genome's sequence-collection digest and, when a fasta recipe is available, builds the fasta asset in the same step.

Terminal window
$ refgenie genome init --fasta rCRSd.fa --name rCRSd --species 'Homo sapiens' \
--description 'human mitochondrial genome'
INFO Asset 'rCRSd/fasta:default' build succeeded
INFO Added: 'rCRSd/fasta:default'
INFO Fasta asset built successfully for rCRSd

The genome is now listed, and its assets have resolvable paths:

Terminal window
$ refgenie list
Refgenie assets. Source: local
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Aliases ┃ Genome digest ┃ Asset group ┃ Asset ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━┩
│ rCRSd │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ fasta │ default │
└─────────┴──────────────────────────────────┴─────────────┴─────────┘
$ refgenie seek rCRSd/fasta
/home/user/.refgenie/genomes/alias/rCRSd/fasta/default/rCRSd.fa

To build the fasta asset separately — for instance after a genome init --no-build — call build with the source file:

Terminal window
refgenie build rCRSd/fasta --files fasta=rCRSd.fa

Once the genome exists and the recipe is registered, other assets build from it:

Terminal window
refgenie build rCRSd/bwa_index

Recipes run real tools. If bwa is not on PATH, either install it or pass -d/--docker to run the recipe in its container. -q/--requirements prints what a recipe needs without building:

Terminal window
refgenie build rCRSd/bwa_index -q
Terminal window
$ refgenie getseq -g rCRSd -l 'rCRSd:0-30'
GATCACAGGTCTATCACCCTATTAACCACT

Most asset commands take one or more registry paths:

<genome>/<asset>
<genome>/<asset>:<tag>
<genome>/<asset>.<seek_key>:<tag>

<genome> is an alias (e.g. hg38). Omitting :<tag> uses the default tag. The .seek_key suffix selects a specific file within an asset (e.g. hg38/fasta.fai).

To name a genome by its sequence-collection digest instead, leave the genome out of the path and pass --genome-digest:

refgenie seek fasta --genome-digest DIGEST

This reaches a genome that has no alias. A digest written into the path is read as an alias, and naming the genome both ways at once is an error. build takes an alias only, because the build folder is named after it.

refgenie --help groups commands as follows.

GroupCommands
Asset managementlist, asset, seek, add, remove, rename, id, build, populate
Remote operationslistr, seekr, pull, push, mirror, populater, compare
Genome managementgenome
Serverserve, dash, subscribe, unsubscribe, catalog-export
Configurationinit, purge, config, plugins, alias, recipe, asset-class, stage
Asset definitionsdata-channel, generate, remote
Sequencesgetseq

Boolean flags shown as -f, --force also accept the negated form (--no-force). Command names use hyphens (asset-class, data-channel, catalog-export).

List available local assets.

refgenie list [-g ALIAS ...] [--genome-digest DIGEST ...]
OptionDescription
-g, --genomeOne or more genome aliases to restrict the listing to.
--genome-digestOne or more genome digests to restrict the listing to.

Alias group. refgenie asset list is equivalent to refgenie list and takes the same -g, --genome and --genome-digest options.

Print the local path of an asset.

refgenie seek ASSET-REGISTRY-PATHS [...]
OptionDescription
-e, --check-existsCheck the returned path for existence on disk.
--absReturn the digest-addressed content path under data/ instead of the human-readable alias path.
--genome-digestName the genome by digest instead of by the alias in the registry path; leave the genome out of the path (e.g. seek fasta --genome-digest DIGEST). Reaches a genome with no alias.

Register an asset that already exists on disk.

refgenie add ASSET-REGISTRY-PATHS [...] -p PATH -c ASSET_CLASS
OptionDescription
-p, --pathRelative local path to the asset. Required.
-c, --asset-className of the asset's asset class. Required.
-d, --descriptionDescription of the asset.
-k, --seek-keysNon-path seek key values, as name=value. Repeat for multiple keys.
--genome-digestName the genome by digest instead of by the alias in the registry path; leave the genome out of the path (e.g. seek fasta --genome-digest DIGEST). Reaches a genome with no alias.

Remove a local asset.

refgenie remove ASSET-REGISTRY-PATHS [...]
OptionDescription
-f, --forceDo not prompt before removing.
-a, --aliasesAlso remove the genome alias if this was the genome's last asset.
--genome-digestName the genome by digest instead of by the alias in the registry path; leave the genome out of the path (e.g. seek fasta --genome-digest DIGEST). Reaches a genome with no alias.

Rename an asset.

refgenie rename ASSET-REGISTRY-PATHS [...] -n NEW_ASSET_NAME
OptionDescription
-n, --new-asset-nameNew name for the asset. Required.
--genome-digestName the genome by digest instead of by the alias in the registry path; leave the genome out of the path (e.g. seek fasta --genome-digest DIGEST). Reaches a genome with no alias.

Return a digest. A genome alias yields the genome digest; a registry path yields the asset digest. A positional name is always an alias; name a genome by digest with --genome-digest.

refgenie id REGISTRY-PATHS [...]
refgenie id --genome-digest DIGEST [--remote]
OptionDescription
--genome-digestName the genome by digest instead of by alias. Alone, prints the digest if the genome is known; with a registry path, leave the genome out of it (e.g. id fasta --genome-digest DIGEST).
-v, --verboseShow detailed genome metadata (sequence count, total length, source).
--validate-storeVerify that the genome's RefgetStore exists and is valid.
--remoteWith --genome-digest, query subscribed seqcolapi servers when the genome is not found locally.
--infoGiven a digest, show its aliases and metadata.

Build genome assets.

refgenie build ASSET-REGISTRY-PATHS [...]
OptionDescription
--asset-descriptionAsset-level description (e.g. built with version 0.3.2).
--recipe-nameRecipe to use.
--recipe-versionRecipe version to use.
-d, --dockerRun all commands in the refgenie docker container.
--pull-parentsAutomatically pull a required parent asset that was not provided.
-q, --requirementsShow the build requirements for the asset and exit.
--stageStage the asset after building. Requires the genome stage folder to be set.
--push-toRemotes, by name or id, to create push intent records for after staging.
--pipeline-kwargsExtra arguments for the build pipeline, as arg_name=arg_val.
--assetsOverride the genome, asset, and tag of parents, e.g. fasta=hg38/fasta:default.
--filesPaths to required input files, e.g. fasta=/path/to/file.fa.gz.
--paramsRequired parameter values, e.g. param1=value1.
--volumesAdditional folders to mount as volumes when using docker.

Replace refgenie registry paths with local paths. Reads the file given by -f, or stdin when -f is omitted.

refgenie populate [-f FILE]
OptionDescription
-f, --fileFile containing registry paths to populate.

List assets available on subscribed servers.

refgenie listr [-g ALIAS ...] [--genome-digest DIGEST ...] [-s URL ...]
OptionDescription
-g, --genomeOne or more local genome aliases to restrict the listing to.
--genome-digestOne or more genome digests to restrict the listing to. The genomes need not exist locally.
-s, --genome-serverOne or more server URLs to use for this call only; not persisted to config.
-p, --append-serverAppend the provided servers to the configured list rather than replacing it.

Print the remote path of an asset.

refgenie seekr ASSET-REGISTRY-PATHS [...]
OptionDescription
-s, --genome-serverOne or more server URLs to use for this call only; not persisted.
-p, --append-serverAppend the provided servers to the configured list.
--genome-digestName the genome by digest instead of by the alias in the registry path; leave the genome out of the path (e.g. seek fasta --genome-digest DIGEST). Reaches a genome with no alias.

Download assets from subscribed servers. Registry paths may be given positionally or with --asset-registry-paths.

refgenie pull ASSET-REGISTRY-PATHS [...]
OptionDescription
--asset-registry-pathsRegistry paths to pull (equivalent to the positional form).
-g, --genomeGenome alias(es), e.g. mm10; comma-separate several for --all or --asset.
--genome-digestGenome digest(s), in place of aliases. With registry paths, one digest, and leave the genome out of the path (pull fasta --genome-digest DIGEST). With --all or --asset, comma-separate several.
--allPull all assets for the specified genome(s).
--all-genomesApply the operation to all genomes available on subscribed servers.
--assetPull one asset type across the specified genomes, e.g. --asset fasta.
--initRegister genome(s) locally (aliases, metadata) without downloading asset files.
--skip-largeDo not pull archives over the size cutoff.
--pull-largePull all archives regardless of size.
--size-cutoffMaximum archive size, in GB, to pull without confirmation. Default 10.
--batchBatch mode: pull all archives regardless of size.
-f, --forceSkip confirmation prompts for multi-asset operations.

Upload staged assets to cloud remotes.

refgenie push [-r REMOTE] [-g ALIAS | --genome-digest DIGEST]
OptionDescription
-r, --remotePush only to this remote (by name or id). Default: all remotes with unpushed assets.
-g, --genomePush only assets for the genome with this alias.
--genome-digestPush only assets for the genome with this digest.
-n, --dry-runShow what would be pushed without executing.
--strategyper_asset (upload each asset) or folder_sync (sync the whole genome stage folder). Default per_asset.

Mirror all assets from all genomes on subscribed servers.

refgenie mirror
OptionDescription
--skip-largeDo not pull archives over the size cutoff.
--pull-largePull all archives regardless of size.
--size-cutoffMaximum archive size, in GB, to pull without confirmation. Default 10.
--batchBatch mode: pull all archives regardless of size.
-f, --forceSkip the confirmation prompt.

Replace refgenie registry paths with remote paths. Reads the file given by -f, or stdin when -f is omitted.

refgenie populater [-f FILE]
OptionDescription
-f, --fileFile containing registry paths to populate.
-s, --genome-serverOne or more server URLs to use for this call only; not persisted.
-p, --append-serverAppend the provided servers to the configured list.

Compare two genomes for compatibility.

refgenie compare ALIAS1 ALIAS2
refgenie compare ALIAS1 --genome-digest DIGEST2
refgenie compare --genome-digest DIGEST1,DIGEST2
OptionDescription
--genome-digestGenome digest(s) to compare, in place of aliases. Aliases and digests together must name exactly two genomes.

Initialize a genome from a FASTA file, a refgenie server, or a RefgetStore. When initialized from a FASTA file it also builds the fasta asset (fa, fai, chrom.sizes).

refgenie genome init -n NAME [--fasta PATH | --server URL | --store URL]
OptionDescription
-n, --nameOne or more alias names for the genome. Required.
--fastaPath to a local FASTA file.
--serverURL of a refgenie server to initialize from.
--storeURL of a RefgetStore to initialize from; requires --digest or --namespace.
--namespaceNamespace for alias lookup when using --store.
--digestSeqcol digest of the genome.
-d, --descriptionGenome description, e.g. Human genome build 38.
-s, --speciesSpecies name, e.g. Homo sapiens.
--fhrPath to an FHR .fhr.json metadata file to apply after init. When given it is authoritative for description/species and writes the RefgetStore sidecar.
-f, --forceAllow re-initialization of an existing genome (adds new aliases).
--build / --no-buildBuild the fasta asset after initialization. Default on; runs only when a fasta recipe is registered.

Apply FHR metadata to an already-registered genome, with no rebuild. Updates the genome's description and species and the RefgetStore sidecar.

refgenie genome set-metadata (-n NAME | --digest DIGEST) --fhr PATH
OptionDescription
-n, --nameGenome alias to update.
--digestGenome seqcol digest to update (alternative to --name).
--fhrPath to the FHR .fhr.json metadata file to apply. Required.

List all genomes, with digests, aliases, source, species, and description.

refgenie genome list

Remove a genome and all its assets.

refgenie genome remove (--genome ALIAS [...] | --genome-digest DIGEST [...])
OptionDescription
--genomeGenome alias(es) to remove.
--genome-digestGenome digest(s) to remove. One of the two is required.
-f, --forceDo not prompt before removing.

Browse genomes available on a refgenie server or RefgetStore.

refgenie genome browse [--server-url URL]
OptionDescription
--server-urlURL of a refgenie server or RefgetStore. Defaults to subscribed server(s).
--pagePage number for paginated results. Default 0.
--page-sizeNumber of results per page. Default 20.

Bulk-register all genomes from subscribed servers or a remote source.

refgenie genome sync [--server-url URL]
OptionDescription
--server-urlURL of a remote source to sync from. Defaults to all subscribed server(s).
--page-sizeNumber of collections to request per page. Default 1000.

Start the production refgenie server. Requires the server extra.

refgenie serve [-p PORT]
OptionDescription
-p, --portPort to run the server on. Default 8000.
-r, --reloadEnable auto-reload on code changes (for development).

Start the local refgenie web UI. Requires the dash extra.

refgenie dash [-p PORT] [-b {off,read,full}]
OptionDescription
-p, --portPort to run the dashboard on. Default 8080.
-b, --bridgeLocalhost-bridge mode for this run, overriding $REFGENIE_BRIDGE_MODE: off = no cross-origin access, read = allowlisted public origins may read, full = additionally allows cross-origin pull.

Add refgenieserver URLs to the config.

refgenie subscribe -s URL [...]
OptionDescription
-s, --genome-serverOne or more URLs to add to the subscription list.
-r, --resetOverwrite the current list of server URLs.

Remove refgenieserver URLs from the config.

refgenie unsubscribe -s URL [...]
OptionDescription
-s, --genome-serverOne or more URLs to remove from the subscription list.

Export a publish catalog covering pushed assets only, for a server to import.

refgenie catalog-export [--dest PATH] [--https-prefix URL]
OptionDescription
--destPath to write the publish-catalog SQLite artifact to.
--https-prefixPublic https base URL mirroring the stage folder that refgenie push uploaded to, e.g. https://<bucket>.s3.amazonaws.com/assets. Download links are served from here.

Initialize the refgenie configuration, database, and folders.

refgenie init
OptionDescription
-f, --genome-folderAbsolute path to the parent folder for refgenie-managed assets.
-a, --genome-stage-folderAbsolute path to the parent stage folder for refgenie-managed assets; used by refgenieserver.
-v, --config-versionConfig version to initialize the config file with.

Purge the genome configuration.

refgenie purge
OptionDescription
-f, --forceDo not prompt before purging.
refgenie config get

config get displays the current configuration, including the database connection and the environment-derived settings. config set is not yet implemented.

plugins list / plugins set / plugins unset

Section titled “plugins list / plugins set / plugins unset”
refgenie plugins [list]
refgenie plugins set PLUGIN KEY=VALUE [KEY=VALUE ...]
refgenie plugins unset PLUGIN [KEY ...]

plugins list (also bare refgenie plugins) shows every installed plugin: its hook, entry-point name, target, package and status (ok, disabled, load error: ..., or unknown hook (never fires) for an entry point in a legacy group such as refgenie.hooks.pre_tag). Below that it shows the stored plugin settings; settings for a plugin that is not installed are marked (not installed). When this refgenie runs no plugins it says why: REFGENIE_DISABLE_PLUGINS, or server mode.

plugins set stores settings for a plugin in the database, merging them with what is already there. Each setting is key=value, split on the first =. Settings may be stored before the plugin is installed; refgenie warns and stores them anyway. plugins unset removes the given keys, or every setting for the plugin when no key is given.

refgenie plugins set nfcore config_path=/abs/path/nf.config
refgenie plugins unset nfcore config_path
refgenie alias get [-a ALIAS ...] [-g DIGEST ...]
refgenie alias set -a ALIAS [...] [-d DIGEST]
refgenie alias remove -a ALIAS [...]

alias get options (mutually exclusive):

OptionDescription
-a, --aliasesAliases to get the digests for.
-g, --genome-digestsGenome digests to get the aliases for.

alias set options:

OptionDescription
-a, --aliasesAliases to set. Required.
-d, --digestDigest to set the aliases on.
-r, --resetRemove all aliases before setting the new ones.
-f, --forceForce the action even if the genome does not exist.

alias remove options:

OptionDescription
-a, --aliasesAliases to remove. Required.
refgenie recipe list
refgenie recipe show RECIPE-NAME [--recipe-version VERSION]
refgenie recipe requirements RECIPE-NAME [--recipe-version VERSION]
refgenie recipe add --source PATH_OR_URL [-f]
refgenie recipe remove RECIPE-NAME [--recipe-version VERSION]
SubcommandDescription
listList local recipes.
showDisplay a recipe.
requirementsShow a recipe's requirements.
addAdd a recipe from a path or URL. --source is required; -f, --force overwrites.
removeRemove a recipe.
refgenie asset-class list
refgenie asset-class show ASSET-CLASS-NAME [--asset-class-version VERSION]
refgenie asset-class add --source PATH_OR_URL [-f]
refgenie asset-class remove ASSET-CLASS-NAME [--asset-class-version VERSION]
SubcommandDescription
listList local asset classes.
showDisplay an asset class.
addAdd an asset class from a path or URL. --source is required; -f, --force forces the action.
removeRemove an asset class.

Manage staged assets — the archive area that refgenie push uploads from.

refgenie stage add ASSET-REGISTRY-PATHS [...]
refgenie stage remove ASSET-REGISTRY-PATHS [...]
refgenie stage list
SubcommandDescription
addStage an asset.
removeUnstage an asset.
listList staged assets, with digest, name, mode, and size.

stage add and stage remove also take --genome-digest DIGEST to name the genome by digest instead of by the alias in the registry path.

refgenie data-channel add NAME TYPE INDEX-ADDRESS [-d DESCRIPTION]
refgenie data-channel list
refgenie data-channel show NAME
refgenie data-channel validate NAME
refgenie data-channel sync NAME [--exists-ok | --exists-overwrite]
refgenie data-channel remove NAME

add positional arguments: the channel name, its type, and the address of its index YAML file.

add optionDescription
-d, --descriptionDescription of the data channel.
--usernameUsername for authentication.
--passwordPassword for authentication.
--tokenAuthentication token.
sync optionDescription
--exists-okSkip existing assets/recipes without error.
--exists-overwriteDelete conflicting items before adding.

--exists-ok and --exists-overwrite are mutually exclusive.

Generate a Snakemake file from the refgenie configuration. Requires the snakemake extra to run the result.

refgenie generate snakefile -o OUTPUT_PATH [-s TEMPLATE_PATH]
OptionDescription
-o, --output-pathPath to save the generated Snakefile. Required.
-s, --snakefile-template-pathPath to the Snakefile template.

Configure the cloud destinations that refgenie push uploads to. Every command that takes a remote (remote remove, remote status -r, push -r, build --push-to) accepts its name or its numeric id. Several remotes can share a type; names are unique.

refgenie remote add --type {s3,http,https} --prefix PREFIX --name NAME [--push-command CMD]
refgenie remote list
refgenie remote status [-r REMOTE]
refgenie remote remove REMOTE
add optionDescription
--typeType of the remote: s3, http, or https. Required.
--prefixPrefix/identifier for the remote. Required.
--nameName of the remote. Must be unique and not all digits, since digits read as an id. Required.
--push-commandShell command template for pushing assets. Placeholders: {local_path}, {relative_path}, {prefix}, {genome_stage_folder}. Example: aws s3 cp {local_path} s3://bucket/{relative_path}.
status optionDescription
-r, --remoteShow status for only this remote (by name or id).
remove argumentDescription
REMOTEName or id of the remote to remove. Required.

Retrieve a sequence region from a genome. Coordinates are 0-based and half-open.

refgenie getseq (-g ALIAS | --genome-digest DIGEST) -l LOCUS
OptionDescription
-g, --genomeGenome alias, e.g. mm10.
--genome-digestGenome digest, in place of an alias. One of -g or --genome-digest is required.
-l, --locusCoordinates of the desired sequence, e.g. chr1:50000-50200. Required.