Skip to content

Genome initialization and the RefgetStore

When you initialize a genome, refgenie loads its sequences into a local RefgetStore, a content-addressable store where each sequence is identified by its GA4GH digest. You can then retrieve any subsequence from the store, build a FASTA asset from it, and compare genomes.

  • refgenie installed (pip install "refgenie>=1.0.0a3"; see Install)
  • refgenie initialized (refgenie init)
  • the fasta recipe, from a synced data channel:
Terminal window
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yaml
refgenie data-channel sync refgenie --exists-ok
Terminal window
refgenie genome init \
--fasta /path/to/hg38.fa.gz \
--name hg38 \
--description "Human genome build GRCh38" \
--species "Homo sapiens"

Refgenie then:

  1. Reads the FASTA file (.fa, .fa.gz, .fasta, or .fasta.gz).
  2. Computes each sequence's GA4GH digest and stores the sequence in the RefgetStore at ~/.refgenie/genomes/.refget_store/.
  3. Computes a sequence collection digest for the whole genome.
  4. Registers the genome under that digest, with the alias hg38.
  5. Builds the genome's fasta asset. Use --no-build to skip this step.

Check that the genome is registered:

Terminal window
refgenie genome list

This shows each genome's digest, aliases, source, species, and description.

You can register a genome from a refgenie server or a RefgetStore. First, see what the server offers:

Terminal window
refgenie genome browse --server-url https://api.refgenie.org

Without --server-url, genome browse lists genomes on your subscribed servers. Then register one by its digest (genome init --server works for any genome the server lists). For example, to register hg38:

Terminal window
refgenie genome init \
--server https://api.refgenie.org \
--digest EiFob05aCWgVU_B_Ae0cypnQut3cxUP1 \
--name hg38 \
--no-build

Without --no-build, this downloads the genome's sequences to build its fasta asset. With --no-build, refgenie registers the genome without downloading them; refgenie getseq then reads only the bases you ask for. Use --store <url> instead of --server to register a genome straight from a RefgetStore. To register every genome on your subscribed servers, run refgenie genome sync.

The fasta asset is built from the sequences in the RefgetStore. It has three files:

FileSeek keyDescription
{digest}.fafasta (default)FASTA file with all sequences
{digest}.fa.faifaiFASTA index, computed by refgenie (no samtools needed)
{digest}.chrom.sizeschrom_sizesSequence names and lengths

For chromosome sizes alone, pull fasta_index (about 12 KB) and use fasta_index.chrom_sizes. It is the same file as fasta.chrom_sizes, without the 3 GB FASTA.

The files live under ~/.refgenie/genomes/data/{genome_digest}/fasta/{asset_digest}/. seek returns paths in the alias tree, named for the alias you used:

Terminal window
refgenie seek hg38/fasta # ~/.refgenie/genomes/alias/hg38/fasta/default/hg38.fa
refgenie seek hg38/fasta.fai # .../hg38.fa.fai
refgenie seek hg38/fasta.chrom_sizes # .../hg38.chrom.sizes

To rebuild the asset later, run refgenie build hg38/fasta.

getseq reads sequences straight from the RefgetStore, so you do not need a FASTA file on disk. It registers a genome from your servers on first use, so genome init is optional. With hg38 registered as above:

Terminal window
# Whole chromosome
refgenie getseq -g hg38 -l chrM
# The first 60 bases (1-based, inclusive, like samtools)
refgenie getseq -g hg38 -l chrM:1-60
# From a position to the end of the chromosome
refgenie getseq -g hg38 -l chrM:16500
# Write FASTA instead of plain bases
refgenie getseq -g hg38 -l chrM:1-60 --fasta

Refgenie resolves the alias hg38 to its genome digest, looks up chrM in the sequence collection, and reads that sequence (or just the requested range) from the store. For a genome registered from a server, it reads only the bytes it needs. A region is 1-based and inclusive; add --zero-based for 0-based, half-open coordinates such as those in a BED file. See Coordinates.

Terminal window
refgenie init
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yaml
refgenie data-channel sync refgenie --exists-ok
# Register a genome and build its fasta asset
refgenie genome init --fasta my_genome.fa.gz --name mygenome --description "My custom genome"
refgenie genome list
# Get paths to the fasta asset files
refgenie seek mygenome/fasta
refgenie seek mygenome/fasta.fai
refgenie seek mygenome/fasta.chrom_sizes
# Read sequence straight from the store
refgenie getseq -g mygenome -l chr1:1-500

The RefgetStore (~/.refgenie/genomes/.refget_store/) stores each sequence under its GA4GH digest. As a result:

  • Identical sequences in different genomes are stored once.
  • Any sequence can be checked against its digest.
  • Subsequences can be read without loading a whole chromosome.
genome initbuild
PurposeRegister a genome and load its sequencesCreate asset files on disk
InputFASTA file, server, or storeA registered genome (and any parent assets)
OutputSequences in the store, a database record, and the fasta assetAsset files, such as an aligner index
Needed forgetseq, build, compareseek

getseq works right after genome init.

Each genome is a GA4GH sequence collection: a set of sequences with names, lengths, and digests. Its identity is the sequence collection digest, so:

  • The same sequences, with the same names and in the same order, give the same digest no matter who computes it or where.
  • Adding or removing one sequence changes the digest.
  • Renaming sequences (for example, chr1 to 1) changes the digest, but a comparison still shows that the sequences match.

refgenie compare reports how two genomes relate:

Terminal window
refgenie compare hg38 hg38_custom

The output lists which attributes (names, lengths, sequences, and their sorted forms) the two genomes share and how many elements each has. See Compare genomes.

Build more assets, stage them for serving, and push them to cloud storage with the Building and serving tutorial.