Genome initialization and the RefgetStore
When you initialize a genome, refgenie loads its sequences into a local RefgetStore, a content-addressable store where each sequence is identified by its GA4GH digest. You can then retrieve any subsequence from the store, build a FASTA asset from it, and compare genomes.
Prerequisites
Section titled “Prerequisites”- refgenie installed (
pip install "refgenie>=1.0.0a3"; see Install) - refgenie initialized (
refgenie init) - the
fastarecipe, from a synced data channel:
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yamlrefgenie data-channel sync refgenie --exists-okInitialize a genome from a FASTA file
Section titled “Initialize a genome from a FASTA file”refgenie genome init \ --fasta /path/to/hg38.fa.gz \ --name hg38 \ --description "Human genome build GRCh38" \ --species "Homo sapiens"Refgenie then:
- Reads the FASTA file (
.fa,.fa.gz,.fasta, or.fasta.gz). - Computes each sequence's GA4GH digest and stores the sequence in the RefgetStore at
~/.refgenie/genomes/.refget_store/. - Computes a sequence collection digest for the whole genome.
- Registers the genome under that digest, with the alias
hg38. - Builds the genome's
fastaasset. Use--no-buildto skip this step.
Check that the genome is registered:
refgenie genome listThis shows each genome's digest, aliases, source, species, and description.
Initialize a genome from a server
Section titled “Initialize a genome from a server”You can register a genome from a refgenie server or a RefgetStore. First, see what the server offers:
refgenie genome browse --server-url https://api.refgenie.orgWithout --server-url, genome browse lists genomes on your subscribed servers. Then register one by its digest (genome init --server works for any genome the server lists). For example, to register hg38:
refgenie genome init \ --server https://api.refgenie.org \ --digest EiFob05aCWgVU_B_Ae0cypnQut3cxUP1 \ --name hg38 \ --no-buildWithout --no-build, this downloads the genome's sequences to build its fasta asset. With --no-build, refgenie registers the genome without downloading them; refgenie getseq then reads only the bases you ask for. Use --store <url> instead of --server to register a genome straight from a RefgetStore. To register every genome on your subscribed servers, run refgenie genome sync.
The fasta asset
Section titled “The fasta asset”The fasta asset is built from the sequences in the RefgetStore. It has three files:
| File | Seek key | Description |
|---|---|---|
{digest}.fa | fasta (default) | FASTA file with all sequences |
{digest}.fa.fai | fai | FASTA index, computed by refgenie (no samtools needed) |
{digest}.chrom.sizes | chrom_sizes | Sequence names and lengths |
For chromosome sizes alone, pull fasta_index (about 12 KB) and use fasta_index.chrom_sizes. It is the same file as fasta.chrom_sizes, without the 3 GB FASTA.
The files live under ~/.refgenie/genomes/data/{genome_digest}/fasta/{asset_digest}/. seek returns paths in the alias tree, named for the alias you used:
refgenie seek hg38/fasta # ~/.refgenie/genomes/alias/hg38/fasta/default/hg38.farefgenie seek hg38/fasta.fai # .../hg38.fa.fairefgenie seek hg38/fasta.chrom_sizes # .../hg38.chrom.sizesTo rebuild the asset later, run refgenie build hg38/fasta.
Retrieve sequences with getseq
Section titled “Retrieve sequences with getseq”getseq reads sequences straight from the RefgetStore, so you do not need a FASTA file on disk. It registers a genome from your servers on first use, so genome init is optional. With hg38 registered as above:
# Whole chromosomerefgenie getseq -g hg38 -l chrM
# The first 60 bases (1-based, inclusive, like samtools)refgenie getseq -g hg38 -l chrM:1-60
# From a position to the end of the chromosomerefgenie getseq -g hg38 -l chrM:16500
# Write FASTA instead of plain basesrefgenie getseq -g hg38 -l chrM:1-60 --fastaRefgenie resolves the alias hg38 to its genome digest, looks up chrM in the sequence collection, and reads that sequence (or just the requested range) from the store. For a genome registered from a server, it reads only the bytes it needs. A region is 1-based and inclusive; add --zero-based for 0-based, half-open coordinates such as those in a BED file. See Coordinates.
Putting it all together
Section titled “Putting it all together”refgenie initrefgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yamlrefgenie data-channel sync refgenie --exists-ok
# Register a genome and build its fasta assetrefgenie genome init --fasta my_genome.fa.gz --name mygenome --description "My custom genome"refgenie genome list
# Get paths to the fasta asset filesrefgenie seek mygenome/fastarefgenie seek mygenome/fasta.fairefgenie seek mygenome/fasta.chrom_sizes
# Read sequence straight from the storerefgenie getseq -g mygenome -l chr1:1-500Key concepts
Section titled “Key concepts”The RefgetStore
Section titled “The RefgetStore”The RefgetStore (~/.refgenie/genomes/.refget_store/) stores each sequence under its GA4GH digest. As a result:
- Identical sequences in different genomes are stored once.
- Any sequence can be checked against its digest.
- Subsequences can be read without loading a whole chromosome.
Genome init and build
Section titled “Genome init and build”genome init | build | |
|---|---|---|
| Purpose | Register a genome and load its sequences | Create asset files on disk |
| Input | FASTA file, server, or store | A registered genome (and any parent assets) |
| Output | Sequences in the store, a database record, and the fasta asset | Asset files, such as an aligner index |
| Needed for | getseq, build, compare | seek |
getseq works right after genome init.
Compare genomes
Section titled “Compare genomes”Each genome is a GA4GH sequence collection: a set of sequences with names, lengths, and digests. Its identity is the sequence collection digest, so:
- The same sequences, with the same names and in the same order, give the same digest no matter who computes it or where.
- Adding or removing one sequence changes the digest.
- Renaming sequences (for example,
chr1to1) changes the digest, but a comparison still shows that the sequences match.
refgenie compare reports how two genomes relate:
refgenie compare hg38 hg38_customThe output lists which attributes (names, lengths, sequences, and their sorted forms) the two genomes share and how many elements each has. See Compare genomes.
Next steps
Section titled “Next steps”Build more assets, stage them for serving, and push them to cloud storage with the Building and serving tutorial.