Refgenie
reference genome manager
What is refgenie?
Section titled “What is refgenie?”Refgenie manages storage, access, and transfer of reference genome resources. It provides command-line and Python interfaces to download pre-built reference genome "assets", like indexes used by bioinformatics tools. It can also build assets for custom genome assemblies. Refgenie provides programmatic access to a standard genome folder structure, so software can swap from one genome to another.
What makes refgenie better?
Section titled “What makes refgenie better?”-
It downloads individual resources from the command line. Think of it as
GitHubfor reference genomes:refgenie pull hg38/bwa_index. -
It's scripted. If you need resources that are not on a server, such as for a custom genome, you can
buildyour own:refgenie build custom_genome/bowtie2_index. -
It finds local asset locations. When you need a path to an asset, you
seekit, so your pipelines are portable across computing environments:refgenie seek hg38/bowtie2_index. -
It works with remote files, which is useful for cloud applications. Get the URL of an asset file on a server without downloading it:
refgenie seekr hg38/fasta. -
It includes a Python API. Tool developers use
from refgenie import Refgenieto get paths to any genome asset from Python. -
It checks genome compatibility. You call genomes by names like "hg38", but refgenie also gives each genome an ID computed from its sequences (a GA4GH sequence collection digest), so assets always match the genome they were built from.
-
It scales. Refgenie keeps its records in a database. It uses a local SQLite file by default and can use PostgreSQL, including on a remote server, for multi-user and distributed setups.
Quick start
Section titled “Quick start”Four commands install refgenie, set it up, download a small asset, and print its path:
pip install "refgenie>=1.0.0a3"refgenie initrefgenie pull hg38/fasta_indexrefgenie seek hg38/fasta_index.chrom_sizes- Install. Version 1.0 is a pre-release, so ask PyPI for it by version: plain
pip install refgeniegives you the old 0.13. See Install. Copied an old command likerefgenie seek -g hg38 -a fasta? See 0.x commands in 1.0. - Initialize.
refgenie initcreates the local database and genome folder, and subscribes to the public server athttps://api.refgenie.org. See Configure for other locations and database options. - Pull.
hg38/fasta_indexis about 12 KB: the FASTA index and chromosome sizes for hg38, without the 3 GB FASTA. Not sure which hg38 you need? See Which hg38 should I use?. - Seek.
refgenie seekprints the local path to a file in the asset, so your scripts work on any computer. See Retrieve paths to assets.
To see everything you can pull, run refgenie listr, or browse refgenie.org. See Download pre-built assets.
Next steps
Section titled “Next steps”Build assets for a custom genome
Section titled “Build assets for a custom genome”If what you need is not on a server, register your genome from its FASTA file and build assets locally. Building needs recipes from a data channel:
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yamlrefgenie data-channel sync refgenie --exists-okrefgenie genome init --fasta mygenome.fa.gz --name mygenomerefgenie build mygenome/bwa_indexSee Build assets.
Get paths to remote assets
Section titled “Get paths to remote assets”Use seekr ("seek remote") to get the URL of a file on a server, here the FASTA index inside the hg38/fasta asset:
refgenie seekr hg38/fasta.faiUse refgenie from Python
Section titled “Use refgenie from Python”from refgenie import Refgenie
rgc = Refgenie()digest = rgc.alias.resolve("hg38") # seek takes the genome's ID (digest), not its namergc.asset.seek(digest, "fasta_index", seek_key_name="chrom_sizes")The Refgenie object connects to the configured database and does everything the CLI does. See the Refgenie Python object.
To read more about why refgenie exists and how it is built, continue to the overview.