Migration reference: 0.x to 1.0
This page lists the changes in refgenie 1.0 that affect commands, code, pipelines, and plugins written for 0.x. To move an existing install, use the upgrade guide.
Commands
Section titled “Commands”Older docs and search results show 0.x commands. Here are their 1.0 forms:
| 0.x | 1.0 |
|---|---|
pip install refgenie | pip install "refgenie>=1.0.0a3" |
refgenie init -c genome_config.yaml | refgenie init |
refgenie seek -c genome_config.yaml -g hg38 -a bowtie2_index | refgenie seek hg38/bowtie2_index |
refgenie pull -g hg38 -a bowtie2_index | refgenie pull hg38/bowtie2_index |
refgenie seek -g hg38 -a chrom_sizes | refgenie seek hg38/fasta_index.chrom_sizes |
refgenie seek hg38/bowtie2_index.dir | refgenie seek hg38/bowtie2_index (see Seek keys) |
refgenie build hg38/fasta --files fasta=hg38.fa.gz | refgenie genome init --fasta hg38.fa.gz --name hg38 (see Build assets) |
There is no -c config file: refgenie finds its files through REFGENIE_HOME_PATH (see Environment variables). The CLI reference lists every command.
Environment variables
Section titled “Environment variables”Refgenie 1.0 does not read $REFGENIE. Use $REFGENIE_HOME_PATH or $REFGENIE_DB_CONFIG_PATH instead. While $REFGENIE is set and $REFGENIE_DB_CONFIG_PATH is not, refgenie prints a warning. See Configure for all variables.
Seek keys
Section titled “Seek keys”Every 0.x asset had an implicit dir seek key that returned its folder. In 1.0, each asset class declares typed seek keys (file, directory, prefix, string, json). Most asset classes have one main seek key, named like the class, which refgenie uses by default. So for most uses of .dir, drop the suffix:
| 0.x | 1.0 |
|---|---|
hg38/bowtie2_index.dir (caller appends /{genome}) | hg38/bowtie2_index (returns the index prefix) |
hg38/bwa_index.dir (caller appends /{genome}.fa) | hg38/bwa_index (returns the index prefix) |
hg38/fasta.dir | hg38/fasta (returns the FASTA file path) |
In a looper pipeline interface (for example, PEPATAC):
--genome-index { refgenie[sample.genome].bowtie2_index.dir }--genome-index { refgenie[sample.genome].bowtie2_index }If a tool needs a directory (for example, STAR's --genomeDir), its asset class declares a directory seek key.
Looper integration
Section titled “Looper integration”The pre-submit hook that fills the refgenie namespace moved:
pre_submit: python_functions: - refgenconf.looper_refgenie_populate - refgenie.integrations.looper.populateIt reads the database from var_templates.refgenie_db_config in the pipeline interface when that is set, and from $REFGENIE_DB_CONFIG_PATH otherwise.
Asset paths
Section titled “Asset paths”Asset files are stored by digest, under data/<genome_digest>/<asset_group>/<asset_digest>/. Refgenie also builds a human-readable tree of symlinks under alias/<alias>/<asset_group>/<asset_name>/, with file names rewritten to the alias. seek returns the alias path by default; --abs (or seek(..., abs_path=True) in Python) returns the digest path. Both point to the same files.
$ refgenie seek hg38/bwa_index:0.7.19<genome_folder>/alias/hg38/bwa_index/0.7.19/hg38
$ refgenie seek --abs hg38/bwa_index:0.7.19<genome_folder>/data/<genome_digest>/bwa_index/<asset_digest>/<genome_digest>Aligner-index recipes
Section titled “Aligner-index recipes”Aligner-index asset classes such as bowtie2_index and bwa_index declare their main seek key as type: prefix, which resolves to <asset_folder>/<genome>, the value bowtie2 -x and bwa mem expect. If you maintain a recipe for an aligner index:
- Declare the seek key as
type: prefixwithvalue: "{genome}". - In the recipe command, use the aligner's prefix option (for example,
bwa index -p <prefix>) so the output files match.
Python API
Section titled “Python API”- The
refgenconf.RefGenConfobject is replaced byrefgenie.Refgenie. Its constructor takesdatabase_config_path: str | Path | None. See the Refgenie Python object. rgc.asset.seek(...)returns astr.
Build logs
Section titled “Build logs”Build logs and other build files (the pypiper log, commands file, profile, stats.yaml, and the completion flag) live in their own tree, <genome_folder>/builds/<genome_alias>/<asset_group>/<asset_name>/, outside the asset folder. Snakemake workflows should get the completion-flag path from rgc.build.target_template(). The flag file name is {genome}_{group}__{asset}.flag.
Plugins
Section titled “Plugins”Plugins still register functions in refgenie.hooks.<hook> entry-point groups. The hooks, the function signature, and where settings live are different.
| 0.x hook | 1.0 |
|---|---|
post_update | post_update: once, after any operation that changed local assets, aliases, or genomes |
pre_pull / post_pull | same, once per asset pulled |
pre_update | removed |
pre_tag / post_tag | removed |
pre_list / post_list | removed |
| (none) | pre_build / post_build |
refgenie plugins shows an entry point in a removed group as unknown hook (never fires).
A hook function is called as func(rg, event), where rg is the Refgenie instance and event is a refgenie.plugins.HookEvent. A function that takes one argument raises TypeError, which refgenie logs with the plugin's name before moving on.
0.x:
def update_config(rgc): for genome, assets in rgc.list_assets_by_genome().items(): for asset in assets: print(genome, asset, rgc.seek(genome, asset))1.0:
from refgenie.models import AssetRegistryPathComponents
def update_config(rg, event): paths = rg.paths() for genome in paths: # genome aliases for group in paths[genome]: # groups with a default asset path = rg.asset.seek_components( AssetRegistryPathComponents(genome=genome, asset_group=group) ) print(genome, group, path)Plugin settings live in the database, not in config-file keys (such as refgenie_nfcore's nextflow_config:). The user runs refgenie plugins set nfcore config_path=/abs/path/nf.config, and the plugin reads rg.plugins.settings("nfcore") in its hook. See Write a plugin.