Build assets
refgenie pull downloads pre-built assets. Use refgenie build for genomes or assets the server does not have. To build, you need the recipe's inputs and the software it runs.
What can refgenie build?
Section titled “What can refgenie build?”Refgenie builds anything it has a recipe for. Recipes and asset classes come from data channels; a new install has none. Add the refgenie channel first:
refgenie data-channel add refgenie https https://refgenie.github.io/refgenie-registry/index.yamlrefgenie data-channel sync refgenie --exists-okrefgenie recipe listTo write your own, see the recipe specification and asset class specification.
Recipes need inputs
Section titled “Recipes need inputs”A recipe takes three kinds of input:
| Input | Recipe key | Command-line flag | Value |
|---|---|---|---|
| assets | input_assets | --assets | asset registry path |
| files | input_files | --files | file or folder path |
| parameters | input_params | --params | string |
To see what a recipe needs, add -q (--requirements):
$ refgenie build hg38/bowtie2_index -qThe table shows that bowtie2_index needs a fasta asset, not a FASTA file. You must already have a fasta asset for hg38. Refgenie uses it automatically and records the link, so you can always tell which FASTA an index was built from:
refgenie build hg38/bowtie2_indexTo use a different input asset, such as another asset name or another genome's fasta, pass --assets:
refgenie build hg38/bowtie2_index --assets fasta=hg38_cdna/fastaAdd --pull-parents to pull a missing input asset from your server before building.
Other recipes take files or parameters:
refgenie build hg38/refgene_anno --files refgene=REFGENE_FILE.txt.gzrefgenie build hg38/bowtie2_index --params threads=8Recipes need software
Section titled “Recipes need software”Each recipe runs command-line tools. Get them one of three ways.
Run in the recipe's container
Section titled “Run in the recipe's container”Each recipe names a container image in docker_image that has every tool it needs. With Docker or Apptainer installed, add --container:
refgenie build hg38/bowtie2_index --containerRefgenie runs each of the recipe's commands inside that image. It uses Docker when the Docker daemon answers, and Apptainer otherwise (most clusters have Apptainer). Set REFGENIE_CONTAINER_ENGINE to docker or apptainer to choose. The image is pulled the first time it is needed. On a cluster, point REFGENIE_APPTAINER_CACHEDIR (or Apptainer's own APPTAINER_CACHEDIR) at scratch space so the image cache does not fill your home folder. Use --volumes to make extra folders visible inside the container.
Install the tools yourself
Section titled “Install the tools yourself”Without --container, every tool the recipe needs must be on your PATH; see the recipe's docker_image for what it uses (refgenie recipe show <name>).
Use bulker
Section titled “Use bulker”Bulker puts containerized tools on your PATH as if they were installed, so a build without --container can still run each tool in a container.
Name the asset
Section titled “Name the asset”Each build creates an asset inside an asset group. Recipes usually name the asset after the tool version, such as 2.5.4. To choose the name yourself, add it after a colon:
refgenie build hg38/bowtie2_index:my_asset_nameThe asset you build becomes the group's default. See Asset groups and naming.
Stage after building
Section titled “Stage after building”To serve an asset or push it to cloud storage, stage it. Stage in a separate step:
refgenie stage add hg38/bowtie2_indexor while building:
refgenie build hg38/bowtie2_index --stageAdd --push-to with one or more remote names to mark the staged asset for upload; refgenie push then uploads it:
refgenie build hg38/bowtie2_index --stage --push-to my-s3-remoterefgenie push