The `Refgenie` class is a key component of the Refgenie package, which is used for managing and organizing reference genome files. It provides a set of methods and functionalities to interact with reference genome assets, and other related resources. ## Purpose of this file This file serves as a tutorial for using the `Refgenie` Python API. It demonstrates how to set up a temporary directory for storing reference genome assets, configure the `Refgenie` instance, and perform various operations such as listing available assets, retrieving asset information, and managing data channels. In order to learn more about any of the contepts indicated in the code, please refer to a specific section of the documentation. ## Installation Before the package gets released, clone the repository install, for example using `uv`: ```bash git clone cd refgenie1 uv pip install . ``` ## Configuration First, let's create a temporary directory that will be used to store the refgenie assets. ```python from pathlib import Path from rich import print import os ``` Let's set a temporary directory to store the refgenie assets. ```python from tempfile import TemporaryDirectory REFGENIE_CODE_PATH = Path.cwd().parent / "refgenie" # set the environment variable archive_tmp_dir = TemporaryDirectory(prefix="refgenie_archive_demo_").name os.environ["REFGENIE_GENOME_STAGE_FOLDER"] = archive_tmp_dir tmp_dir = TemporaryDirectory(prefix="refgenie_demo_").name os.environ["REFGENIE_GENOME_FOLDER"] = tmp_dir # set the REFGENIE_DB_CONFIG_PATH to a sqlite config file in the refgenie package os.environ["REFGENIE_DB_CONFIG_PATH"] = (REFGENIE_CODE_PATH / "config" / "sqlite_config.yaml").as_posix() ``` Let's inspect the refgenie configuration object. ```python from refgenie.config import config print(config) ```

RefgenieConfig(
    log_level=<LogLevel.INFO: 'INFO'>,
    genome_folder=PosixPath('/tmp/refgenie_demo_2hzk45es'),
    genome_stage_folder=PosixPath('/tmp/refgenie_archive_demo_3ps6xfvv'),
    database_config_path=PosixPath('/home/nsheff/Dropbox/workspaces/refgenie/repos/refgenie1/refgenie/config/sqlite
_config.yaml'),
    bridge_mode='read',
    bridge_origins='https://refgenie.org,https://ui.refgenie.org',
    bridge_origin_regex='',
    bridge_expose_paths=False
)

### Database backend As you can see, refgenie configuration points to a database configuration file, as by default refgenie is backed by a SQLite database. Let's inspect the refgenie database configuration file. ```python %cat {config.database_config_path} ``` type: sqlite path: ~/refgenie_db/refgenie Make sure the directory where the SQLite database file is stored exists, and create it if it doesn't. ```python !refgenie purge --force !rm -rf ~/refgenie_db !mkdir -p ~/refgenie_db ``` Traceback (most recent call last): File "/home/nsheff/.local/bin/refgenie", line 5, in from refgenie.cli.cli_pydantic import main_cli ModuleNotFoundError: No module named 'refgenie.cli.cli_pydantic' In practice, you don't even need to create the configuration file manually, as refgenie ships with a default configuration file that is used if no configuration file is provided. Just as we've seen above. For production deployments you may want to use a different database backend, such as MySQL or PostgreSQL. In this case, you can provide the database configuration file path by setting `REFGENIE_DB_CONFIG_PATH` environment variable, or even set/override the database engine using `database_engine` in the `Refgenie` constructor. The object must be a `sqlalchemy.engine.Engine` object. ### Refgenieserver client Similarly, refgenie ships with a Refgenieserver client, which is used by default to retrieve remote genome assets and does not need to be replaced in majority of use cases. However, you can provide a custom URL-client mapping to `Refgenie` constructor, by setting the `server_client_mapping` argument. Please note that, the clients need to follow a specific interface, defined in `refgenie.server.ServerClient` protocol. More details below. ```python from refgenie.managers.sources.client import ServerClient from rich import inspect inspect( ServerClient, methods=True, docs=True, help=True, title="ServerClient Protocol structure", ) ```

╭──────────────────────────────────────── ServerClient Protocol structure ────────────────────────────────────────╮
│ class ServerClient(*args, **kwargs):                                                                            │
│                                                                                                                 │
│ Protocol for the server client.                                                                                 │
│                                                                                                                 │
│             server_url = <property object at 0x7ad9be7b03b0>                                                    │
│          download_file = def download_file(self, asset_digest: str, file_path: str, output_path: pathlib.Path)  │
│                          -> pathlib.Path: Download a single file from a file-mode asset.                        │
│ download_with_progress = def download_with_progress(self, operation_id: str, output_path: pathlib.Path, params: │
│                          dict | None = None, url_format_params: dict[str, str | None] = None, name: str | None  │
│                          = None) -> pathlib.Path:                                                               │
│                          Download the asset served by the given operation_id to output_path,                    │
│                          showing progress along the way.                                                        │
│                                                                                                                 │
│                          Implementations should report byte progress to `refgenie.progress`                     │
│                          whenever `refgenie.progress.active_sink()` is not None, and should not                 │
│                          build a `rich` live display in that case -- something other than a                     │
│                          terminal is consuming the progress.                                                    │
│                    get = def get(self, operation_id: str, params: dict | None = None, url_format_params:        │
│                          dict[str, str | None] = None) -> dict: Send a GET request to the specified operation   │
│                          ID.                                                                                    │
│        get_all_aliases = def get_all_aliases(self, params: dict | None = None) -> list[dict]: Get all aliases   │
│                          using pagination.                                                                      │
│   get_all_asset_groups = def get_all_asset_groups(self, params: dict | None = None) -> list[dict]: Get all      │
│                          asset groups using pagination.                                                         │
│         get_all_assets = def get_all_assets(self, params: dict | None = None) -> list[dict]: Get all assets     │
│                          using pagination.                                                                      │
│        get_all_genomes = def get_all_genomes(self, params: dict | None = None) -> list[dict]: Get all genomes   │
│                          using pagination.                                                                      │
│  get_all_staged_assets = def get_all_staged_assets(self, params: dict | None = None) -> list[dict]: Get all     │
│                          staged assets using pagination.                                                        │
│    get_asset_file_list = def get_asset_file_list(self, asset_digest: str) -> list[str]: Get the list of files   │
│                          available for a file-mode asset.                                                       │
│       get_asset_groups = def get_asset_groups(self, params: dict | None = None) -> list[dict]: Get one page of  │
│                          asset groups, unwrapping the paginated response envelope.                              │
│             get_assets = def get_assets(self, params: dict | None = None) -> list[dict]: Get one page of        │
│                          assets, unwrapping the paginated response envelope.                                    │
│      get_staged_assets = def get_staged_assets(self, params: dict | None = None) -> list[dict]: Get one page of │
│                          staged assets, unwrapping the paginated response envelope.                             │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

First, let's import the `Refgenie` class from the `refgenie` package. ```python from refgenie import GenomeAlias, Refgenie refgenie = Refgenie(suppress_migrations=True) ``` Let's ensure we start with a clean slate by removing any existing refgenie metadata and initializing a new refgenie instance. ```python refgenie.init() # initialize new refgenie instance ```

INFO     Genome folder ready: /tmp/refgenie_demo_2hzk45es                                          lifecycle.py:182

INFO     Genome stage folder ready: /tmp/refgenie_archive_demo_3ps6xfvv                            lifecycle.py:186

INFO     Initialized refgenie backend: 'sqlite:////home/nsheff/refgenie_db/refgenie'               lifecycle.py:106

Before we can pull or build a `fasta` asset, refgenie needs to know the `fasta` asset class and recipe. Let's register them from the recipes repository (we'll add a whole data channel of recipes later, but for now we just need `fasta`): ```python refgenie.asset_class.add( "https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/fasta_asset_class.yaml" ) refgenie.recipe.add( "https://github.com/refgenie/recipes/raw/refgenie1/recipes/fasta_asset_recipe.yaml" ) ```

INFO     Reading YAML from URL:                                                                            io.py:31
         https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/fasta_asset_class.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/fasta_asset_class.yaml                    
         "HTTP/1.1 302 Found"

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://raw.githubusercontent.com/refgenie/recipes/refgenie1/asset_classes/fasta_asset_cla                
         ss.yaml "HTTP/1.1 200 OK"

INFO     Registered 'fasta' asset class                                                           asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://github.com/refgenie/recipes/raw/refgenie1/recipes/fasta_asset_recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://github.com/refgenie/recipes/raw/refgenie1/recipes/fasta_asset_recipe.yaml                         
         "HTTP/1.1 302 Found"

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://raw.githubusercontent.com/refgenie/recipes/refgenie1/recipes/fasta_asset_recipe.ya                
         ml "HTTP/1.1 200 OK"

INFO     Registered 'fasta' recipe                                                                    recipe.py:140

Recipe(id=1, name='fasta', version='0.1.0', description='DNA sequences in the FASTA format, exported from RefgetStore. Includes FASTA index (.fai) and chromosome sizes file. Requires genome to be initialized in RefgetStore first (via refgenie genome init).', output_asset_class_id=1, command_templates=['refgenie-build-fasta {{values.refget_store_path}} {{values.genome_digest}} {{values.output_folder}}'], input_params=None, input_files=None, input_assets=None, docker_image=None, custom_seek_keys={}, default_asset='default', inherent=None, updated_at=datetime.datetime(2026, 8, 21, 1, 55, 43, 254318, tzinfo=datetime.timezone.utc), created_at=datetime.datetime(2026, 8, 21, 1, 55, 43, 254321, tzinfo=datetime.timezone.utc)) Let's subscribe to the default refgenie server. This method will reach out to the server at the provided URL and query the OpenAPI specification to determine whether ther server is refgenie-compatible. If it is, the server will be added to the list of subscribed servers. Note: there's currently no public compatible refgenieserver instance deployed, so the following code snippets use a local refgenieserver instance serving the latest API. ```python refgenie.servers.subscribe("http://localhost:8000") ```

INFO     Subscribed to servers: http://localhost:8000                                           configuration.py:55

And that's it! We have now configured a refgenie instance and subscribed to a refgenie-compatible server. We can now start using the refgenie instance to manage reference genome assets. ### Pull an asset Let's initialize a new genome by pulling an asset of fasta class. This will create a new directory in the `data` subdirectory of the `genome_folder` and mirror it in the `alias` directory with symbolic links, rather than copies of the files. ```python refgenie.pull(genome=GenomeAlias("rCRSd"), asset_group_name="fasta") ```

INFO     HTTP Request: GET http://localhost:8000/openapi.json "HTTP/1.1 200 OK"                     _client.py:1025

INFO     Connected to server: http://localhost:8000 title='Refgenieserver REST API' version='1.0.0a1' client.py:221
         description='a web interface and RESTful API for reference genome assets'

WARNING  No local digest for genome alias: rCRSd. Setting genome identity with server:                puller.py:452
         http://localhost:8000

INFO     HTTP Request: GET http://localhost:8000/rgstore.json "HTTP/1.1 200 OK"                     _client.py:1025

INFO     http://localhost:8000 is not a RefgetStore root; looking for a server service-info.         genomes.py:355

INFO     HTTP Request: GET http://localhost:8000/service-info "HTTP/1.1 200 OK"                     _client.py:1025

INFO     HTTP Request: GET http://localhost:8000/v4/aliases/rCRSd "HTTP/1.1 200 OK"                 _client.py:1025

INFO     HTTP Request: GET http://localhost:8000/v4/genomes/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP        _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Resolved alias 'rCRSd' via server client: jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP         genome_bootstrap.py:110

INFO     Determined digest for rCRSd: jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP                      genome_bootstrap.py:118

INFO     HTTP Request: GET http://localhost:8000/rgstore.json "HTTP/1.1 200 OK"                     _client.py:1025

INFO     http://localhost:8000 is not a RefgetStore root; looking for a server service-info.         genomes.py:355

INFO     HTTP Request: GET http://localhost:8000/service-info "HTTP/1.1 200 OK"                     _client.py:1025

INFO     Added genome: jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP                                                genome.py:90

INFO     Added alias: rCRSd                                                                            alias.py:305

INFO     Querying server http://localhost:8000 for jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP/fasta             puller.py:691

INFO     HTTP Request: GET                                                                          _client.py:1025
         http://localhost:8000/v4/asset_groups?genome_digest=jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP&asset                
         _group_name=fasta "HTTP/1.1 200 OK"

INFO     HTTP Request: GET http://localhost:8000/v4/assets?asset_group_id=1&name= "HTTP/1.1 200 OK" _client.py:1025

INFO     HTTP Request: GET                                                                          _client.py:1025
         http://localhost:8000/v4/relationships/03aa63e8accae38cf4d6bcbb7865ef85f95a54020c4f4d48921                
         e67ce382f188c?expand=true "HTTP/1.1 200 OK"

INFO     HTTP Request: GET                                                                          _client.py:1025
         http://localhost:8000/v4/staged_assets?asset_digest=03aa63e8accae38cf4d6bcbb7865ef85f95a54                
         020c4f4d48921e67ce382f188c "HTTP/1.1 200 OK"

/home/nsheff/.local/lib/python3.12/site-packages/rich/live.py:260: UserWarning: install "ipywidgets" for Jupyter 
support
  warnings.warn('install "ipywidgets" for Jupyter support')

INFO     HTTP Request: GET                                                                          _client.py:1025
         http://localhost:8000/v4/archives/03aa63e8accae38cf4d6bcbb7865ef85f95a54020c4f4d48921e67ce                
         382f188c/download "HTTP/1.1 200 OK"

INFO     Extracting asset tarball:                                                                   puller.py:1014
         /tmp/refgenie_demo_2hzk45es/data/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP/fasta/fasta__default.tgz

INFO     HTTP Request: GET http://localhost:8000/v4/genomes/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP        _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Initializing genome from FASTA file:                                                         genome.py:417
         /tmp/refgenie_demo_2hzk45es/data/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP/fasta/03aa63e8accae38cf4d6              
         bcbb7865ef85f95a54020c4f4d48921e67ce382f188c/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP.fa

INFO     Added: 'rCRSd/fasta:default'                                                                content.py:381

INFO     Set default asset: 'jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP/fasta:default'                            seek.py:612

INFO     Created alias directories: /tmp/refgenie_demo_2hzk45es/alias/rCRSd/fasta/default           symlinks.py:223

Asset(name='default', description=None, size=34113, colocate=None, updated_at=datetime.datetime(2026, 8, 21, 1, 55, 43, 893614), path='data/jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP/fasta/03aa63e8accae38cf4d6bcbb7865ef85f95a54020c4f4d48921e67ce382f188c', digest='03aa63e8accae38cf4d6bcbb7865ef85f95a54020c4f4d48921e67ce382f188c', recipe_id=None, serving_modes_override=None, asset_group_id=1, created_at=datetime.datetime(2026, 8, 21, 1, 55, 43, 893622)) As you can see above, the genome has been initialized and `fasta` asset was pulled. Let's inspect the initialized genome. ```python print(refgenie.genome.table()) ```

                                       Genomes                                       
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Digest                           ┃ Aliases ┃ Source ┃ Species ┃ Description       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │ rCRSd   │ local  │         │ rCRSd test genome │
└──────────────────────────────────┴─────────┴────────┴─────────┴───────────────────┘

Now, that a `fasta` asset has been built for the `dm6` genome, let's add some custom asset classes and recipes to build an asset based on that. ### Add `bowtie2_index` asset class and recipe By supplying a URL (`str` object) rather than a local path (`pathlib.Path` object), refgenie will grab the remote file and register it as if it was a local file. ```python refgenie.asset_class.add( "https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/bowtie2_index_asset_class.yaml" ) refgenie.recipe.add( "https://github.com/refgenie/recipes/raw/refgenie1/recipes/bowtie2_index_asset_recipe.yaml" ) ```

INFO     Reading YAML from URL:                                                                            io.py:31
         https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/bowtie2_index_asset_class.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://github.com/refgenie/recipes/raw/refgenie1/asset_classes/bowtie2_index_asset_class.                
         yaml "HTTP/1.1 302 Found"

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://raw.githubusercontent.com/refgenie/recipes/refgenie1/asset_classes/bowtie2_index_a                
         sset_class.yaml "HTTP/1.1 200 OK"

INFO     Registered 'bowtie2_index' asset class                                                   asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://github.com/refgenie/recipes/raw/refgenie1/recipes/bowtie2_index_asset_recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://github.com/refgenie/recipes/raw/refgenie1/recipes/bowtie2_index_asset_recipe.yaml                 
         "HTTP/1.1 302 Found"

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://raw.githubusercontent.com/refgenie/recipes/refgenie1/recipes/bowtie2_index_asset_r                
         ecipe.yaml "HTTP/1.1 200 OK"

INFO     Registered 'bowtie2_index' recipe                                                            recipe.py:140

Recipe(id=2, name='bowtie2_index', version='0.0.1', description='Genome index for bowtie2, produced with bowtie2-build', output_asset_class_id=2, command_templates=['bowtie2-build --threads {{values.params["threads"]}} {{values.genome_folder}}/{{values.assets["fasta"].seek_keys_dict["fasta"]}} {{values.output_folder}}/{{values.genome_digest}}'], input_params={'threads': {'description': 'Number of threads to use', 'default': 1}}, input_files=None, input_assets={'fasta': {'asset_class': 'fasta', 'description': 'fasta asset for genome', 'default': 'fasta'}}, docker_image='docker.io/databio/refgenie', custom_seek_keys={'version': "bowtie2-build --version | awk 'NR==1{print $3}'"}, default_asset='{{values.custom_seek_keys.version}}', inherent=None, updated_at=datetime.datetime(2026, 8, 21, 1, 55, 44, 426089, tzinfo=datetime.timezone.utc), created_at=datetime.datetime(2026, 8, 21, 1, 55, 44, 426090, tzinfo=datetime.timezone.utc)) Let's verify that it worked by listing the available asset classes and recipes: ```python from rich import print print(refgenie.recipe.table()) print(refgenie.asset_class.table()) ```

                                                      Recipes                                                      
┏━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃               ┃         ┃ Output asset     ┃ Input asset     ┃             ┃                  ┃                 ┃
┃ Name          ┃ Version ┃ class            ┃ classes         ┃ Input files ┃ Input params     ┃ Docker image    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ fasta         │ 0.1.0   │ fasta            │ None            │ None        │ None             │                 │
├───────────────┼─────────┼──────────────────┼─────────────────┼─────────────┼──────────────────┼─────────────────┤
│ bowtie2_index │ 0.0.1   │ bowtie2_index    │ • fasta (fasta  │ None        │ • threads        │ docker.io/data… │
│               │         │                  │ asset for       │             │ (Number of       │                 │
│               │         │                  │ genome)         │             │ threads to use)  │                 │
│               │         │                  │ default=fasta   │             │ default=1        │                 │
└───────────────┴─────────┴──────────────────┴─────────────────┴─────────────┴──────────────────┴─────────────────┘

                                                   Asset Classes                                                   
┏━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name          ┃ Version ┃ Seek keys               ┃ Description                                                 ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ fasta         │ 0.1.0   │ fasta, fai, chrom_sizes │ Sequences in the FASTA format, indexed FASTA and chromosome │
│               │         │                         │ sizes file                                                  │
├───────────────┼─────────┼─────────────────────────┼─────────────────────────────────────────────────────────────┤
│ bowtie2_index │ 0.0.1   │ bowtie2_index           │ Genome index for bowtie2, produced with bowtie2-build       │
└───────────────┴─────────┴─────────────────────────┴─────────────────────────────────────────────────────────────┘

## Build a `fasta` asset ```python from refgenie import BuildParams from pathlib import Path # Step 1: Initialize genome from FASTA refgenie.genome.initialize_genome( fasta_file_path=REFGENIE_CODE_PATH.parent / "tests/data/t7.fa", alias_names=["t7"], description="Genome of T7 phage", ) # Step 2: Build fasta asset (exports from RefgetStore, no input file needed) refgenie.build.run( recipe_name="fasta", genome_alias="t7", asset_group_name="fasta", ) ```

INFO     Initializing genome from FASTA file:                                                         genome.py:417
         /home/nsheff/Dropbox/workspaces/refgenie/repos/refgenie1/tests/data/t7.fa

INFO     Added genome: kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB                                                genome.py:90

INFO     Added alias: t7                                                                               alias.py:305

INFO     Building 't7/fasta' using recipe 'fasta (v0.1.0)'                                           builder.py:513

INFO     Could not locate config file.                                                                yacman.py:674

INFO     No output schema supplied. Running in schema-optional mode (validate_results=False).      pipestat.py:1008
         Results will not be validated against a schema.

INFO     Initializing results file                                                               filebackend.py:785
         '/tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default/stats.yaml'

INFO     File does not exist, but create_file is true. Creating...                              file_locking.py:313

### Pipeline run code and environment: * Command: `/home/nsheff/.local/lib/python3.12/site-packages/ipykernel_launcher.py -f /tmp/tmp30pcozza.json --HistoryManager.hist_file=:memory:` * Compute host: `zither` * Working dir: `/home/nsheff/Dropbox/workspaces/refgenie/repos/refgenie1/docs` * Outfolder: `/tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default/` * Log file: `/tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default/refgenie_t7_fasta_default_log.md` * Start time: (08-20 21:55:44) elapsed: 0.0 _TIME_ ### Version log: * Python version: `3.12.3` * Pypiper dir: `/home/nsheff/.local/lib/python3.12/site-packages/pypiper` * Pypiper version: `0.15.1` * Pipestat version: `0.13.1` * Pipeline dir: `/home/nsheff/.local/lib/python3.12/site-packages` * Pipeline version: ### Arguments passed to pipeline: ### Initialized Pipestat Object: * PipestatManager (refgenie_t7_fasta_default) * Backend: File * - results: /tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default/stats.yaml * - status: /tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default * Multiple Pipelines Allowed: False * Pipeline name: refgenie_t7_fasta_default * Pipeline type: sample * Status Schema key: None * Results formatter: default_formatter * Results schema source: None * Status schema source: None * Records count: 0 * Sample name: DEFAULT_SAMPLE_NAME ---------------------------------------- Target to produce: `/tmp/refgenie_demo_2hzk45es/builds/t7/fasta/default/t7_fasta__default.flag` > `refgenie-build-fasta /tmp/refgenie_demo_2hzk45es/.refget_store kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta/default` (1045421)



    Loading collection metadata kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB...


    Loading sequence k0lfTnVEgjj6zguTU2MWdfYLenQFT6zW...

Command completed. Elapsed time: 0:00:00. Running peak memory: 0.015GB. PID: 1045421; Command: refgenie-build-fasta; Return code: 0; Memory used: 0.015GB

INFO     Asset 't7/fasta:default' build succeeded                                                    builder.py:666

### Pipeline completed. Epilogue * Elapsed time (this run): 0:00:00 * Total elapsed time (all runs): 0:00:00 * Peak memory (this run): 0.0154 GB * Pipeline completed time: 2026-08-20 21:55:44

INFO     Added: 't7/fasta:default'                                                                   content.py:381

INFO     Set default asset: 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta:default'                            seek.py:612

INFO     Added asset: 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta:default'                               builder.py:705

INFO     Created alias directories: /tmp/refgenie_demo_2hzk45es/alias/t7/fasta/default              symlinks.py:223

Asset(name='default', description='DNA sequences in the FASTA format, exported from RefgetStore. Includes FASTA index (.fai) and chromosome sizes file. Requires genome to be initialized in RefgetStore first (via refgenie genome init).', size=40486, colocate=None, updated_at=datetime.datetime(2026, 8, 21, 1, 55, 44, 814061), path='data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta/949c705a1c1668f0ea3d7da883d1a65e902b945b67de576407349ff72a8b1bf1', digest='949c705a1c1668f0ea3d7da883d1a65e902b945b67de576407349ff72a8b1bf1', recipe_id=1, serving_modes_override=None, asset_group_id=2, created_at=datetime.datetime(2026, 8, 21, 1, 55, 44, 814074)) ### Build a `bowtie2_index` asset The `bowtie2_index` asset class and recipe have been added successfully. Let's build the `bowtie2_index` asset for the `dm6` genome. ```python from refgenie.models import BuildParams refgenie.build.run( recipe_name="bowtie2_index", genome_alias="t7", asset_group_name="bowtie2_index", params=BuildParams(params={"threads": 8}), stage=True, # stage the asset right after building ) ```

INFO     Building 't7/bowtie2_index' using recipe 'bowtie2_index (v0.0.1)'                           builder.py:513

[2026-08-21T01:55:44Z WARN bulker::shimlink] 'bowtie2-build': `docker_command` is deprecated; use the `entrypoint` manifest field instead

INFO     Could not locate config file.                                                                yacman.py:674

INFO     No output schema supplied. Running in schema-optional mode (validate_results=False).      pipestat.py:1008
         Results will not be validated against a schema.

INFO     Initializing results file                                                               filebackend.py:785
         '/tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5/stats.yaml'

INFO     File does not exist, but create_file is true. Creating...                              file_locking.py:313

### Pipeline run code and environment: * Command: `/home/nsheff/.local/lib/python3.12/site-packages/ipykernel_launcher.py -f /tmp/tmp30pcozza.json --HistoryManager.hist_file=:memory:` * Compute host: `zither` * Working dir: `/home/nsheff/Dropbox/workspaces/refgenie/repos/refgenie1/docs` * Outfolder: `/tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5/` * Log file: `/tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5/refgenie_t7_bowtie2_index_2.3.5_log.md` * Start time: (08-20 21:55:45) elapsed: 0.0 _TIME_ ### Version log: * Python version: `3.12.3` * Pypiper dir: `/home/nsheff/.local/lib/python3.12/site-packages/pypiper` * Pypiper version: `0.15.1` * Pipestat version: `0.13.1` * Pipeline dir: `/home/nsheff/.local/lib/python3.12/site-packages` * Pipeline version: ### Arguments passed to pipeline: ### Initialized Pipestat Object: * PipestatManager (refgenie_t7_bowtie2_index_2.3.5) * Backend: File * - results: /tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5/stats.yaml * - status: /tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5 * Multiple Pipelines Allowed: False * Pipeline name: refgenie_t7_bowtie2_index_2.3.5 * Pipeline type: sample * Status Schema key: None * Results formatter: default_formatter * Results schema source: None * Status schema source: None * Records count: 0 * Sample name: DEFAULT_SAMPLE_NAME ---------------------------------------- Target to produce: `/tmp/refgenie_demo_2hzk45es/builds/t7/bowtie2_index/2.3.5/t7_bowtie2_index__2.3.5.flag` > `bowtie2-build --threads 8 /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta/949c705a1c1668f0ea3d7da883d1a65e902b945b67de576407349ff72a8b1bf1/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.fa /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB` (1045591)



    [2026-08-21T01:55:45Z WARN  bulker::shimlink] 'bowtie2-build': `docker_command` is deprecated; use the `entrypoint` manifest field instead


    Settings:


    Building a SMALL index
      Output files: "/tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.*.bt2"


      Line rate: 6 (line is 64 bytes)


      Lines per side: 1 (side is 64 bytes)


      Offset rate: 4 (one in 16)


      FTable chars: 10


      Strings: unpacked


      Max bucket size: default


      Max bucket size, sqrt multiplier: default


      Max bucket size, len divisor: 32


      Difference-cover sample period: 1024


      Endianness: little


      Actual local endianness: little


      Sanity checking: disabled


      Assertions: disabled


      Random seed: 0


      Sizeofs: void*:8, int:4, long:8, size_t:8


    Input files DNA, FASTA:


      /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/fasta/949c705a1c1668f0ea3d7da883d1a65e902b945b67de576407349ff72a8b1bf1/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.fa


    Reading reference sizes


      Time reading reference sizes: 00:00:00


    Calculating joined length


    Writing header


    Reserving space for joined string


    Joining reference sequences


      Time to join reference sequences: 00:00:00


    bmax according to bmaxDivN setting: 1248


    Using parameters --bmax 936 --dcv 1024


      Doing ahead-of-time memory usage test


      Passed!  Constructing with these parameters: --bmax 936 --dcv 1024


    Constructing suffix-array element generator


    Building DifferenceCoverSample


      Building sPrime


      Building sPrimeOrder


      V-Sorting samples


      V-Sorting samples time: 00:00:00


      Allocating rank array


      Ranking v-sort output


      Ranking v-sort output time: 00:00:00


      Invoking Larsson-Sadakane on ranks


      Invoking Larsson-Sadakane on ranks time: 00:00:00


      Sanity-checking and returning


    Building samples


    Reserving space for 86 sample suffixes


    Generating random suffixes


    QSorting 86 sample offsets, eliminating duplicates


    QSorting sample offsets, eliminating duplicates time: 00:00:00


    Multikey QSorting 86 samples


      (Using difference cover)


      Multikey QSorting samples time: 00:00:00


    Calculating bucket sizes


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 13, merged 35; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 7, merged 7; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 2, merged 4; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Avg bucket size: 653.721 (target: 935)


    Converting suffix-array elements to index image


    Allocating ftab, absorbFtab


    Entering Ebwt loop


    Getting block 1 of 61


      Reserving size (936) for bucket 1


    Getting block 2 of 61


    Getting block 3 of 61


    Getting block 4 of 61


    Getting block 5 of 61


    Getting block 6 of 61


    Getting block 7 of 61


    Getting block 8 of 61


      Calculating Z arrays for bucket 1


      Reserving size (936) for bucket 2


      Reserving size (936) for bucket 3


      Reserving size (936) for bucket 4


      Reserving size (936) for bucket 5


      Reserving size (936) for bucket 6


      Reserving size (936) for bucket 7


      Reserving size (936) for bucket 8


      Entering block accumulator loop for bucket 1:


      Calculating Z arrays for bucket 2


      Calculating Z arrays for bucket 3


      Calculating Z arrays for bucket 4


      Calculating Z arrays for bucket 5


      Calculating Z arrays for bucket 6


      Calculating Z arrays for bucket 7


      Calculating Z arrays for bucket 8


      Entering block accumulator loop for bucket 2:


      Entering block accumulator loop for bucket 3:


      Entering block accumulator loop for bucket 5:


      Entering block accumulator loop for bucket 4:


      Entering block accumulator loop for bucket 8:


      Entering block accumulator loop for bucket 7:


      Entering block accumulator loop for bucket 6:


      bucket 1: 10%


      bucket 5: 10%


      bucket 1: 20%


      bucket 2: 10%


      bucket 4: 10%


      bucket 3: 10%


      bucket 8: 10%


      bucket 7: 10%


      bucket 6: 10%


      bucket 5: 20%


      bucket 1: 30%


      bucket 2: 20%


      bucket 4: 20%


      bucket 3: 20%


      bucket 7: 20%


      bucket 8: 20%


      bucket 5: 30%


      bucket 1: 40%


      bucket 6: 20%


      bucket 2: 30%


      bucket 4: 30%


      bucket 7: 30%


      bucket 3: 30%


      bucket 8: 30%


      bucket 5: 40%


      bucket 1: 50%


      bucket 6: 30%


      bucket 2: 40%


      bucket 4: 40%


      bucket 7: 40%


      bucket 3: 40%


      bucket 8: 40%


      bucket 1: 60%


      bucket 5: 50%


      bucket 6: 40%


      bucket 2: 50%


      bucket 4: 50%


      bucket 7: 50%


      bucket 1: 70%


      bucket 3: 50%


      bucket 5: 60%


      bucket 8: 50%


      bucket 6: 50%


      bucket 1: 80%


      bucket 2: 60%


      bucket 7: 60%


      bucket 4: 60%


      bucket 3: 60%


      bucket 5: 70%


      bucket 8: 60%


      bucket 1: 90%


      bucket 6: 60%


      bucket 2: 70%


      bucket 7: 70%


      bucket 4: 70%


      bucket 3: 70%


      bucket 5: 80%


      bucket 1: 100%


      bucket 8: 70%


      bucket 6: 70%


      bucket 7: 80%


      bucket 2: 80%


      bucket 4: 80%


      Sorting block of length 878 for bucket 1


      (Using difference cover)


      bucket 3: 80%


      bucket 5: 90%


      bucket 8: 80%


      bucket 6: 80%


      bucket 7: 90%


      bucket 2: 90%


      bucket 4: 90%


      bucket 3: 90%


      bucket 5: 100%


      bucket 8: 90%


      bucket 7: 100%


      bucket 6: 90%


      bucket 2: 100%


      Sorting block of length 395 for bucket 5


      (Using difference cover)


      bucket 4: 100%


      Sorting block time: 00:00:00


      Sorting block time: 00:00:00


      Sorting block of length 709 for bucket 7


      (Using difference cover)


      bucket 3: 100%


      Sorting block of length 639 for bucket 2


      (Using difference cover)


      bucket 8: 100%


      Sorting block of length 552 for bucket 4


      (Using difference cover)


    Returning block of 879 for bucket 1


      bucket 6: 100%


    Returning block of 396 for bucket 5


      Sorting block of length 516 for bucket 3


      (Using difference cover)


      Sorting block of length 598 for bucket 8


      (Using difference cover)


      Sorting block of length 596 for bucket 6


      (Using difference cover)


    Getting block 9 of 61


      Sorting block time: 00:00:00


    Getting block 10 of 61


      Reserving size (936) for bucket 9


    Returning block of 710 for bucket 7


      Reserving size (936) for bucket 10


      Calculating Z arrays for bucket 9


      Calculating Z arrays for bucket 10


      Entering block accumulator loop for bucket 9:


      Entering block accumulator loop for bucket 10:


    Getting block 11 of 61


      bucket 10: 10%


      bucket 9: 10%


      Reserving size (936) for bucket 11


      Calculating Z arrays for bucket 11


      Sorting block time: 00:00:00


    Returning block of 599 for bucket 8


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 11:


    Returning block of 597 for bucket 6


      bucket 10: 20%


      bucket 9: 20%


    Getting block 12 of 61


      Reserving size (936) for bucket 12


      Calculating Z arrays for bucket 12


      Entering block accumulator loop for bucket 12:


      bucket 11: 10%


      Sorting block time: 00:00:00


      bucket 9: 30%


      bucket 10: 30%


      Sorting block time: 00:00:00


    Returning block of 640 for bucket 2


      Sorting block time: 00:00:00


    Getting block 13 of 61


      bucket 11: 20%


      bucket 12: 10%


      bucket 9: 40%


    Returning block of 553 for bucket 4


    Returning block of 517 for bucket 3


      Reserving size (936) for bucket 13


      bucket 10: 40%


      bucket 11: 30%


      Calculating Z arrays for bucket 13


      bucket 12: 20%


      bucket 9: 50%


    Getting block 14 of 61


      Entering block accumulator loop for bucket 13:


      bucket 10: 50%


      bucket 11: 40%


      bucket 9: 60%


      bucket 12: 30%


      Reserving size (936) for bucket 14


    Getting block 15 of 61


    Getting block 16 of 61


      bucket 10: 60%


      bucket 9: 70%


      bucket 11: 50%


      bucket 12: 40%


      Calculating Z arrays for bucket 14


      Reserving size (936) for bucket 15


      Reserving size (936) for bucket 16


      bucket 13: 10%


      bucket 9: 80%


      bucket 10: 70%


      bucket 11: 60%


      bucket 12: 50%


      Entering block accumulator loop for bucket 14:


      Calculating Z arrays for bucket 15


      Calculating Z arrays for bucket 16


      bucket 9: 90%


      bucket 13: 20%


      bucket 10: 80%


      bucket 11: 70%


      bucket 12: 60%


      Entering block accumulator loop for bucket 15:


      Entering block accumulator loop for bucket 16:


      bucket 9: 100%


      bucket 12: 70%


      bucket 13: 30%


      bucket 11: 80%


      bucket 10: 90%


      Sorting block of length 581 for bucket 9


      (Using difference cover)


      bucket 14: 10%


      Sorting block time: 00:00:00


      bucket 12: 80%


      bucket 11: 90%


      bucket 13: 40%


      bucket 10: 100%


      bucket 16: 10%


      bucket 15: 10%


    Returning block of 582 for bucket 9


      bucket 14: 20%


      Sorting block of length 703 for bucket 10


      (Using difference cover)


      bucket 11: 100%


      bucket 12: 90%


      bucket 13: 50%


      bucket 16: 20%


      bucket 15: 20%


      Sorting block time: 00:00:00


      bucket 14: 30%


      Sorting block of length 457 for bucket 11


      (Using difference cover)


      bucket 12: 100%


      bucket 13: 60%


      Sorting block time: 00:00:00


    Getting block 17 of 61


    Returning block of 704 for bucket 10


      bucket 16: 30%


      bucket 15: 30%


      bucket 14: 40%


      Sorting block of length 686 for bucket 12


      (Using difference cover)


    Returning block of 458 for bucket 11


      Reserving size (936) for bucket 17


      bucket 13: 70%


      bucket 16: 40%


      bucket 15: 40%


      Sorting block time: 00:00:00


      bucket 14: 50%


      Calculating Z arrays for bucket 17


    Getting block 18 of 61


      bucket 13: 80%


      bucket 16: 50%


      bucket 15: 50%


    Returning block of 687 for bucket 12


      Reserving size (936) for bucket 18


    Getting block 19 of 61


      Entering block accumulator loop for bucket 17:


      bucket 14: 60%


      Calculating Z arrays for bucket 18


      Reserving size (936) for bucket 19


      bucket 16: 60%


      bucket 13: 90%


      bucket 14: 70%


      bucket 15: 60%


      Entering block accumulator loop for bucket 18:


      Calculating Z arrays for bucket 19


    Getting block 20 of 61


      bucket 17: 10%


      Entering block accumulator loop for bucket 19:


      bucket 16: 70%


      bucket 13: 100%


      bucket 14: 80%


      bucket 15: 70%


      Reserving size (936) for bucket 20


      Sorting block of length 595 for bucket 13


      (Using difference cover)


      bucket 17: 20%


      bucket 16: 80%


      bucket 18: 10%


      bucket 14: 90%


      bucket 15: 80%


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 20


      bucket 19: 10%


      bucket 17: 30%


      bucket 16: 90%


    Returning block of 596 for bucket 13


      bucket 14: 100%


      bucket 18: 20%


      bucket 15: 90%


      Entering block accumulator loop for bucket 20:


      Sorting block of length 402 for bucket 14


      (Using difference cover)


      bucket 19: 20%


      bucket 17: 40%


      bucket 16: 100%


      Sorting block time: 00:00:00


      Sorting block of length 768 for bucket 16


      (Using difference cover)


      bucket 18: 30%


      bucket 15: 100%


      bucket 17: 50%


      bucket 19: 30%


    Returning block of 403 for bucket 14


    Getting block 21 of 61


      Sorting block time: 00:00:00


      bucket 20: 10%


      Sorting block of length 819 for bucket 15


      (Using difference cover)


      bucket 18: 40%


      bucket 17: 60%


      bucket 19: 40%


      Reserving size (936) for bucket 21


    Returning block of 769 for bucket 16


      bucket 20: 20%


      Sorting block time: 00:00:00


      bucket 18: 50%


      Calculating Z arrays for bucket 21


      bucket 17: 70%


      bucket 19: 50%


    Getting block 22 of 61


    Returning block of 820 for bucket 15


      Entering block accumulator loop for bucket 21:


      bucket 20: 30%


      bucket 17: 80%


      bucket 18: 60%


      bucket 19: 60%


      Reserving size (936) for bucket 22


    Getting block 23 of 61


      bucket 20: 40%


      bucket 17: 90%


      Calculating Z arrays for bucket 22


    Getting block 24 of 61


      Reserving size (936) for bucket 23


      bucket 19: 70%


      Entering block accumulator loop for bucket 22:


      bucket 21: 10%


      bucket 20: 50%


      bucket 17: 100%


      Reserving size (936) for bucket 24


      Calculating Z arrays for bucket 23


      bucket 19: 80%


      Sorting block of length 736 for bucket 17


      (Using difference cover)


      bucket 21: 20%


      Calculating Z arrays for bucket 24


      bucket 20: 60%


      bucket 18: 70%


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 23:


      bucket 22: 10%


      bucket 19: 90%


      Entering block accumulator loop for bucket 24:


      bucket 21: 30%


      bucket 20: 70%


    Returning block of 737 for bucket 17


      bucket 22: 20%


      bucket 19: 100%


      bucket 21: 40%


      bucket 20: 80%


      bucket 18: 80%


      Sorting block of length 902 for bucket 19


      (Using difference cover)


      bucket 24: 10%


      bucket 22: 30%


      bucket 21: 50%


      bucket 20: 90%


      bucket 23: 10%


      Sorting block time: 00:00:00


      bucket 18: 90%


    Getting block 25 of 61


      bucket 24: 20%


      bucket 22: 40%


      bucket 21: 60%


      bucket 20: 100%


    Returning block of 903 for bucket 19


      bucket 23: 20%


      Reserving size (936) for bucket 25


      bucket 18: 100%


      bucket 24: 30%


      bucket 22: 50%


      Sorting block of length 578 for bucket 20


      (Using difference cover)


      bucket 21: 70%


      Calculating Z arrays for bucket 25


      bucket 23: 30%


      Sorting block time: 00:00:00


      Sorting block of length 693 for bucket 18


      (Using difference cover)


      bucket 24: 40%


      bucket 22: 60%


    Getting block 26 of 61


      Entering block accumulator loop for bucket 25:


      bucket 21: 80%


      Sorting block time: 00:00:00


    Returning block of 579 for bucket 20


      bucket 23: 40%


      bucket 24: 50%


      Reserving size (936) for bucket 26


      bucket 22: 70%


    Returning block of 694 for bucket 18


      bucket 21: 90%


      Calculating Z arrays for bucket 26


      bucket 24: 60%


      bucket 25: 10%


      bucket 22: 80%


    Getting block 27 of 61


      bucket 23: 50%


      Entering block accumulator loop for bucket 26:


      bucket 21: 100%


      bucket 24: 70%


      Reserving size (936) for bucket 27


      bucket 25: 20%


      bucket 22: 90%


      Sorting block of length 599 for bucket 21


      (Using difference cover)


      bucket 23: 60%


      Calculating Z arrays for bucket 27


      bucket 24: 80%


    Getting block 28 of 61


      Sorting block time: 00:00:00


      bucket 25: 30%


      bucket 22: 100%


      bucket 26: 10%


      Entering block accumulator loop for bucket 27:


      bucket 23: 70%


      Reserving size (936) for bucket 28


    Returning block of 600 for bucket 21


      bucket 24: 90%


      Sorting block of length 895 for bucket 22


      (Using difference cover)


      bucket 25: 40%


      bucket 26: 20%


      Calculating Z arrays for bucket 28


      bucket 23: 80%


      bucket 25: 50%


      bucket 24: 100%


      bucket 26: 30%


      Entering block accumulator loop for bucket 28:


      bucket 27: 10%


    Getting block 29 of 61


      bucket 23: 90%


      Sorting block of length 583 for bucket 24


      (Using difference cover)


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 29


    Returning block of 896 for bucket 22


      bucket 25: 60%


      bucket 26: 40%


      bucket 27: 20%


      bucket 23: 100%


      bucket 28: 10%


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 29


      Sorting block of length 917 for bucket 23


      (Using difference cover)


    Returning block of 584 for bucket 24


      bucket 25: 70%


      bucket 27: 30%


      bucket 26: 50%


      bucket 28: 20%


      Entering block accumulator loop for bucket 29:


    Getting block 30 of 61


      bucket 25: 80%


      bucket 26: 60%


      bucket 27: 40%


      bucket 28: 30%


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 30


    Getting block 31 of 61


      bucket 25: 90%


    Returning block of 918 for bucket 23


      bucket 26: 70%


      bucket 27: 50%


      bucket 28: 40%


      Calculating Z arrays for bucket 30


      Reserving size (936) for bucket 31


      bucket 29: 10%


      bucket 25: 100%


      Calculating Z arrays for bucket 31


      Entering block accumulator loop for bucket 30:


      bucket 26: 80%


      bucket 27: 60%


      bucket 28: 50%


      Sorting block of length 490 for bucket 25


      (Using difference cover)


    Getting block 32 of 61


      Entering block accumulator loop for bucket 31:


      bucket 29: 20%


      Reserving size (936) for bucket 32


      bucket 26: 90%


      bucket 27: 70%


      Sorting block time: 00:00:00


      bucket 28: 60%


      bucket 30: 10%


      Calculating Z arrays for bucket 32


    Returning block of 491 for bucket 25


      bucket 29: 30%


      bucket 26: 100%


      bucket 31: 10%


      bucket 27: 80%


      Entering block accumulator loop for bucket 32:


      bucket 28: 70%


      bucket 30: 20%


      Sorting block of length 625 for bucket 26


      (Using difference cover)


      bucket 29: 40%


    Getting block 33 of 61


      bucket 27: 90%


      bucket 31: 20%


      Sorting block time: 00:00:00


      bucket 28: 80%


      Reserving size (936) for bucket 33


      bucket 30: 30%


    Returning block of 626 for bucket 26


      bucket 32: 10%


      bucket 29: 50%


      Calculating Z arrays for bucket 33


      bucket 27: 100%


      bucket 31: 30%


      bucket 28: 90%


      bucket 30: 40%


      Sorting block of length 532 for bucket 27


      (Using difference cover)


    Getting block 34 of 61


      bucket 32: 20%


      Entering block accumulator loop for bucket 33:


      bucket 29: 60%


      bucket 31: 40%


      Reserving size (936) for bucket 34


      Sorting block time: 00:00:00


      bucket 28: 100%


      bucket 30: 50%


      Calculating Z arrays for bucket 34


    Returning block of 533 for bucket 27


      bucket 32: 30%


      bucket 29: 70%


      Sorting block of length 775 for bucket 28


      (Using difference cover)


      bucket 31: 50%


      bucket 33: 10%


      Entering block accumulator loop for bucket 34:


      bucket 30: 60%


      bucket 32: 40%


      Sorting block time: 00:00:00


      bucket 29: 80%


      bucket 31: 60%


    Getting block 35 of 61


      bucket 33: 20%


    Returning block of 776 for bucket 28


      bucket 30: 70%


      Reserving size (936) for bucket 35


      bucket 32: 50%


      bucket 31: 70%


      bucket 33: 30%


      bucket 29: 90%


      bucket 34: 10%


      Calculating Z arrays for bucket 35


      bucket 30: 80%


      bucket 32: 60%


      Entering block accumulator loop for bucket 35:


      bucket 31: 80%


      bucket 33: 40%


      bucket 29: 100%


    Getting block 36 of 61


      bucket 34: 20%


      bucket 30: 90%


      bucket 32: 70%


      Sorting block of length 397 for bucket 29


      (Using difference cover)


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 36


      bucket 31: 90%


      bucket 33: 50%


      bucket 34: 30%


      bucket 32: 80%


      bucket 35: 10%


      bucket 30: 100%


    Returning block of 398 for bucket 29


      Calculating Z arrays for bucket 36


      bucket 31: 100%


      bucket 33: 60%


      bucket 34: 40%


      Sorting block of length 837 for bucket 30


      (Using difference cover)


      bucket 32: 90%


      bucket 35: 20%


      Sorting block of length 852 for bucket 31


      (Using difference cover)


      Entering block accumulator loop for bucket 36:


      bucket 33: 70%


      bucket 34: 50%


    Getting block 37 of 61


      Sorting block time: 00:00:00


      bucket 32: 100%


      Sorting block time: 00:00:00


      bucket 35: 30%


      Reserving size (936) for bucket 37


    Returning block of 838 for bucket 30


      bucket 33: 80%


      Sorting block of length 542 for bucket 32


      (Using difference cover)


      bucket 34: 60%


    Returning block of 853 for bucket 31


      Calculating Z arrays for bucket 37


      Sorting block time: 00:00:00


      bucket 36: 10%


      bucket 35: 40%


      bucket 33: 90%


      bucket 34: 70%


      Entering block accumulator loop for bucket 37:


    Returning block of 543 for bucket 32


    Getting block 38 of 61


      bucket 36: 20%


      bucket 35: 50%


      bucket 33: 100%


    Getting block 39 of 61


      bucket 34: 80%


      Reserving size (936) for bucket 38


      Sorting block of length 571 for bucket 33


      (Using difference cover)


      bucket 37: 10%


      bucket 36: 30%


      bucket 35: 60%


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 39


      Calculating Z arrays for bucket 38


      bucket 34: 90%


    Getting block 40 of 61


      bucket 37: 20%


      bucket 36: 40%


      bucket 35: 70%


    Returning block of 572 for bucket 33


      Calculating Z arrays for bucket 39


      Entering block accumulator loop for bucket 38:


      Reserving size (936) for bucket 40


      bucket 34: 100%


      bucket 37: 30%


      bucket 36: 50%


      bucket 35: 80%


      Entering block accumulator loop for bucket 39:


      Calculating Z arrays for bucket 40


      Sorting block of length 581 for bucket 34


      (Using difference cover)


    Getting block 41 of 61


      bucket 37: 40%


      bucket 38: 10%


      bucket 36: 60%


      bucket 35: 90%


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 40:


      Reserving size (936) for bucket 41


      bucket 37: 50%


    Returning block of 582 for bucket 34


      bucket 36: 70%


      bucket 38: 20%


      Calculating Z arrays for bucket 41


      bucket 35: 100%


      bucket 39: 10%


      Sorting block of length 593 for bucket 35


      (Using difference cover)


      Entering block accumulator loop for bucket 41:


      bucket 37: 60%


      bucket 36: 80%


      bucket 38: 30%


      bucket 40: 10%


    Getting block 42 of 61


      Sorting block time: 00:00:00


      bucket 39: 20%


      bucket 37: 70%


      Reserving size (936) for bucket 42


    Returning block of 594 for bucket 35


      bucket 36: 90%


      bucket 40: 20%


      bucket 38: 40%


      bucket 41: 10%


      Calculating Z arrays for bucket 42


      bucket 39: 30%


      bucket 37: 80%


      Entering block accumulator loop for bucket 42:


      bucket 36: 100%


      bucket 40: 30%


      bucket 38: 50%


    Getting block 43 of 61


      bucket 41: 20%


      bucket 37: 90%


      bucket 39: 40%


      Sorting block of length 379 for bucket 36


      (Using difference cover)


      Reserving size (936) for bucket 43


      bucket 40: 40%


      bucket 42: 10%


      bucket 38: 60%


      bucket 41: 30%


      Sorting block time: 00:00:00


      bucket 37: 100%


      bucket 39: 50%


      Calculating Z arrays for bucket 43


    Returning block of 380 for bucket 36


      bucket 42: 20%


      bucket 38: 70%


      bucket 41: 40%


      Sorting block of length 600 for bucket 37


      (Using difference cover)


      Entering block accumulator loop for bucket 43:


      bucket 39: 60%


      bucket 42: 30%


      bucket 38: 80%


      bucket 39: 70%


      Sorting block time: 00:00:00


      bucket 41: 50%


    Getting block 44 of 61


    Returning block of 601 for bucket 37


      bucket 43: 10%


      bucket 42: 40%


      bucket 38: 90%


      bucket 39: 80%


      bucket 41: 60%


      Reserving size (936) for bucket 44


      bucket 43: 20%


      bucket 42: 50%


      bucket 38: 100%


      bucket 41: 70%


      bucket 39: 90%


      Calculating Z arrays for bucket 44


    Getting block 45 of 61


      Sorting block of length 697 for bucket 38


      (Using difference cover)


      bucket 43: 30%


      bucket 42: 60%


      bucket 41: 80%


      bucket 39: 100%


      Reserving size (936) for bucket 45


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 44:


      bucket 43: 40%


      Sorting block of length 889 for bucket 39


      (Using difference cover)


      bucket 42: 70%


      Calculating Z arrays for bucket 45


      bucket 41: 90%


    Returning block of 698 for bucket 38


      Entering block accumulator loop for bucket 45:


      bucket 43: 50%


      bucket 42: 80%


      Sorting block time: 00:00:00


      bucket 41: 100%


      bucket 44: 10%


    Returning block of 890 for bucket 39


      Sorting block of length 892 for bucket 41


      (Using difference cover)


    Getting block 46 of 61


      bucket 43: 60%


      bucket 42: 90%


      bucket 45: 10%


      Sorting block time: 00:00:00


      bucket 44: 20%


      Reserving size (936) for bucket 46


      bucket 43: 70%


      bucket 42: 100%


      bucket 45: 20%


    Returning block of 893 for bucket 41


      Calculating Z arrays for bucket 46


    Getting block 47 of 61


      Sorting block of length 652 for bucket 42


      (Using difference cover)


      bucket 44: 30%


      bucket 43: 80%


      Reserving size (936) for bucket 47


      bucket 45: 30%


      Entering block accumulator loop for bucket 46:


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 47


    Returning block of 653 for bucket 42


      bucket 44: 40%


    Getting block 48 of 61


      bucket 43: 90%


      bucket 45: 40%


      Entering block accumulator loop for bucket 47:


      bucket 46: 10%


      bucket 44: 50%


      Reserving size (936) for bucket 48


    Getting block 49 of 61


      bucket 43: 100%


      bucket 45: 50%


      bucket 46: 20%


      bucket 44: 60%


      Calculating Z arrays for bucket 48


      Reserving size (936) for bucket 49


      Sorting block of length 811 for bucket 43


      (Using difference cover)


      bucket 47: 10%


      bucket 45: 60%


      bucket 46: 30%


      bucket 44: 70%


      Calculating Z arrays for bucket 49


      Entering block accumulator loop for bucket 48:


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 49:


      bucket 45: 70%


      bucket 47: 20%


      bucket 46: 40%


      bucket 44: 80%


    Returning block of 812 for bucket 43


      bucket 45: 80%


      bucket 47: 30%


      bucket 46: 50%


      bucket 44: 90%


      bucket 48: 10%


      bucket 49: 10%


      bucket 47: 40%


      bucket 45: 90%


      bucket 46: 60%


      bucket 44: 100%


    Getting block 50 of 61


      bucket 40: 50%


      bucket 48: 20%


      Sorting block of length 463 for bucket 44


      (Using difference cover)


      bucket 49: 20%


      Reserving size (936) for bucket 50


      bucket 47: 50%


      bucket 45: 100%


      bucket 46: 70%


      bucket 40: 60%


      Sorting block time: 00:00:00


      bucket 48: 30%


      Calculating Z arrays for bucket 50


      Sorting block of length 749 for bucket 45


      (Using difference cover)


      bucket 49: 30%


      bucket 47: 60%


      bucket 46: 80%


    Returning block of 464 for bucket 44


      bucket 40: 70%


      Entering block accumulator loop for bucket 50:


      bucket 48: 40%


      bucket 49: 40%


      bucket 47: 70%


      Sorting block time: 00:00:00


      bucket 46: 90%


      bucket 40: 80%


    Returning block of 750 for bucket 45


      bucket 48: 50%


    Getting block 51 of 61


      bucket 49: 50%


      bucket 50: 10%


      bucket 47: 80%


      bucket 40: 90%


      bucket 48: 60%


      Reserving size (936) for bucket 51


      bucket 49: 60%


      bucket 40: 100%


      bucket 48: 70%


      bucket 50: 20%


      Calculating Z arrays for bucket 51


      bucket 47: 90%


      Sorting block of length 619 for bucket 40


      (Using difference cover)


      bucket 49: 70%


      bucket 48: 80%


      bucket 50: 30%


      Entering block accumulator loop for bucket 51:


      bucket 49: 80%


      bucket 48: 90%


      Sorting block time: 00:00:00


      bucket 47: 100%


      bucket 50: 40%


      bucket 49: 90%


    Returning block of 620 for bucket 40


      bucket 48: 100%


      Sorting block of length 928 for bucket 47


      (Using difference cover)


      bucket 51: 10%


      bucket 50: 50%


      bucket 46: 100%


      Sorting block of length 419 for bucket 48


      (Using difference cover)


      bucket 49: 100%


      Sorting block of length 368 for bucket 46


      (Using difference cover)


      bucket 50: 60%


      Sorting block time: 00:00:00


      Sorting block time: 00:00:00


      bucket 51: 20%


      Sorting block of length 893 for bucket 49


      (Using difference cover)


    Getting block 52 of 61


      Sorting block time: 00:00:00


    Getting block 53 of 61


      Sorting block time: 00:00:00


    Returning block of 929 for bucket 47


    Returning block of 420 for bucket 48


      bucket 50: 70%


      bucket 51: 30%


      Reserving size (936) for bucket 52


    Returning block of 369 for bucket 46


    Returning block of 894 for bucket 49


      Reserving size (936) for bucket 53


      bucket 50: 80%


      Calculating Z arrays for bucket 52


      Calculating Z arrays for bucket 53


      bucket 51: 40%


    Getting block 54 of 61


    Getting block 55 of 61


      Entering block accumulator loop for bucket 52:


    Getting block 56 of 61


      bucket 50: 90%


      Entering block accumulator loop for bucket 53:


      bucket 51: 50%


      Reserving size (936) for bucket 54


      Reserving size (936) for bucket 55


    Getting block 57 of 61


      Reserving size (936) for bucket 56


      bucket 50: 100%


      bucket 51: 60%


      Calculating Z arrays for bucket 54


      Calculating Z arrays for bucket 55


      Reserving size (936) for bucket 57


      Calculating Z arrays for bucket 56


      Sorting block of length 218 for bucket 50


      (Using difference cover)


      bucket 52: 10%


      bucket 53: 10%


      bucket 51: 70%


      Calculating Z arrays for bucket 57


      Entering block accumulator loop for bucket 54:


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 55:


      Entering block accumulator loop for bucket 56:


      Entering block accumulator loop for bucket 57:


      bucket 52: 20%


      bucket 51: 80%


      bucket 53: 20%


    Returning block of 219 for bucket 50


      bucket 52: 30%


      bucket 51: 90%


      bucket 53: 30%


      bucket 55: 10%


      bucket 54: 10%


      bucket 56: 10%


      bucket 57: 10%


      bucket 52: 40%


    Getting block 58 of 61


      bucket 51: 100%


      bucket 53: 40%


      bucket 55: 20%


      Reserving size (936) for bucket 58


      bucket 54: 20%


      Sorting block of length 889 for bucket 51


      (Using difference cover)


      bucket 56: 20%


      Calculating Z arrays for bucket 58


      bucket 57: 20%


      bucket 52: 50%


      bucket 55: 30%


      bucket 53: 50%


      Sorting block time: 00:00:00


      bucket 54: 30%


      Entering block accumulator loop for bucket 58:


      bucket 56: 30%


      bucket 57: 30%


      bucket 52: 60%


      bucket 55: 40%


    Returning block of 890 for bucket 51


      bucket 53: 60%


      bucket 54: 40%


      bucket 56: 40%


      bucket 57: 40%


      bucket 52: 70%


      bucket 55: 50%


      bucket 58: 10%


      bucket 53: 70%


      bucket 54: 50%


      bucket 56: 50%


      bucket 57: 50%


    Getting block 59 of 61


      bucket 52: 80%


      bucket 55: 60%


      bucket 53: 80%


      bucket 58: 20%


      Reserving size (936) for bucket 59


      bucket 57: 60%


      bucket 54: 60%


      bucket 56: 60%


      bucket 52: 90%


      bucket 53: 90%


      bucket 55: 70%


      Calculating Z arrays for bucket 59


      bucket 58: 30%


      bucket 57: 70%


      bucket 53: 100%


      bucket 54: 70%


      bucket 55: 80%


      bucket 56: 70%


      bucket 52: 100%


      Entering block accumulator loop for bucket 59:


      Sorting block of length 934 for bucket 53


      (Using difference cover)


      Sorting block of length 845 for bucket 52


      Sorting block time: 00:00:00


      bucket 58: 40%


      bucket 57: 80%


      bucket 54: 80%


      bucket 55: 90%


      bucket 56: 80%


    Returning block of 935 for bucket 53


      bucket 59: 10%


      bucket 58: 50%


      bucket 57: 90%


      bucket 54: 90%


      bucket 55: 100%


      bucket 56: 90%


      Sorting block of length 897 for bucket 55


      (Using difference cover)


      bucket 59: 20%


      bucket 58: 60%


      bucket 57: 100%


    Getting block 60 of 61


      bucket 54: 100%


      Sorting block of length 493 for bucket 57


      (Using difference cover)


      bucket 56: 100%


      bucket 59: 30%


      Reserving size (936) for bucket 60


      Sorting block of length 62 for bucket 54


      (Using difference cover)


      bucket 58: 70%


      Sorting block of length 606 for bucket 56


      Sorting block time: 00:00:00


      (Using difference cover)


      bucket 59: 40%


      Sorting block time: 00:00:00


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 60


    Returning block of 494 for bucket 57


      bucket 58: 80%


    Returning block of 898 for bucket 55


    Returning block of 63 for bucket 54


      bucket 59: 50%


      Entering block accumulator loop for bucket 60:


      bucket 58: 90%


      bucket 59: 60%


      Sorting block time: 00:00:00


    Getting block 61 of 61


    Returning block of 607 for bucket 56


      bucket 59: 70%


      bucket 58: 100%


      Reserving size (936) for bucket 61


      bucket 60: 10%


      Sorting block of length 924 for bucket 58


      (Using difference cover)


      Calculating Z arrays for bucket 61


      bucket 59: 80%


      Entering block accumulator loop for bucket 61:


      bucket 60: 20%


      Sorting block time: 00:00:00


      bucket 59: 90%


    Returning block of 925 for bucket 58


      bucket 61: 10%


      bucket 60: 30%


      bucket 61: 20%


      bucket 59: 100%


      bucket 61: 30%


      bucket 60: 40%


      Sorting block of length 638 for bucket 59


      (Using difference cover)


      bucket 61: 40%


      Sorting block time: 00:00:00


      bucket 60: 50%


      bucket 61: 50%


    Returning block of 639 for bucket 59


      bucket 61: 60%


      (Using difference cover)


      bucket 60: 60%


      bucket 61: 70%


      bucket 61: 80%


      bucket 60: 70%


      Sorting block time: 00:00:00


    Returning block of 846 for bucket 52


      bucket 61: 90%


      bucket 60: 80%


      bucket 61: 100%


      Sorting block of length 693 for bucket 61


      (Using difference cover)


      bucket 60: 90%


      bucket 60: 100%


      Sorting block of length 727 for bucket 60


      (Using difference cover)


      Sorting block time: 00:00:00


    Returning block of 694 for bucket 61


      Sorting block time: 00:00:00


    Returning block of 728 for bucket 60


    Exited Ebwt loop


    fchr[A]: 0


    fchr[C]: 10842


    fchr[G]: 19880


    fchr[T]: 30171


    fchr[$]: 39937


    Exiting Ebwt::buildToDisk()


    Returning from initFromVector


    Wrote 4207850 bytes to primary EBWT file: /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.1.bt2


    Wrote 9992 bytes to secondary EBWT file: /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.2.bt2


    Re-opening _in1 and _in2 as input streams


    Returning from Ebwt constructor


    Headers:


        len: 39937


        bwtLen: 39938


        sz: 9985


        bwtSz: 9985


        lineRate: 6


        offRate: 4


        offMask: 0xfffffff0


        ftabChars: 10


        eftabLen: 20


        eftabSz: 80


        ftabLen: 1048577


        ftabSz: 4194308


        offsLen: 2497


        offsSz: 9988


        lineSz: 64


        sideSz: 64


        sideBwtSz: 48


        sideBwtLen: 192


        numSides: 209


        numLines: 209


        ebwtTotLen: 13376


        ebwtTotSz: 13376


        color: 0


        reverse: 0


    Total time for call to driver() for forward index: 00:00:00


    Reading reference sizes


      Time reading reference sizes: 00:00:00


    Calculating joined length


    Writing header


    Reserving space for joined string


    Joining reference sequences


      Time to join reference sequences: 00:00:00


      Time to reverse reference sequence: 00:00:00


    bmax according to bmaxDivN setting: 1248


    Using parameters --bmax 936 --dcv 1024


      Doing ahead-of-time memory usage test


      Passed!  Constructing with these parameters: --bmax 936 --dcv 1024


    Constructing suffix-array element generator


    Building DifferenceCoverSample


      Building sPrime


      Building sPrimeOrder


      V-Sorting samples


      V-Sorting samples time: 00:00:00


      Allocating rank array


      Ranking v-sort output


      Ranking v-sort output time: 00:00:00


      Invoking Larsson-Sadakane on ranks


      Invoking Larsson-Sadakane on ranks time: 00:00:00


      Sanity-checking and returning


    Building samples


    Reserving space for 86 sample suffixes


    Generating random suffixes


    QSorting 86 sample offsets, eliminating duplicates


    QSorting sample offsets, eliminating duplicates time: 00:00:00


    Multikey QSorting 86 samples


      (Using difference cover)


      Multikey QSorting samples time: 00:00:00


    Calculating bucket sizes


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 13, merged 40; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 7, merged 6; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 3, merged 4; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 2, merged 3; iterating...


    Splitting and merging


      Splitting and merging time: 00:00:00


    Split 2, merged 1; iterating...


    Avg bucket size: 664.633 (target: 935)


    Converting suffix-array elements to index image


    Allocating ftab, absorbFtab


    Entering Ebwt loop


    Getting block 1 of 60


      Reserving size (936) for bucket 1


    Getting block 2 of 60


    Getting block 3 of 60


    Getting block 4 of 60


      Calculating Z arrays for bucket 1


    Getting block 5 of 60


      Reserving size (936) for bucket 2


    Getting block 6 of 60


      Reserving size (936) for bucket 3


      Reserving size (936) for bucket 4


    Getting block 7 of 60


      Entering block accumulator loop for bucket 1:


      Reserving size (936) for bucket 5


    Getting block 8 of 60


      Calculating Z arrays for bucket 2


      Reserving size (936) for bucket 6


      Calculating Z arrays for bucket 3


      Calculating Z arrays for bucket 4


      Reserving size (936) for bucket 7


      Calculating Z arrays for bucket 5


      Reserving size (936) for bucket 8


      Calculating Z arrays for bucket 6


      Entering block accumulator loop for bucket 3:


      Calculating Z arrays for bucket 7


      Entering block accumulator loop for bucket 2:


      Entering block accumulator loop for bucket 4:


      Calculating Z arrays for bucket 8


      Entering block accumulator loop for bucket 5:


      Entering block accumulator loop for bucket 6:


      bucket 1: 10%


      Entering block accumulator loop for bucket 7:


      Entering block accumulator loop for bucket 8:


      bucket 3: 10%


      bucket 4: 10%


      bucket 1: 20%


      bucket 5: 10%


      bucket 2: 10%


      bucket 6: 10%


      bucket 4: 20%


      bucket 3: 20%


      bucket 8: 10%


      bucket 1: 30%


      bucket 5: 20%


      bucket 7: 10%


      bucket 4: 30%


      bucket 2: 20%


      bucket 3: 30%


      bucket 1: 40%


      bucket 6: 20%


      bucket 8: 20%


      bucket 4: 40%


      bucket 5: 30%


      bucket 3: 40%


      bucket 1: 50%


      bucket 2: 30%


      bucket 6: 30%


      bucket 8: 30%


      bucket 4: 50%


      bucket 3: 50%


      bucket 7: 20%


      bucket 1: 60%


      bucket 5: 40%


      bucket 4: 60%


      bucket 2: 40%


      bucket 3: 60%


      bucket 6: 40%


      bucket 8: 40%


      bucket 1: 70%


      bucket 4: 70%


      bucket 3: 70%


      bucket 5: 50%


      bucket 2: 50%


      bucket 7: 30%


      bucket 6: 50%


      bucket 1: 80%


      bucket 8: 50%


      bucket 4: 80%


      bucket 3: 80%


      bucket 5: 60%


      bucket 1: 90%


      bucket 2: 60%


      bucket 6: 60%


      bucket 8: 60%


      bucket 4: 90%


      bucket 3: 90%


      bucket 7: 40%


      bucket 1: 100%


      Sorting block of length 854 for bucket 1


      (Using difference cover)


      bucket 5: 70%


      bucket 2: 70%


      bucket 4: 100%


      Sorting block of length 602 for bucket 4


      (Using difference cover)


      bucket 3: 100%


      bucket 8: 70%


      Sorting block of length 814 for bucket 3


      (Using difference cover)


      bucket 6: 70%


      bucket 5: 80%


      bucket 2: 80%


      bucket 7: 50%


      bucket 8: 80%


      bucket 6: 80%


      bucket 2: 90%


      bucket 5: 90%


      bucket 8: 90%


      bucket 2: 100%


      Sorting block of length 671 for bucket 2


      (Using difference cover)


      bucket 6: 90%


      Sorting block time: 00:00:00


      bucket 7: 60%


      bucket 5: 100%


    Returning block of 815 for bucket 3


      bucket 8: 100%


      Sorting block of length 754 for bucket 5


      (Using difference cover)


      Sorting block of length 902 for bucket 8


      (Using difference cover)


      Sorting block time: 00:00:00


    Returning block of 603 for bucket 4


      bucket 6: 100%


      Sorting block of length 214 for bucket 6


      (Using difference cover)


      bucket 7: 70%


    Getting block 9 of 60


      Reserving size (936) for bucket 9


      Calculating Z arrays for bucket 9


      Entering block accumulator loop for bucket 9:


      Sorting block time: 00:00:00


    Returning block of 855 for bucket 1


      Sorting block time: 00:00:00


    Returning block of 755 for bucket 5


    Getting block 10 of 60


      bucket 9: 10%


      Reserving size (936) for bucket 10


      Calculating Z arrays for bucket 10


      bucket 7: 80%


      Entering block accumulator loop for bucket 10:


      bucket 9: 20%


      Sorting block time: 00:00:00


    Returning block of 903 for bucket 8


      bucket 10: 10%


      Sorting block time: 00:00:00


      bucket 9: 30%


    Returning block of 672 for bucket 2


    Getting block 11 of 60


      Reserving size (936) for bucket 11


    Getting block 12 of 60


      Calculating Z arrays for bucket 11


      bucket 7: 90%


      Reserving size (936) for bucket 12


      bucket 10: 20%


      Sorting block time: 00:00:00


      bucket 9: 40%


      Entering block accumulator loop for bucket 11:


      Calculating Z arrays for bucket 12


    Getting block 13 of 60


    Returning block of 215 for bucket 6


      Reserving size (936) for bucket 13


      Entering block accumulator loop for bucket 12:


      Calculating Z arrays for bucket 13


    Getting block 14 of 60


      Reserving size (936) for bucket 14


      Calculating Z arrays for bucket 14


      Entering block accumulator loop for bucket 13:


      bucket 9: 50%


      Entering block accumulator loop for bucket 14:


      bucket 10: 30%


      bucket 7: 100%


      bucket 11: 10%


      bucket 12: 10%


      Sorting block of length 914 for bucket 7


      (Using difference cover)


      bucket 9: 60%


      bucket 13: 10%


    Getting block 15 of 60


      bucket 10: 40%


      Reserving size (936) for bucket 15


      bucket 14: 10%


      Calculating Z arrays for bucket 15


      bucket 11: 20%


      bucket 12: 20%


      bucket 9: 70%


      Entering block accumulator loop for bucket 15:


      bucket 13: 20%


      bucket 10: 50%


      bucket 14: 20%


      bucket 9: 80%


      bucket 11: 30%


      bucket 12: 30%


      bucket 13: 30%


      bucket 10: 60%


      bucket 15: 10%


      bucket 14: 30%


      bucket 9: 90%


      bucket 11: 40%


      bucket 12: 40%


      bucket 13: 40%


      bucket 10: 70%


      bucket 9: 100%


      Sorting block of length 908 for bucket 9


      (Using difference cover)


      bucket 14: 40%


      bucket 11: 50%


      bucket 12: 50%


      bucket 13: 50%


      bucket 10: 80%


      Sorting block time: 00:00:00


      bucket 15: 20%


      bucket 14: 50%


    Returning block of 915 for bucket 7


      bucket 12: 60%


      Sorting block time: 00:00:00


      bucket 10: 90%


      bucket 11: 60%


      bucket 13: 60%


      bucket 14: 60%


    Returning block of 909 for bucket 9


      bucket 15: 30%


      bucket 10: 100%


      Sorting block of length 806 for bucket 10


      (Using difference cover)


      bucket 12: 70%


      bucket 13: 70%


      bucket 11: 70%


      bucket 14: 70%


      bucket 15: 40%


    Getting block 16 of 60


      Reserving size (936) for bucket 16


      Calculating Z arrays for bucket 16


      bucket 12: 80%


      Entering block accumulator loop for bucket 16:


      bucket 13: 80%


      bucket 14: 80%


    Getting block 17 of 60


      bucket 11: 80%


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 17


    Returning block of 807 for bucket 10


      Calculating Z arrays for bucket 17


      bucket 15: 50%


      bucket 12: 90%


      bucket 16: 10%


      bucket 13: 90%


      Entering block accumulator loop for bucket 17:


      bucket 14: 90%


      bucket 11: 90%


    Getting block 18 of 60


      bucket 12: 100%


      bucket 16: 20%


      bucket 13: 100%


      Reserving size (936) for bucket 18


      Sorting block of length 684 for bucket 12


      (Using difference cover)


      bucket 14: 100%


      bucket 15: 60%


      Sorting block of length 290 for bucket 13


      (Using difference cover)


      Calculating Z arrays for bucket 18


      Sorting block of length 759 for bucket 14


      (Using difference cover)


      bucket 11: 100%


      bucket 17: 10%


      Sorting block of length 538 for bucket 11


      (Using difference cover)


      Entering block accumulator loop for bucket 18:


      bucket 16: 30%


      bucket 15: 70%


      bucket 16: 40%


      bucket 18: 10%


      bucket 17: 20%


      Sorting block time: 00:00:00


    Returning block of 685 for bucket 12


      Sorting block time: 00:00:00


    Returning block of 291 for bucket 13


      bucket 16: 50%


      bucket 18: 20%


      Sorting block time: 00:00:00


    Returning block of 539 for bucket 11


      bucket 15: 80%


      bucket 17: 30%


    Getting block 19 of 60


      Sorting block time: 00:00:00


    Getting block 20 of 60


      Reserving size (936) for bucket 19


    Returning block of 760 for bucket 14


      Reserving size (936) for bucket 20


      bucket 16: 60%


      Calculating Z arrays for bucket 19


      bucket 18: 30%


      Calculating Z arrays for bucket 20


    Getting block 21 of 60


      Entering block accumulator loop for bucket 19:


      Reserving size (936) for bucket 21


      Calculating Z arrays for bucket 21


      Entering block accumulator loop for bucket 20:


    Getting block 22 of 60


      Reserving size (936) for bucket 22


      Entering block accumulator loop for bucket 21:


      Calculating Z arrays for bucket 22


      Entering block accumulator loop for bucket 22:


      bucket 18: 40%


      bucket 16: 70%


      bucket 15: 90%


      bucket 17: 40%


      bucket 19: 10%


      bucket 20: 10%


      bucket 18: 50%


      bucket 21: 10%


      bucket 22: 10%


      bucket 16: 80%


      bucket 19: 20%


      bucket 18: 60%


      bucket 17: 50%


      bucket 15: 100%


      bucket 20: 20%


      Sorting block of length 715 for bucket 15


      (Using difference cover)


      bucket 16: 90%


      bucket 21: 20%


      bucket 22: 20%


      bucket 18: 70%


      bucket 19: 30%


      bucket 17: 60%


      bucket 18: 80%


      bucket 16: 100%


      bucket 20: 30%


      Sorting block of length 865 for bucket 16


      (Using difference cover)


      bucket 22: 30%


      bucket 21: 30%


      bucket 19: 40%


      bucket 18: 90%


      Sorting block time: 00:00:00


    Returning block of 716 for bucket 15


      bucket 20: 40%


      bucket 17: 70%


      Sorting block time: 00:00:00


    Returning block of 866 for bucket 16


      bucket 22: 40%


      bucket 21: 40%


      bucket 18: 100%


      Sorting block of length 604 for bucket 18


      (Using difference cover)


      bucket 19: 50%


      bucket 20: 50%


    Getting block 23 of 60


      bucket 22: 50%


      Reserving size (936) for bucket 23


      bucket 21: 50%


      Calculating Z arrays for bucket 23


      bucket 17: 80%


    Getting block 24 of 60


      bucket 19: 60%


      Reserving size (936) for bucket 24


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 23:


      bucket 20: 60%


      Calculating Z arrays for bucket 24


    Returning block of 605 for bucket 18


      Entering block accumulator loop for bucket 24:


      bucket 22: 60%


      bucket 21: 60%


    Getting block 25 of 60


      Reserving size (936) for bucket 25


      Calculating Z arrays for bucket 25


      Entering block accumulator loop for bucket 25:


      bucket 19: 70%


      bucket 24: 10%


      bucket 20: 70%


      bucket 17: 90%


      bucket 22: 70%


      bucket 21: 70%


      bucket 25: 10%


      bucket 23: 10%


      bucket 24: 20%


      bucket 19: 80%


      bucket 20: 80%


      bucket 22: 80%


      bucket 25: 20%


      bucket 21: 80%


      bucket 17: 100%


      Sorting block of length 817 for bucket 17


      (Using difference cover)


      bucket 24: 30%


      bucket 19: 90%


      bucket 20: 90%


      bucket 22: 90%


      bucket 23: 20%


      bucket 25: 30%


      bucket 21: 90%


      bucket 24: 40%


      bucket 20: 100%


      bucket 19: 100%


      bucket 22: 100%


      Sorting block of length 408 for bucket 20


      Sorting block time: 00:00:00


      (Using difference cover)


      bucket 24: 50%


      bucket 21: 100%


      bucket 25: 40%


      Sorting block of length 736 for bucket 19


      (Using difference cover)


      Sorting block of length 839 for bucket 22


      (Using difference cover)


    Returning block of 818 for bucket 17


      Sorting block of length 681 for bucket 21


      (Using difference cover)


      bucket 23: 30%


      bucket 24: 60%


      bucket 25: 50%


      Sorting block time: 00:00:00


    Returning block of 409 for bucket 20


    Getting block 26 of 60


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 26


    Returning block of 840 for bucket 22


      Calculating Z arrays for bucket 26


      Sorting block time: 00:00:00


      bucket 24: 70%


    Getting block 27 of 60


    Returning block of 682 for bucket 21


      Sorting block time: 00:00:00


      bucket 23: 40%


      bucket 25: 60%


      Reserving size (936) for bucket 27


      Entering block accumulator loop for bucket 26:


    Returning block of 737 for bucket 19


      bucket 24: 80%


      Calculating Z arrays for bucket 27


    Getting block 28 of 60


    Getting block 29 of 60


      Reserving size (936) for bucket 28


      Reserving size (936) for bucket 29


      Calculating Z arrays for bucket 28


      Entering block accumulator loop for bucket 27:


      Calculating Z arrays for bucket 29


      Entering block accumulator loop for bucket 28:


      Entering block accumulator loop for bucket 29:


      bucket 25: 70%


    Getting block 30 of 60


      Reserving size (936) for bucket 30


      Calculating Z arrays for bucket 30


      Entering block accumulator loop for bucket 30:


      bucket 24: 90%


      bucket 23: 50%


      bucket 25: 80%


      bucket 27: 10%


      bucket 26: 10%


      bucket 28: 10%


      bucket 29: 10%


      bucket 30: 10%


      bucket 25: 90%


      bucket 24: 100%


      Sorting block of length 461 for bucket 24


      (Using difference cover)


      bucket 28: 20%


      bucket 27: 20%


      bucket 25: 100%


      Sorting block of length 879 for bucket 25


      (Using difference cover)


      bucket 26: 20%


      bucket 23: 60%


      bucket 29: 20%


      bucket 30: 20%


      bucket 27: 30%


      Sorting block time: 00:00:00


      bucket 28: 30%


      bucket 26: 30%


      Sorting block time: 00:00:00


    Returning block of 462 for bucket 24


    Returning block of 880 for bucket 25


      bucket 29: 30%


    Getting block 31 of 60


      Reserving size (936) for bucket 31


      bucket 23: 70%


      Calculating Z arrays for bucket 31


      bucket 30: 30%


      Entering block accumulator loop for bucket 31:


    Getting block 32 of 60


      bucket 27: 40%


      Reserving size (936) for bucket 32


      bucket 28: 40%


      Calculating Z arrays for bucket 32


      Entering block accumulator loop for bucket 32:


      bucket 26: 40%


      bucket 29: 40%


      bucket 31: 10%


      bucket 30: 40%


      bucket 23: 80%


      bucket 27: 50%


      bucket 32: 10%


      bucket 28: 50%


      bucket 31: 20%


      bucket 26: 50%


      bucket 30: 50%


      bucket 29: 50%


      bucket 32: 20%


      bucket 27: 60%


      bucket 28: 60%


      bucket 32: 30%


      bucket 31: 30%


      bucket 23: 90%


      bucket 30: 60%


      bucket 26: 60%


      bucket 27: 70%


      bucket 29: 60%


      bucket 32: 40%


      bucket 28: 70%


      bucket 31: 40%


      bucket 30: 70%


      bucket 32: 50%


      bucket 27: 80%


      bucket 23: 100%


      bucket 31: 50%


      Sorting block of length 601 for bucket 23


      bucket 28: 80%


      (Using difference cover)


      bucket 29: 70%


      bucket 26: 70%


      bucket 30: 80%


      bucket 32: 60%


      bucket 27: 90%


      bucket 31: 60%


      bucket 28: 90%


      bucket 32: 70%


      bucket 29: 80%


      bucket 30: 90%


      bucket 31: 70%


      bucket 26: 80%


      bucket 27: 100%


      Sorting block of length 827 for bucket 27


      (Using difference cover)


      bucket 32: 80%


      bucket 28: 100%


      Sorting block of length 862 for bucket 28


      (Using difference cover)


      bucket 31: 80%


      bucket 30: 100%


      bucket 29: 90%


      Sorting block of length 449 for bucket 30


      (Using difference cover)


      bucket 32: 90%


      bucket 31: 90%


      bucket 26: 90%


      Sorting block time: 00:00:00


    Returning block of 602 for bucket 23


      bucket 32: 100%


      bucket 29: 100%


      Sorting block of length 607 for bucket 32


      Sorting block time: 00:00:00


      (Using difference cover)


      Sorting block of length 616 for bucket 29


      (Using difference cover)


    Returning block of 828 for bucket 27


      bucket 31: 100%


      Sorting block of length 720 for bucket 31


      (Using difference cover)


      bucket 26: 100%


      Sorting block of length 732 for bucket 26


      Sorting block time: 00:00:00


    Returning block of 450 for bucket 30


    Getting block 33 of 60


      (Using difference cover)


    Getting block 34 of 60


      Reserving size (936) for bucket 33


      Reserving size (936) for bucket 34


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 33


      Calculating Z arrays for bucket 34


    Returning block of 863 for bucket 28


      Entering block accumulator loop for bucket 33:


    Getting block 35 of 60


      Reserving size (936) for bucket 35


      Sorting block time: 00:00:00


      Calculating Z arrays for bucket 35


      Entering block accumulator loop for bucket 34:


    Returning block of 608 for bucket 32


      Entering block accumulator loop for bucket 35:


      bucket 33: 10%


      Sorting block time: 00:00:00


    Getting block 36 of 60


    Returning block of 617 for bucket 29


      Reserving size (936) for bucket 36


      Calculating Z arrays for bucket 36


      Sorting block time: 00:00:00


    Returning block of 721 for bucket 31


    Getting block 37 of 60


      Entering block accumulator loop for bucket 36:


      Reserving size (936) for bucket 37


      bucket 35: 10%


      Calculating Z arrays for bucket 37


      Entering block accumulator loop for bucket 37:


      bucket 33: 20%


    Getting block 38 of 60


      Reserving size (936) for bucket 38


      Calculating Z arrays for bucket 38


      bucket 34: 10%


      Entering block accumulator loop for bucket 38:


    Getting block 39 of 60


      bucket 36: 10%


      Reserving size (936) for bucket 39


      bucket 35: 20%


      Calculating Z arrays for bucket 39


      bucket 37: 10%


      Sorting block time: 00:00:00


    Returning block of 733 for bucket 26


      bucket 33: 30%


      Entering block accumulator loop for bucket 39:


      bucket 38: 10%


      bucket 36: 20%


      bucket 37: 20%


      bucket 35: 30%


      bucket 34: 20%


      bucket 39: 10%


      bucket 33: 40%


    Getting block 40 of 60


      Reserving size (936) for bucket 40


      Calculating Z arrays for bucket 40


      Entering block accumulator loop for bucket 40:


      bucket 38: 20%


      bucket 37: 30%


      bucket 36: 30%


      bucket 39: 20%


      bucket 35: 40%


      bucket 33: 50%


      bucket 34: 30%


      bucket 37: 40%


      bucket 38: 30%


      bucket 36: 40%


      bucket 39: 30%


      bucket 35: 50%


      bucket 33: 60%


      bucket 40: 10%


      bucket 37: 50%


      bucket 38: 40%


      bucket 36: 50%


      bucket 39: 40%


      bucket 34: 40%


      bucket 35: 60%


      bucket 33: 70%


      bucket 40: 20%


      bucket 37: 60%


      bucket 39: 50%


      bucket 36: 60%


      bucket 38: 50%


      bucket 35: 70%


      bucket 33: 80%


      bucket 34: 50%


      bucket 39: 60%


      bucket 37: 70%


      bucket 40: 30%


      bucket 36: 70%


      bucket 38: 60%


      bucket 35: 80%


      bucket 33: 90%


      bucket 39: 70%


      bucket 37: 80%


      bucket 36: 80%


      bucket 34: 60%


      bucket 40: 40%


      bucket 35: 90%


      bucket 38: 70%


      bucket 33: 100%


      bucket 39: 80%


      Sorting block of length 608 for bucket 33


      (Using difference cover)


      bucket 37: 90%


      bucket 36: 90%


      bucket 40: 50%


      bucket 39: 90%


      bucket 38: 80%


      bucket 35: 100%


      Sorting block time: 00:00:00


      bucket 34: 70%


      Sorting block of length 346 for bucket 35


      (Using difference cover)


    Returning block of 609 for bucket 33


      bucket 40: 60%


      bucket 36: 100%


      bucket 37: 100%


      bucket 39: 100%


      Sorting block of length 446 for bucket 36


      (Using difference cover)


      bucket 38: 90%


      Sorting block of length 926 for bucket 37


      (Using difference cover)


      Sorting block of length 988 for bucket 39


      (Using difference cover)


    Getting block 41 of 60


      Reserving size (936) for bucket 41


      Calculating Z arrays for bucket 41


      Entering block accumulator loop for bucket 41:


      Sorting block time: 00:00:00


    Returning block of 347 for bucket 35


      bucket 40: 70%


      bucket 34: 80%


      bucket 38: 100%


      Sorting block of length 774 for bucket 38


      (Using difference cover)


      Sorting block time: 00:00:00


    Returning block of 447 for bucket 36


    Getting block 42 of 60


      bucket 41: 10%


      Reserving size (936) for bucket 42


      Calculating Z arrays for bucket 42


      Entering block accumulator loop for bucket 42:


      bucket 40: 80%


    Getting block 43 of 60


      Reserving size (936) for bucket 43


      Calculating Z arrays for bucket 43


      Entering block accumulator loop for bucket 43:


      Sorting block time: 00:00:00


    Returning block of 927 for bucket 37


      bucket 41: 20%


      bucket 34: 90%


      bucket 40: 90%


      bucket 42: 10%


      Sorting block time: 00:00:00


    Returning block of 775 for bucket 38


      bucket 43: 10%


    Getting block 44 of 60


      bucket 40: 100%


      bucket 41: 30%


      Reserving size (936) for bucket 44


      Sorting block of length 146 for bucket 40


      (Using difference cover)


      Calculating Z arrays for bucket 44


    Getting block 45 of 60


      Reserving size (936) for bucket 45


      Calculating Z arrays for bucket 45


      bucket 42: 20%


      Sorting block time: 00:00:00


      Entering block accumulator loop for bucket 44:


    Returning block of 989 for bucket 39


      Entering block accumulator loop for bucket 45:


      bucket 34: 100%


      bucket 43: 20%


      Sorting block of length 631 for bucket 34


      (Using difference cover)


      bucket 41: 40%


      Sorting block time: 00:00:00


    Returning block of 147 for bucket 40


      bucket 42: 30%


    Getting block 46 of 60


      Reserving size (936) for bucket 46


      Calculating Z arrays for bucket 46


    Getting block 47 of 60


      bucket 44: 10%


      Reserving size (936) for bucket 47


      Entering block accumulator loop for bucket 46:


      Calculating Z arrays for bucket 47


      bucket 43: 30%


      Entering block accumulator loop for bucket 47:


      bucket 41: 50%


      bucket 45: 10%


      Sorting block time: 00:00:00


    Returning block of 632 for bucket 34


      bucket 46: 10%


      bucket 42: 40%


      bucket 44: 20%


      bucket 43: 40%


      bucket 41: 60%


      bucket 45: 20%


      bucket 46: 20%


      bucket 44: 30%


      bucket 47: 10%


      bucket 42: 50%


    Getting block 48 of 60


      Reserving size (936) for bucket 48


      Calculating Z arrays for bucket 48


      bucket 43: 50%


      bucket 46: 30%


      Entering block accumulator loop for bucket 48:


      bucket 44: 40%


      bucket 41: 70%


      bucket 47: 20%


      bucket 45: 30%


      bucket 46: 40%


      bucket 42: 60%


      bucket 43: 60%


      bucket 44: 50%


      bucket 47: 30%


      bucket 41: 80%


      bucket 46: 50%


      bucket 48: 10%


      bucket 44: 60%


      bucket 42: 70%


      bucket 47: 40%


      bucket 45: 40%


      bucket 43: 70%


      bucket 41: 90%


      bucket 46: 60%


      bucket 47: 50%


      bucket 44: 70%


      bucket 42: 80%


      bucket 43: 80%


      bucket 48: 20%


      bucket 45: 50%


      bucket 46: 70%


      bucket 41: 100%


      bucket 47: 60%


      bucket 44: 80%


      Sorting block of length 554 for bucket 41


      (Using difference cover)


      bucket 42: 90%


      bucket 43: 90%


      bucket 46: 80%


      bucket 47: 70%


      bucket 45: 60%


      bucket 44: 90%


      bucket 48: 30%


      bucket 46: 90%


      bucket 42: 100%


      bucket 47: 80%


      Sorting block of length 478 for bucket 42


      (Using difference cover)


      bucket 43: 100%


      bucket 44: 100%


      Sorting block time: 00:00:00


      Sorting block of length 734 for bucket 43


      (Using difference cover)


      Sorting block of length 628 for bucket 44


      (Using difference cover)


    Returning block of 555 for bucket 41


      bucket 45: 70%


      bucket 46: 100%


      bucket 47: 90%


      bucket 48: 40%


      Sorting block of length 683 for bucket 46


      (Using difference cover)


    Getting block 49 of 60


      Reserving size (936) for bucket 49


      Calculating Z arrays for bucket 49


      bucket 45: 80%


      Entering block accumulator loop for bucket 49:


      Sorting block time: 00:00:00


    Returning block of 479 for bucket 42


      bucket 47: 100%


      Sorting block of length 434 for bucket 47


      (Using difference cover)


      Sorting block time: 00:00:00


    Returning block of 735 for bucket 43


      bucket 48: 50%


    Getting block 50 of 60


      Reserving size (936) for bucket 50


      Calculating Z arrays for bucket 50


      bucket 45: 90%


      Sorting block time: 00:00:00


      bucket 49: 10%


    Returning block of 629 for bucket 44


      Entering block accumulator loop for bucket 50:


      Sorting block time: 00:00:00


    Returning block of 684 for bucket 46


    Getting block 51 of 60


      Reserving size (936) for bucket 51


      Calculating Z arrays for bucket 51


      Entering block accumulator loop for bucket 51:


    Getting block 52 of 60


      bucket 48: 60%


      bucket 49: 20%


      bucket 50: 10%


      bucket 45: 100%


      Sorting block time: 00:00:00


      Reserving size (936) for bucket 52


      Sorting block of length 883 for bucket 45


      (Using difference cover)


    Returning block of 435 for bucket 47


      Calculating Z arrays for bucket 52


    Getting block 53 of 60


      Reserving size (936) for bucket 53


      Calculating Z arrays for bucket 53


      bucket 51: 10%


      Entering block accumulator loop for bucket 52:


      Entering block accumulator loop for bucket 53:


      bucket 49: 30%


    Getting block 54 of 60


      bucket 50: 20%


      Reserving size (936) for bucket 54


      bucket 48: 70%


      Calculating Z arrays for bucket 54


      bucket 52: 10%


      Entering block accumulator loop for bucket 54:


      bucket 51: 20%


      bucket 53: 10%


      bucket 49: 40%


      Sorting block time: 00:00:00


    Returning block of 884 for bucket 45


      bucket 50: 30%


      bucket 52: 20%


      bucket 54: 10%


      bucket 53: 20%


      bucket 48: 80%


      bucket 51: 30%


      bucket 49: 50%


    Getting block 55 of 60


      Reserving size (936) for bucket 55


      Calculating Z arrays for bucket 55


      bucket 52: 30%


      bucket 50: 40%


      Entering block accumulator loop for bucket 55:


      bucket 53: 30%


      bucket 54: 20%


      bucket 49: 60%


      bucket 51: 40%


      bucket 48: 90%


      bucket 53: 40%


      bucket 52: 40%


      bucket 54: 30%


      bucket 50: 50%


      bucket 55: 10%


      bucket 53: 50%


      bucket 49: 70%


      bucket 51: 50%


      bucket 52: 50%


      bucket 48: 100%


      Sorting block of length 643 for bucket 48


      (Using difference cover)


      bucket 54: 40%


      bucket 53: 60%


      bucket 50: 60%


      bucket 55: 20%


      bucket 49: 80%


      bucket 51: 60%


      bucket 52: 60%


      bucket 53: 70%


      bucket 54: 50%


      bucket 50: 70%


      Sorting block time: 00:00:00


    Returning block of 644 for bucket 48


      bucket 49: 90%


      bucket 52: 70%


      bucket 55: 30%


      bucket 53: 80%


      bucket 51: 70%


      bucket 54: 60%


      bucket 52: 80%


      bucket 50: 80%


      bucket 53: 90%


      bucket 49: 100%


    Getting block 56 of 60


      Sorting block of length 757 for bucket 49


      (Using difference cover)


      Reserving size (936) for bucket 56


      Calculating Z arrays for bucket 56


      bucket 55: 40%


      bucket 51: 80%


      Entering block accumulator loop for bucket 56:


      bucket 54: 70%


      bucket 52: 90%


      bucket 53: 100%


      Sorting block of length 614 for bucket 53


      (Using difference cover)


      bucket 50: 90%


      bucket 55: 50%


      bucket 52: 100%


      Sorting block of length 613 for bucket 52


      Sorting block time: 00:00:00


      (Using difference cover)


      bucket 51: 90%


    Returning block of 758 for bucket 49


      bucket 54: 80%


      Sorting block time: 00:00:00


    Returning block of 615 for bucket 53


      bucket 56: 10%


      bucket 55: 60%


      bucket 50: 100%


    Getting block 57 of 60


      Sorting block time: 00:00:00


      Sorting block of length 302 for bucket 50


      (Using difference cover)


      Reserving size (936) for bucket 57


    Returning block of 614 for bucket 52


      Calculating Z arrays for bucket 57


      bucket 51: 100%


      Sorting block of length 653 for bucket 51


      (Using difference cover)


      bucket 54: 90%


      Entering block accumulator loop for bucket 57:


    Getting block 58 of 60


      Reserving size (936) for bucket 58


      Calculating Z arrays for bucket 58


      Entering block accumulator loop for bucket 58:


    Getting block 59 of 60


      bucket 55: 70%


      Reserving size (936) for bucket 59


      Sorting block time: 00:00:00


      Sorting block time: 00:00:00


    Returning block of 654 for bucket 51


    Returning block of 303 for bucket 50


      Calculating Z arrays for bucket 59


      bucket 54: 100%


      bucket 56: 20%


      bucket 57: 10%


      Sorting block of length 507 for bucket 54


      (Using difference cover)


      Entering block accumulator loop for bucket 59:


      bucket 58: 10%


      bucket 55: 80%


    Getting block 60 of 60


      Reserving size (936) for bucket 60


      Calculating Z arrays for bucket 60


      Entering block accumulator loop for bucket 60:


      bucket 60: 10%


      bucket 58: 20%


      bucket 57: 20%


      Sorting block time: 00:00:00


    Returning block of 508 for bucket 54


      bucket 59: 10%


      bucket 56: 30%


      bucket 60: 20%


      bucket 55: 90%


      bucket 58: 30%


      bucket 57: 30%


      bucket 60: 30%


      bucket 58: 40%


      bucket 59: 20%


      bucket 60: 40%


      bucket 57: 40%


      bucket 55: 100%


      Sorting block of length 812 for bucket 55


      (Using difference cover)


      bucket 58: 50%


      bucket 59: 30%


      bucket 60: 50%


      bucket 56: 40%


      bucket 58: 60%


      bucket 57: 50%


      bucket 60: 60%


      bucket 59: 40%


      Sorting block time: 00:00:00


    Returning block of 813 for bucket 55


      bucket 58: 70%


      bucket 60: 70%


      bucket 59: 50%


      bucket 57: 60%


      bucket 56: 50%


      bucket 60: 80%


      bucket 58: 80%


      bucket 59: 60%


      bucket 57: 70%


      bucket 60: 90%


      bucket 59: 70%


      bucket 58: 90%


      bucket 57: 80%


      bucket 60: 100%


      Sorting block of length 754 for bucket 60


      (Using difference cover)


      bucket 56: 60%


      bucket 58: 100%


      Sorting block of length 603 for bucket 58


      (Using difference cover)


      bucket 59: 80%


      bucket 57: 90%


      Sorting block time: 00:00:00


    Returning block of 755 for bucket 60


      bucket 59: 90%


      bucket 57: 100%


      Sorting block of length 633 for bucket 57


      (Using difference cover)


      Sorting block time: 00:00:00


      bucket 56: 70%


    Returning block of 604 for bucket 58


      bucket 59: 100%


      Sorting block of length 878 for bucket 59


      (Using difference cover)


      Sorting block time: 00:00:00


      bucket 56: 80%


    Returning block of 634 for bucket 57


      Sorting block time: 00:00:00


    Returning block of 879 for bucket 59


      bucket 56: 90%


      bucket 56: 100%


      Sorting block of length 721 for bucket 56


      (Using difference cover)


      Sorting block time: 00:00:00


    Returning block of 722 for bucket 56


    Exited Ebwt loop


    fchr[A]: 0


    fchr[C]: 10842


    fchr[G]: 19880


    fchr[T]: 30171


    fchr[$]: 39937


    Exiting Ebwt::buildToDisk()


    Returning from initFromVector


    Wrote 4207850 bytes to primary EBWT file: /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.rev.1.bt2


    Wrote 9992 bytes to secondary EBWT file: /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB.rev.2.bt2


    Re-opening _in1 and _in2 as input streams


    Returning from Ebwt constructor


    Headers:


        len: 39937


        bwtLen: 39938


        sz: 9985


        bwtSz: 9985


        lineRate: 6


        offRate: 4


        offMask: 0xfffffff0


        ftabChars: 10


        eftabLen: 20


        eftabSz: 80


        ftabLen: 1048577


        ftabSz: 4194308


        offsLen: 2497


        offsSz: 9988


        lineSz: 64


        sideSz: 64


        sideBwtSz: 48


        sideBwtLen: 192


        numSides: 209


        numLines: 209


        ebwtTotLen: 13376


        ebwtTotSz: 13376


        color: 0


        reverse: 1


    Total time for backward call to driver() for mirror index: 00:00:00

Command completed. Elapsed time: 0:00:01. Running peak memory: 0.037GB. PID: 1045591; Command: bowtie2-build; Return code: 0; Memory used: 0.037GB

INFO     Asset 't7/bowtie2_index:2.3.5' build succeeded                                              builder.py:666

### Pipeline completed. Epilogue * Elapsed time (this run): 0:00:01 * Total elapsed time (all runs): 0:00:00 * Peak memory (this run): 0.0368 GB * Pipeline completed time: 2026-08-20 21:55:45

INFO     Added: 't7/bowtie2_index:2.3.5'                                                             content.py:381

INFO     Set default asset: 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index:2.3.5'                      seek.py:612

INFO     Added asset: 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index:2.3.5'                         builder.py:705

INFO     Created alias directories: /tmp/refgenie_demo_2hzk45es/alias/t7/bowtie2_index/2.3.5        symlinks.py:223

INFO     Updated parents of 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index:2.3.5'                 relations.py:166

INFO     Staging asset: kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index:2.3.5                            stage.py:67

INFO     Reconciled file-serving stage dir:                                                            stage.py:335
         /tmp/refgenie_archive_demo_3ps6xfvv/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5 (6               
         link(s), 6 written or removed; excluded 0 colocation symlink(s))

INFO     Created file-serving StagedAsset for kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index:2.3.5     stage.py:217

Asset(name='2.3.5', description='Genome index for bowtie2, produced with bowtie2-build', size=8445686, colocate=None, updated_at=datetime.datetime(2026, 8, 21, 1, 55, 46, 9717), path='data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2b4c1900ec5a2b9fce39e45a71e6bed38ca234d394b8af28c2a3b0e09548d51e', digest='2b4c1900ec5a2b9fce39e45a71e6bed38ca234d394b8af28c2a3b0e09548d51e', recipe_id=2, serving_modes_override=None, asset_group_id=3, created_at=datetime.datetime(2026, 8, 21, 1, 55, 46, 9733)) Let's list the assets for the genome `t7` to verify that the `bowtie2_index` asset has been built successfully. ```python refgenie.asset.table(genome_digests=[refgenie.alias.resolve(GenomeAlias("t7"))])[0] ```

                     Refgenie assets. Source: local                     
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Aliases ┃ Genome digest                    ┃ Asset group   ┃ Asset   ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ t7      │ kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB │ fasta         │ default │
│ t7      │ kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB │ bowtie2_index │ 2.3.5   │
└─────────┴──────────────────────────────────┴───────────────┴─────────┘

One of the assets was also archived (a neccessary step to serve the assets via the refgenie server). Let's list the archived assets. ```python print(refgenie.stage.table()) ```

                                                   Staged Assets                                                   
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━┓
┃ Asset Digest                                    ┃ Asset name                                      ┃ Mode ┃ Size ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━┩
│ 2b4c1900ec5a2b9fce39e45a71e6bed38ca234d394b8af… │ kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index… │ file │ -    │
└─────────────────────────────────────────────────┴─────────────────────────────────────────────────┴──────┴──────┘

Asset `bowtie2_index` has been built successfully for the `t7` genome, and automatically tagged with `2.3.5`, indicating the version of Bowtie2 software used (this behavior is encoded in the recipe). ## Interact with aliases Let's list the aliases: ```python refgenie.alias.table() ```

                Genome aliases                
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Aliases ┃ Genome digest                    ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ rCRSd   │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │
│ t7      │ kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB │
└─────────┴──────────────────────────────────┘

Let's assign another alias to the same genome digest, this way we can refer to the same genome in multiple ways. ```python t7_alias = refgenie.set_genome_alias( alias_name="Bacteriophage-T7", genome_digest="kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB", genome_description="My favorite genome", ) fav_alias = refgenie.set_genome_alias( alias_name="myFavGenome", genome_digest="kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB", genome_description="My favorite genome", ) ```

INFO     Added alias: Bacteriophage-T7                                                                 alias.py:305

INFO     Created alias directories:                                                                 symlinks.py:223
         /tmp/refgenie_demo_2hzk45es/alias/Bacteriophage-T7/fasta/default

INFO     Created alias directories:                                                                 symlinks.py:223
         /tmp/refgenie_demo_2hzk45es/alias/Bacteriophage-T7/bowtie2_index/2.3.5

INFO     Added alias: myFavGenome                                                                      alias.py:305

INFO     Created alias directories: /tmp/refgenie_demo_2hzk45es/alias/myFavGenome/fasta/default     symlinks.py:223

INFO     Created alias directories:                                                                 symlinks.py:223
         /tmp/refgenie_demo_2hzk45es/alias/myFavGenome/bowtie2_index/2.3.5

The new alias should be listed in the aliases: ```python refgenie.alias.table() ```

                             Genome aliases                             
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Aliases                           ┃ Genome digest                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ rCRSd                             │ jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP │
│ t7, Bacteriophage-T7, myFavGenome │ kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB │
└───────────────────────────────────┴──────────────────────────────────┘

The command not only creates a new alias, but also creates a symbolic links to the files in the `data` directory for that genome. Conversely, alias removal will remove the symbolic links, but not the files in the `data` directory. ```python refgenie.alias.remove("myFavGenome") ```

INFO     Deleting alias-owned files: /tmp/refgenie_demo_2hzk45es/alias/myFavGenome                  symlinks.py:145

INFO     Deleting alias-owned files: /tmp/refgenie_demo_2hzk45es/builds/myFavGenome                 symlinks.py:145

INFO     Removed alias: myFavGenome                                                                    alias.py:337

## Retrieve paths to assets Most importantly, we can retrieve paths to refgenie-managed files. All below commands will return the same path to the fasta file managed by Refgenie: ```python # A registry path names its genome by alias; the path is under that alias print(refgenie.asset.seek_components(refgenie.parse_asset_registry_path("t7/fasta"))) print( refgenie.asset.seek_components( refgenie.parse_asset_registry_path("Bacteriophage-T7/fasta:default") ) ) # Library methods take the genome digest; resolve an alias to get one t7 = refgenie.alias.resolve(GenomeAlias("t7")) print(refgenie.asset.seek(t7, "fasta", "default", "fasta")) ```

/tmp/refgenie_demo_2hzk45es/alias/t7/fasta/default/t7.fa

/tmp/refgenie_demo_2hzk45es/alias/Bacteriophage-T7/fasta/default/Bacteriophage-T7.fa

/tmp/refgenie_demo_2hzk45es/alias/t7/fasta/default/t7.fa

## Remove an asset Let's remove the `bowtie2_index` asset for the `dm6` genome. ```python refgenie.asset.remove("bowtie2_index", "2.3.5", genome_digest=t7) ```

INFO     Queued asset files for removal after commit:                                                 events.py:108
         /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2b4c1900ec5a              
         2b9fce39e45a71e6bed38ca234d394b8af28c2a3b0e09548d51e

INFO     Removed directory:                                                                          cleanup.py:107
         /tmp/refgenie_archive_demo_3ps6xfvv/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2.3.5

INFO     Removed empty directory:                                                                     events.py:185
         /tmp/refgenie_archive_demo_3ps6xfvv/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index

INFO     Removed directory:                                                                          cleanup.py:107
         /tmp/refgenie_demo_2hzk45es/data/kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index/2b4c1900ec5               
         a2b9fce39e45a71e6bed38ca234d394b8af28c2a3b0e09548d51e

INFO     Removed asset group and all assets 'kN9XHLKLS_u7ei2GH87H-qpQrkz8moPB/bowtie2_index'         manager.py:568

## Data channels Refgenie supports data channels, which are used to allow third-party tool developers to expose their recipes and asset classes to Refgenie ecosystem. In the simplest case, data channels is just a github repository with an index file that lists available asset classes and recipes, like so: ```yaml asset_class: dir: asset_classes # optional, needed only if the asset classes are stored in a subdirectory files: # list of asset class files, relative to the index file (or directory) - fasta.yaml - bowtie2_index.yaml recipe: dir: recipes # optional, needed only if the recipes are stored in a subdirectory files: # list of recipe files, relative to the index file (or directory) - fasta.yaml - bowtie2_index.yaml ``` One such example is the [refgenie/recipes](https://github.com/refgenie/recipes/blob/refgenie1/index.yaml) repository, which can be added as a data channel to refgenie in the following way: ```python from refgenie.db.tables import DataChannelType data_channel = refgenie.sources.add_channel( name="refgenie-recipes", type=DataChannelType.https, index_address="https://refgenie.github.io/refgenie-registry/index.yaml", description="Refgenie recipes channel", ) print(refgenie.sources.channels_table()) ```

                                                   Data Channels                                                   
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Name             ┃ Type  ┃ Index Address                           ┃ Description              ┃ Credentials set ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ refgenie-recipes │ https │ https://refgenie.github.io/refgenie-re… │ Refgenie recipes channel │ False           │
└──────────────────┴───────┴─────────────────────────────────────────┴──────────────────────────┴─────────────────┘

Subsequently, the asset classes and recipes from the data channel can be listed and added to the refgenie instance. ```python for asset_class in refgenie.sources.iter_asset_classes("refgenie-recipes"): try: refgenie.asset_class.add(asset_class) except Exception as e: print(e) for recipe in refgenie.sources.iter_recipes("refgenie-recipes"): try: refgenie.recipe.add(recipe) except Exception as e: print(e) ```

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/index.yaml "HTTP/1.1 200    _client.py:1025
         OK"

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/abundant_sequences.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/abundant_sequences.yaml                        
         "HTTP/1.1 200 OK"

INFO     Registered 'abundant_sequences' asset class                                              asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/bed.yaml        io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/bed.yaml      _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'bed' asset class                                                             asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/bismark_bt2_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/bismark_bt2_index.yaml                         
         "HTTP/1.1 200 OK"

INFO     Registered 'bismark_bt2_index' asset class                                               asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/blacklist.yaml  io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/blacklist.yaml "HTTP/1.1 200                   
         OK"

INFO     Registered 'blacklist' asset class                                                       asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/bowtie1_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/bowtie1_index.yaml "HTTP/1.1                   
         200 OK"

INFO     Registered 'bowtie1_index' asset class                                                   asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/bowtie2_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/bowtie2_index.yaml "HTTP/1.1                   
         200 OK"

Asset class 'bowtie2_index' version '0.0.1' already exists.

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/bwa_index.yaml  io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/bwa_index.yaml "HTTP/1.1 200                   
         OK"

INFO     Registered 'bwa_index' asset class                                                       asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/cellranger_reference.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/cellranger_reference.yaml                      
         "HTTP/1.1 200 OK"

INFO     Registered 'cellranger_reference' asset class                                            asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/dbnsfp.yaml     io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/dbnsfp.yaml   _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'dbnsfp' asset class                                                          asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/dbsnp.yaml      io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/dbsnp.yaml    _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'dbsnp' asset class                                                           asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/ensembl_rb.yaml io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/ensembl_rb.yaml "HTTP/1.1 200                  
         OK"

INFO     Registered 'ensembl_rb' asset class                                                      asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/epilog_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/epilog_index.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'epilog_index' asset class                                                    asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/fasta.yaml      io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/fasta.yaml    _client.py:1025
         "HTTP/1.1 200 OK"

Asset class 'fasta' version '0.1.0' already exists.

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/fasta_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/fasta_index.yaml "HTTP/1.1 200                 
         OK"

INFO     Registered 'fasta_index' asset class                                                     asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/feat_annotation.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/feat_annotation.yaml "HTTP/1.1                 
         200 OK"

INFO     Registered 'feat_annotation' asset class                                                 asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/gtf.yaml        io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/gtf.yaml      _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'gtf' asset class                                                             asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/hisat2_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/hisat2_index.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'hisat2_index' asset class                                                    asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/kallisto_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/kallisto_index.yaml "HTTP/1.1                  
         200 OK"

INFO     Registered 'kallisto_index' asset class                                                  asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/refgene_anno.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/refgene_anno.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'refgene_anno' asset class                                                    asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_index.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'salmon_index' asset class                                                    asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_partial_sa_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_partial_sa_index.yaml                   
         "HTTP/1.1 200 OK"

INFO     Registered 'salmon_partial_sa_index' asset class                                         asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_sa_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/salmon_sa_index.yaml "HTTP/1.1                 
         200 OK"

INFO     Registered 'salmon_sa_index' asset class                                                 asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/small_rna.yaml  io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/small_rna.yaml "HTTP/1.1 200                   
         OK"

INFO     Registered 'small_rna' asset class                                                       asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/star_index.yaml io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/star_index.yaml "HTTP/1.1 200                  
         OK"

INFO     Registered 'star_index' asset class                                                      asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/suffixerator_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/suffixerator_index.yaml                        
         "HTTP/1.1 200 OK"

INFO     Registered 'suffixerator_index' asset class                                              asset_class.py:90

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/asset_classes/tallymer_index.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/asset_classes/tallymer_index.yaml "HTTP/1.1                  
         200 OK"

INFO     Registered 'tallymer_index' asset class                                                  asset_class.py:90

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/asset_classes/tgMap.yaml      io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/asset_classes/tgMap.yaml    _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'tgMap' asset class                                                           asset_class.py:90

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/index.yaml "HTTP/1.1 200    _client.py:1025
         OK"

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/abundant_sequences/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/abundant_sequences/recipe.yaml                       
         "HTTP/1.1 200 OK"

INFO     Registered 'abundant_sequences' recipe                                                       recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/bed12/recipe.yaml     io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/recipes/bed12/recipe.yaml   _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'bed12' recipe                                                                    recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/bismark_bt2_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/bismark_bt2_index/recipe.yaml                        
         "HTTP/1.1 200 OK"

INFO     Registered 'bismark_bt2_index' recipe                                                        recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/blacklist/recipe.yaml io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/blacklist/recipe.yaml "HTTP/1.1 200                  
         OK"

INFO     Registered 'blacklist' recipe                                                                recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/bowtie1_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/bowtie1_index/recipe.yaml "HTTP/1.1                  
         200 OK"

INFO     Registered 'bowtie1_index' recipe                                                            recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/bowtie2_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/bowtie2_index/recipe.yaml "HTTP/1.1                  
         200 OK"

Recipe 'bowtie2_index' version '0.0.1' already exists.

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/bwa_index/recipe.yaml io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/bwa_index/recipe.yaml "HTTP/1.1 200                  
         OK"

INFO     Registered 'bwa_index' recipe                                                                recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/cellranger_reference/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/cellranger_reference/recipe.yaml                     
         "HTTP/1.1 200 OK"

INFO     Registered 'cellranger_reference' recipe                                                     recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/dbnsfp/recipe.yaml    io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/recipes/dbnsfp/recipe.yaml  _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'dbnsfp' recipe                                                                   recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/dbsnp/recipe.yaml     io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/recipes/dbsnp/recipe.yaml   _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'dbsnp' recipe                                                                    recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/ensembl_gtf/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/ensembl_gtf/recipe.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'ensembl_gtf' recipe                                                              recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/ensembl_rb/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/ensembl_rb/recipe.yaml "HTTP/1.1 200                 
         OK"

INFO     Registered 'ensembl_rb' recipe                                                               recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/epilog_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/epilog_index/recipe.yaml "HTTP/1.1                   
         200 OK"

INFO     Registered 'epilog_index' recipe                                                             recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/fasta/recipe.yaml     io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/recipes/fasta/recipe.yaml   _client.py:1025
         "HTTP/1.1 200 OK"

Recipe 'fasta' version '0.1.0' already exists.

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/fasta_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/fasta_index/recipe.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'fasta_index' recipe                                                              recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/fasta_txome/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/fasta_txome/recipe.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'fasta_txome' recipe                                                              recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/feat_annotation/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/feat_annotation/recipe.yaml "HTTP/1.1                
         200 OK"

INFO     Registered 'feat_annotation' recipe                                                          recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/gencode_gtf/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/gencode_gtf/recipe.yaml "HTTP/1.1 200                
         OK"

INFO     Registered 'gencode_gtf' recipe                                                              recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/hisat2_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/hisat2_index/recipe.yaml "HTTP/1.1                   
         200 OK"

INFO     Registered 'hisat2_index' recipe                                                             recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/kallisto_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/kallisto_index/recipe.yaml "HTTP/1.1                 
         200 OK"

INFO     Registered 'kallisto_index' recipe                                                           recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/refgene_anno/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/refgene_anno/recipe.yaml "HTTP/1.1                   
         200 OK"

INFO     Registered 'refgene_anno' recipe                                                             recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/salmon_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/salmon_index/recipe.yaml "HTTP/1.1                   
         200 OK"

INFO     Registered 'salmon_index' recipe                                                             recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/salmon_partial_sa_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/salmon_partial_sa_index/recipe.yaml                  
         "HTTP/1.1 200 OK"

INFO     Registered 'salmon_partial_sa_index' recipe                                                  recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/salmon_sa_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/salmon_sa_index/recipe.yaml "HTTP/1.1                
         200 OK"

INFO     Registered 'salmon_sa_index' recipe                                                          recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/small_rna/recipe.yaml io.py:31

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/small_rna/recipe.yaml "HTTP/1.1 200                  
         OK"

INFO     Registered 'small_rna' recipe                                                                recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/star_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/star_index/recipe.yaml "HTTP/1.1 200                 
         OK"

INFO     Registered 'star_index' recipe                                                               recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/suffixerator_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/suffixerator_index/recipe.yaml                       
         "HTTP/1.1 200 OK"

INFO     Registered 'suffixerator_index' recipe                                                       recipe.py:140

INFO     Reading YAML from URL:                                                                            io.py:31
         https://refgenie.github.io/refgenie-registry/recipes/tallymer_index/recipe.yaml

INFO     HTTP Request: GET                                                                          _client.py:1025
         https://refgenie.github.io/refgenie-registry/recipes/tallymer_index/recipe.yaml "HTTP/1.1                 
         200 OK"

INFO     Registered 'tallymer_index' recipe                                                           recipe.py:140

INFO     Reading YAML from URL: https://refgenie.github.io/refgenie-registry/recipes/tgMap/recipe.yaml     io.py:31

INFO     HTTP Request: GET https://refgenie.github.io/refgenie-registry/recipes/tgMap/recipe.yaml   _client.py:1025
         "HTTP/1.1 200 OK"

INFO     Registered 'tgMap' recipe                                                                    recipe.py:140

Alternatively, the same can be achieved by running the following CLI command: ```bash refgenie1 data_channel sync refgenie-recipes --exists-ok ``` ## RefgetStore sequence collection comparison Refgenie uses RefgetStore (from gtars) for sequence collection operations. Under the hood, refgenie uses the SeqCol digests to uniquely identify genomes. You can use RefgetStore directly to compute digests and compare sequence collections. ```python from gtars.refget import RefgetStore store = RefgetStore.in_memory() store.set_quiet(True) d1, _ = store.add_sequence_collection_from_fasta( REFGENIE_CODE_PATH.parent / "tests/data/rCRSd.fa" ) d2, _ = store.add_sequence_collection_from_fasta( REFGENIE_CODE_PATH.parent / "tests/data/rCRSd-extra.fa" ) store.compare(d1.digest, d2.digest) ``` {'digests': {'a': 'jthDpfNIgzM5AGJlOkRtfnky4rXMBIUP', 'b': 'smiTbD3jP5EwF4DNWVm0c6DGKRlFHfas'}, 'attributes': {'a_only': [], 'b_only': [], 'a_and_b': ['lengths', 'name_length_pairs', 'names', 'sequences', 'sorted_name_length_pairs', 'sorted_sequences']}, 'array_elements': {'a_count': {'sorted_name_length_pairs': 1, 'sequences': 1, 'sorted_sequences': 1, 'lengths': 1, 'names': 1, 'name_length_pairs': 1}, 'b_count': {'sequences': 2, 'names': 2, 'sorted_name_length_pairs': 2, 'sorted_sequences': 2, 'name_length_pairs': 2, 'lengths': 2}, 'a_and_b_count': {'sorted_name_length_pairs': 1, 'sorted_sequences': 1, 'name_length_pairs': 1, 'names': 1, 'sequences': 1, 'lengths': 1}, 'a_and_b_same_order': {'lengths': None, 'name_length_pairs': None, 'sorted_sequences': None, 'names': None, 'sequences': None, 'sorted_name_length_pairs': None}}}