The refget Python package
Section titled “The refget Python package”The refget Python package provides a Python implementation of the GA4GH Refget Specifications, which define standards for identifying and distributing reference biological sequences, like reference genomes. It provides standards at 3 levels of data: sequences, sequence collections, and pangenomes (in progress).
The refget Python package includes these capabilities:
Section titled “The refget Python package includes these capabilities:”| Standard | Local use (computing digests locally) | Client (connecting to a remote API) | API (implementing an http interface) | Agent (managing a SQL database) |
|---|---|---|---|---|
| Sequences | ✓ | ✓ | ✓ | ✓ |
| Sequence Collections | ✓ | ✓ | ✓ | ✓ |
| Pangenomes | ✗ | ✗ | ✓ | ✓ |
Package components
Section titled “Package components”The refget package provides several components for working with GA4GH refget standards:
-
Local digest functions - Python interface to fast Rust-based implementations of GA4GH digests for sequences and sequence collections.
-
RefgetStore - High-performance local storage for sequences and collections. Supports in-memory and on-disk modes, sequence retrieval by digest, FASTA export, and connecting to remote stores.
-
Clients - For interacting with remote Refget APIs:
SequenceClient,SequenceCollectionClient, andFastaDrsClient. -
Agents - For creating refget services with a PostgreSQL database backend.
RefgetDBAgentis the primary interface. -
FastAPI router - Implements the refget API endpoints. Attach to an existing FastAPI service to deploy your own sequence collections API.
-
Compliance tests - Evaluate a remote API instance against the sequence collections standard.
-
CLI - Commands for computing digests (
refget fasta), managing local stores (refget store), querying remote servers (refget seqcol), and database administration (refget admin).
How the pieces fit together
Section titled “How the pieces fit together”Installing refget pulls in three layers that are developed in two repositories.
| Layer | What it is | Where it lives |
|---|---|---|
| gtars-refget | The Rust core: digest algorithms, alphabet detection and encoding, the RefgetStore on-disk format, the FASTA import pipeline, and the store types | The gtars Rust workspace |
| gtars (Python bindings) | A PyO3 extension module exposing the Rust types to Python as gtars.refget | The gtars-python crate in the same workspace, published to PyPI as gtars |
| refget | The Python package: clients, the FastAPI router, the database agent, compliance tests, the CLI, and thin re-exports of the bindings | The refget repository |
The layers divide along a clear line. Anything that touches sequence bytes at scale, meaning digesting, encoding, storing, and retrieving, is implemented once in Rust and reached through the bindings. Anything that talks HTTP, SQL, or JSON schemas is implemented in Python.
The PyO3 bindings are a translation layer, not a reimplementation. Each Python class wraps the corresponding Rust struct and forwards calls to it, converting arguments and results at the boundary. There is no second copy of the digest logic, so a digest computed by the Python API, the CLI, the R bindings, or the Rust library is the same digest computed by the same code.
Because gtars is a compiled dependency, refget guards against its absence: refget.const.GTARS_INSTALLED reports whether the extension module loaded, and the store re-exports are None when it did not. Import the store types from refget.store:
from refget.store import RefgetStore, digest_fastaThis is equivalent to importing from gtars.refget directly, but it keeps your code pointed at the package’s stable surface.
Install
Section titled “Install”pip install refgetQuick start
Section titled “Quick start”Compute a sequence collection digest from a FASTA file
Section titled “Compute a sequence collection digest from a FASTA file”refget fasta digest genome.faQuery a remote seqcol server
Section titled “Query a remote seqcol server”# Get a collection by digestrefget seqcol show XZlrcEGi6mlopZ2uD8ObHkQB1d0oDwKk
# Compare two collectionsrefget seqcol compare digest1 digest2
# List collections on the serverrefget seqcol listUse the Python client
Section titled “Use the Python client”from refget.clients import SequenceCollectionClient
client = SequenceCollectionClient()collection = client.get_collection("XZlrcEGi6mlopZ2uD8ObHkQB1d0oDwKk")print(collection)Set up a local RefgetStore
Section titled “Set up a local RefgetStore”RefgetStore is basically an attempt to:
- solve efficiency issues with the original refget sequences protocol.
- provide a way to download the actual data in a sequence collection, which is not provided by the current sequence collection standard.
# Initialize a local storerefget store init
# Import a FASTA filerefget store add genome.fa
# Export sequencesrefget store export <digest> --output output.faFor a hands-on walkthrough, see the Getting Started tutorial.