Data Models

The refget package uses Pydantic and SQLModel for data validation and database ORM. These models represent the core data structures for sequence collections, DRS objects, and related metadata.

!!! success “Data models” Data Models are only needed if you want to develop new packages that rely on the refget Python API.

Model hierarchy

DrsObject (base)
└── FastaDrsObject (table)

SQLModel (base)
├── SequenceCollection (table)
├── Pangenome (table)
├── Sequence (table)
├── AccessMethod
├── AccessURL
└── Checksum

Core Models

SequenceCollection

The primary model representing a GA4GH sequence collection.

class SequenceCollection

A SQLModel/pydantic model that represents a refget sequence collection.

Properties

digest: str : Top-level digest of the SequenceCollection.

human_readable_names: List[HumanReadableNames]

lengths: LengthsAttr : Array of sequence lengths.

lengths_digest: str

name_length_pairs: NameLengthPairsAttr : Array of name-length pairs, representing the coordinate system of the collection.

name_length_pairs_digest: str

names: NamesAttr : Array of sequence names.

names_digest: str

pangenomes: List[Pangenome]

sequences: SequencesAttr : Array of sequence digests.

sequences_digest: str

sorted_name_length_pairs_digest: str : Digest of the sorted name-length pairs, representing a unique digest of sort-invariant coordinate system.

sorted_sequences: SortedSequencesAttr : Array of sorted sequence digests.

sorted_sequences_digest: str

Class Methods

from_PySequenceCollection(gtars_seq_col: gtarsSequenceCollection)

Given a PySequenceCollection object (from Rust bindings), create a SequenceCollection object.

Args: gtars_seq_col (PySequenceCollection): PySequenceCollection object from Rust bindings.

Returns: (SequenceCollection): The SequenceCollection object.

Raises: ImportError: If gtars is not installed (required for this conversion)

from_dict(seqcol_dict: dict, inherent_attrs: Optional[list] = DEFAULT_INHERENT_ATTRS)

Given a dict representation of a sequence collection, create a SequenceCollection object. This is the primary way to create a SequenceCollection object.

Args: seqcol_dict (dict): Dictionary representation of a canonical sequence collection object inherent_attrs (list, optional): List of inherent attributes to digest

Returns: (SequenceCollection): The SequenceCollection object

from_fasta_file(fasta_file: str)

Given a FASTA file, create a SequenceCollection object.

Args: fasta_file (str): Path to a FASTA file

Returns: (SequenceCollection): The SequenceCollection object

Raises: ImportError: If gtars is not installed (required for FASTA processing)

Methods

itemwise(limit = None)

Converts object into a list of dictionaries, one for each sequence in the collection.

level1()

Converts object into dict of level 1 representation of the SequenceCollection.

Returns attribute digests for most attributes, but returns raw values for passthru attributes. Note: Passthru handling for dict-based construction happens in seqcol_dict_to_level1_dict(). When passthru attributes are added to the database model, return .value instead of .digest here.

level2()

Converts object into dict of level 2 representation of the SequenceCollection.

FastaDrsObject

A DRS object specialized for FASTA files, storing file metadata and FAI index information.

class FastaDrsObject

A DRS object specialized for FASTA sequence files. Stores file metadata including size, checksums (SHA-256, MD5, and refget sequence collection digest), and creation time. The refget digest serves as the object ID, enabling content-addressable retrieval.

Properties

extra_line_bytes: Optional[int]

id: str

line_bases: Optional[int]

offsets: Optional[List[int]]

self_uri: Optional[str]

Class Methods

from_fasta_file(fasta_file: str, digest: str = None)

Given a FASTA file, create a FastaDrsObject object, return a populated FastaDrsObject with computed size and checksum.

Args: fasta_file (str): Path to a FASTA file digest (str): The refget digest of the sequence collection (optional). If not included, it will be computed

Returns: (FastaDrsObject): The FastaDrsObject object

Raises: ImportError: If gtars is not installed (required for FASTA processing)

Methods

to_response(base_uri: str = None)

Return a copy of this object with self_uri populated for API response.

Args: base_uri: Base URI for the DRS service (e.g., “drs://seqcolapi.databio.org”) If not provided, returns self unchanged.

Returns: FastaDrsObject with self_uri populated

DrsObject

Base model for GA4GH Data Repository Service (DRS) objects.

class DrsObject

A data object representing a single blob of bytes with metadata, checksums, and access methods. DRS objects are self-contained and provide all information needed for clients to retrieve the data. Conforms to GA4GH Data Repository Service (DRS) specification v1.4.0.

Properties

access_methods: List[AccessMethod]

aliases: List[str]

checksums: List[Checksum]

created_time: datetime

description: Optional[str]

id: str

mime_type: Optional[str]

name: Optional[str]

self_uri: str

size: int

updated_time: Optional[datetime]

version: Optional[str]

Class Methods

coerce_access_methods(v)

Coerce dicts to AccessMethod objects when loading from JSON.

coerce_checksums(v)

Coerce dicts to Checksum objects when loading from JSON.

Methods

serialize_access_methods(v)

Serialize AccessMethod objects (or dicts) to dicts for JSON output.

serialize_checksums(v)

Serialize Checksum objects (or dicts) to dicts for JSON output.

Pangenome

A collection of sequence collections representing a pangenome.

class Pangenome

Properties

collections: List[SequenceCollection]

collections_digest: str

digest: str

names: CollectionNamesAttr

names_digest: str

Class Methods

from_dict(pangenome_obj: dict, inherent_attrs: Optional[list] = None)

Given a dict representation of a pangenome, create a Pangenome object. This is the primary way to create a Pangenome object.

Args: pangenome_obj (dict): Dictionary representation of a canonical pangenome object

Returns: (Pangenome): The Pangenome object

Methods

level1()

Converts object into dict of level 1 representation of the Pangenome.

level2()

Converts object into dict of level 2 representation of the Pangenome.

level3()

Converts object into dict of level 3 representation of the Pangenome.

level4()

Converts object into dict of level 4 representation of the Pangenome.

Sequence

An individual sequence with its digest and content.

class Sequence

Properties

digest: str

length: int

sequence: str

Supporting Models

AccessMethod

Describes how to access object bytes (protocol type, URL, region).

class AccessMethod

Describes a method for accessing object bytes, including the protocol type (e.g., https, s3, gs) and either a direct URL or an access_id for the /access endpoint. At least one of access_url or access_id must be provided.

DRS 1.5.0 adds the ‘cloud’ field to explicitly specify the cloud provider.

Properties

access_id: Optional[str]

access_url: Optional[AccessURL]

cloud: Optional[str]

region: Optional[str]

type: Literal[‘s3’, ‘gs’, ‘ftp’, ‘gsiftp’, ‘globus’, ‘htsget’, ‘https’, ‘file’]

AccessURL

A fully resolvable URL with optional headers for authentication.

class AccessURL

A fully resolvable URL that can be used to fetch the actual object bytes. Optionally includes headers (e.g., authorization tokens) required for access.

Properties

headers: Optional[List[str]]

url: str

Checksum

A checksum for data integrity verification.

class Checksum

A checksum for data integrity verification. The type field indicates the hash algorithm (e.g., “sha-256”, “md5”) and the checksum field contains the hex-string encoded hash value.

Properties

checksum: str

type: str

Response Models

PaginationResult

Pagination metadata for list endpoints.

class PaginationResult

Properties

page: int

page_size: int

total: int

ResultsSequenceCollections

Paginated sequence collection results.

class ResultsSequenceCollections

Sequence collection results with pagination

Properties

pagination: PaginationResult

results: Dict[str, dict]

Similarities

Results from Jaccard similarity calculations.

class Similarities

Model to contain results from similarities calculations

Properties

pagination: PaginationResult

reference_digest: Optional[str]

similarities: List[Dict[str, Any]]

Attribute Tables

These models store individual attributes of sequence collections in normalized database tables:

NamesAttr

class NamesAttr

Properties

collection: List[SequenceCollection]

digest: str

value: list

LengthsAttr

class LengthsAttr

Properties

collection: List[SequenceCollection]

digest: str

value: list

SequencesAttr

class SequencesAttr

Properties

collection: List[SequenceCollection]

digest: str

value: list

NameLengthPairsAttr

class NameLengthPairsAttr

Properties

collection: List[SequenceCollection]

digest: str

value: list

Usage Examples

Creating a SequenceCollection from a FASTA file

from refget.models import SequenceCollection

# From a FASTA file (requires gtars)
seqcol = SequenceCollection.from_fasta_file("genome.fa")

# Access different representations
print(seqcol.digest)  # Top-level digest
print(seqcol.level1())  # Attribute digests
print(seqcol.level2())  # Full arrays
print(seqcol.itemwise())  # Per-sequence dicts

Creating a SequenceCollection from a dictionary

from refget.models import SequenceCollection

seqcol_dict = {
    "names": ["chr1", "chr2"],
    "lengths": [1000, 2000],
    "sequences": ["SQ.abc123...", "SQ.def456..."]
}

seqcol = SequenceCollection.from_dict(seqcol_dict)

Creating a FastaDrsObject

from refget.models import FastaDrsObject

# From a FASTA file
drs_obj = FastaDrsObject.from_fasta_file("genome.fa")

# Access DRS metadata
print(drs_obj.id)  # Sequence collection digest
print(drs_obj.size)  # File size in bytes
print(drs_obj.checksums)  # SHA-256, MD5
print(drs_obj.access_methods)  # How to download