Retrieving Subsequences by Digest
Given a sorted_sequences digest -- which identifies a set of sequences
regardless of ordering or naming -- this guide shows how to expand it into
individual sequence digests, inspect sequence metadata, and extract
subsequences.
Prerequisites: You should know how to install the refget package,
create a RefgetStore, and load FASTA files (see the RefgetStore tutorial).
For how to find collections that share a sorted_sequences digest, see
Matching by sequences.
Create a small FASTA file, load it into an in-memory store, and obtain the
sorted_sequences digest from the level 1 representation.
import osimport tempfile
from refget.store import RefgetStore, digest_fasta
temp_dir = tempfile.mkdtemp(prefix="subseq_howto_")
fasta_path = os.path.join(temp_dir, "genome.fa")with open(fasta_path, "w") as f: f.write(">chr1\nACGTACGTACGTACGTACGTACGTACGTACGT\n") f.write(">chr2\nGGGGAAAATTTTCCCCGGGGAAAATTTTCCCC\n") f.write(">chrM\nTTTTAAAACCCCGGGGNNNNACGT\n")
store = RefgetStore.in_memory()store.set_quiet(True)store.add_sequence_collection_from_fasta(fasta_path)
collection_digest = digest_fasta(fasta_path).digestlvl1 = store.get_collection_level1(collection_digest)sorted_seq_digest = lvl1["sorted_sequences"]
print(f"Collection digest: {collection_digest}")print(f"sorted_sequences digest: {sorted_seq_digest}")Collection digest: jxN4oDGY6BPur9HRIC2J-B7zNjl3MZEZsorted_sequences digest: wqf_wXxA05duQJOy5XTQ1TKWaakwd8VvExpand sorted_sequences into individual sequence digests
Section titled “Expand sorted_sequences into individual sequence digests”The level 1 sorted_sequences value is a single digest representing the
entire sorted array. Use get_attribute to expand it into the individual
SQ.* sequence digests.
seq_digests = store.get_attribute("sorted_sequences", sorted_seq_digest)print(f"Individual sequence digests ({len(seq_digests)}):")for d in seq_digests: print(f" {d}")Individual sequence digests (3): SQ.18Mvmw4FxlYcghyGZLvLEoY4NRTBknAK SQ.3EzGZ_lkfBucvpzz7eaMUdmdc2qAf8b4 SQ.LBh5K2bNKvmfsWSVKvPbjXFrmIOxxr_bRetrieve sequence metadata
Section titled “Retrieve sequence metadata”For each individual digest, get_sequence returns a record with metadata
including the sequence name, length, and digest.
for seq_digest in seq_digests: record = store.get_sequence(seq_digest) meta = record.metadata print(f"{meta.name}: length={meta.length}, digest={meta.sha512t24u}")chr1: length=32, digest=18Mvmw4FxlYcghyGZLvLEoY4NRTBknAKchrM: length=24, digest=3EzGZ_lkfBucvpzz7eaMUdmdc2qAf8b4chr2: length=32, digest=LBh5K2bNKvmfsWSVKvPbjXFrmIOxxr_bExtract subsequences
Section titled “Extract subsequences”Use get_substring to extract a region from any individual sequence.
Coordinates are 0-indexed with a half-open interval: start is inclusive,
end is exclusive.
# Get the first sequence digestfirst_digest = seq_digests[0]record = store.get_sequence(first_digest)print(f"Sequence: {record.metadata.name} ({record.metadata.length} bp)")
# Extract bases 0-8 (first 8 bases)subseq = store.get_substring(first_digest, 0, 8)print(f"Bases 0-8: {subseq}")
# Extract bases 16-24subseq2 = store.get_substring(first_digest, 16, 24)print(f"Bases 16-24: {subseq2}")Sequence: chr1 (32 bp)Bases 0-8: ACGTACGTBases 16-24: ACGTACGTBatch extraction with a BED file
Section titled “Batch extraction with a BED file”For bulk extraction, substrings_from_regions reads a BED file and returns
all requested subsequences at once. This method requires a collection
digest (not a sorted_sequences digest), because it needs chromosome name
mappings to resolve BED coordinates.
Use find_collections_by_attribute to find a collection that contains these
sequences.
# Find a collection containing this sorted_sequences setcollections = store.find_collections_by_attribute("sorted_sequences", sorted_seq_digest)coll_digest = collections[0]print(f"Using collection: {coll_digest}")
# Create a BED file with regions to extractbed_path = os.path.join(temp_dir, "regions.bed")with open(bed_path, "w") as f: f.write("chr1\t0\t8\n") f.write("chr1\t24\t32\n") f.write("chr2\t4\t12\n") f.write("chrM\t0\t10\n")
# Extract all regionsresults = store.substrings_from_regions(coll_digest, bed_path)for r in results: print(f"{r.chrom_name}:{r.start}-{r.end} {r.sequence}")Using collection: jxN4oDGY6BPur9HRIC2J-B7zNjl3MZEZchr1:0-8 ACGTACGTchr1:24-32 ACGTACGTchr2:4-12 AAAATTTTchrM:0-10 TTTTAAAACCFor more on BED extraction, including writing regions straight to a FASTA file, see Extracting Regions from BED Files in the RefgetStore tutorial.
Related guides
Section titled “Related guides”- Matching by sequences -- Finding collections that share the same sequences
- RefgetStore tutorial -- Creating and loading stores
Cleanup
Section titled “Cleanup”import shutilshutil.rmtree(temp_dir)