hg38 reference indexes¶

A prebuilt index avoids the one-time genome-index construction step. It does not remove RAM needed for alignment, sorting, CNA analysis or optional models. Start with oncotracer system --path /absolute/path/project to see planning limits.

Install the OncoTracer command before using these commands.

Choose how to prepare hg38¶

All setup examples, batch setup, and public tutorials accept these options:

Setup option	What happens when you run
`--hg38_build /data/shared-reference`	Reuse that prepared OncoTracer reference
`--hg38_build` with no path	Download prebuilt indexes into `PROJECT/reference`
Neither reference option supplied	Wizard asks; default and scripted behavior are automatic download
`--build_reference`	Build missing indexes locally on CPU under `PROJECT/reference`

Choose only one option. For --hg38_build, supply the reference parent or the actual build folder (including OncoTracer's own local cache); not a FASTA, .mmi file, or shell script. BWA indexes are required for Illumina, minimap2 for ONT. A build containing both can be shared by both platforms.

The table shows setup destinations. With auto, the default reference folder is CONFIG_DIR/reference/ instead. The saved lpwgs_root shows the exact path. A hand-written YAML defaults to local preparation unless you explicitly set hg38_auto_download: true; setup and auto save that download choice for you.

Setup saves lpwgs_root and hg38_auto_download in YAML. Choose save at the wizard's final question to finish without downloads or analysis. Manual/scripted setup without --run, check and dry-run also download and build nothing. Choosing run or passing --run starts analysis. With the default reference choice, run downloads only the needed platform's bundle, checks the pinned manifest and every file, and then starts analysis. Existing references are validated and reused, never overwritten by automatic download. Transient network failures retry up to five times. Verified chunks survive a failed run; repeat the same command to reuse them. The interrupted chunk restarts from its beginning. Download progress includes percentage, speed and estimated time remaining. A failed download stops the run; it does not fall back to building indexes.

New setup projects share a .oncotracer-reference-downloads cache beside their project folders. Override it with setup --reference-cache PATH. Existing YAML can set reference_download_cache: /absolute/path/hg38-downloads; otherwise its reference parent holds the cache. Both platforms reuse the shared genome chunks.

The manual commands below are useful for offline transfers or preparing a shared bundle containing both index types before any analysis.

Build your own indexes¶

Replace --hg38_build with --build_reference in any setup command. Then use the same check and run commands as usual. Run downloads OncoTracer's pinned hg38 source files if needed and constructs the indexes required by your analysis: BWA for Illumina, minimap2 for ONT CNA. It uses CPU only. Methylation-only runs prepare the genome without the CNA mapping index.

Local construction needs more RAM, temporary disk and time than importing prebuilt indexes. For limited-RAM computers, keep the default download option. Completed local indexes are validated and reused on later runs; this flag does not force a rebuild or overwrite a shared reference. To reuse them in another project, supply --hg38_build /absolute/path/first-project/reference.

Download the ready-made indexes¶

The hg38 reference release contains the public genome and indexes, not patient data or classifier models. Choose a new reference parent and preview the installation:

oncotracer reference install \
  --lpwgs-root /absolute/path/shared-reference \
  --mode ont --dry-run

Replace /absolute/path/shared-reference with your chosen folder. To install the ONT bundle after checking the preview:

oncotracer reference install \
  --lpwgs-root /absolute/path/shared-reference \
  --mode ont

There is no need to download individual parts manually.

Flag	Meaning
`--manifest`	Small JSON inventory URL, not the FASTA or reads
`--sha256`	Required for a custom remote inventory; the default bundle is already pinned
`--cache`	Shared verified chunk cache for manual imports
`--lpwgs-root`	Parent where reusable references will live
`--mode ont`	Genome and minimap2 index: 9.7 GiB
`--mode illumina`	Genome and BWA indexes: 8.0 GiB
`--mode both`	Genome and both index sets: 14.8 GiB
`--dry-run`	Show required space and destination without transferring genome files

Allow up to twice the listed size for the installed reference and retained download chunks, plus 1 GiB headroom. Local bundle imports need only the installed size plus headroom at the destination. When setting up a project, pass --hg38_build /absolute/path/shared-reference. This is the same path supplied to reference install --lpwgs-root, not the FASTA or index file. For a project already configured, set that parent in its YAML:

lpwgs_root: /absolute/path/shared-reference
hg38_auto_download: false

The installed files are under shared-reference/references/samurai_hg38/. Existing reference directories are never overwritten or silently adopted. If you need both platforms, select --mode both at the first install; otherwise use a new parent for a different bundle. Retained chunks are verified again before reuse. TLS certificate checks and all manifest, chunk and complete-file checks stay enabled.

If this validated reference directory already exists, skip the download and use its path with setup --hg38_build. No separate genome-build script is required.

The default manifest has SHA-256 1c704b1522fe37c13bf4272858792d03e827bc476fdb6712fa7f760f3c944632. Custom HTTPS manifests require their publisher’s trusted --sha256.

RAM and compatibility¶

Import uses a 1-MiB transfer buffer and checks every chunk and complete file before making the reference available. No index is built or loaded into RAM during import. Alignment still needs more memory; this is not a way to run the whole analysis with 1 GiB RAM.

This bundle requires Linux x86-64 Bioconda BWA 0.7.19=h577a1d6_1 and minimap2 2.30=h577a1d6_0 for their respective platforms. Normal analysis verifies tool identities before using the indexes. A different build stops with an error rather than rebuilding the shared reference. Use a compatible bundle; do not manually replace tools inside a managed OncoTracer installation.

Copy a bundle between computers¶

Copy hg38-reference.json and all its .part files into one folder. Use that local JSON path for --manifest. For a copied bundle containing both index sets:

oncotracer reference install \
  --lpwgs-root /absolute/path/offline-reference \
  --manifest /absolute/path/copied-bundle/hg38-reference.json \
  --mode both

Installation uses the copied files without an internet connection. Choose ont or illumina for a single-platform bundle. Keep the files until import succeeds.

Prepare a bundle on a larger computer¶

Use an already validated reference with both BWA and minimap2 indexes:

oncotracer reference export \
  --reference /absolute/path/reference/references/samurai_hg38 \
  --core-prefix /absolute/path/envs/core \
  --output /absolute/path/new-hg38-bundle

--core-prefix contains the exact BWA/minimap2 tools that built the indexes. Omit it only when those tools are on PATH. Export checks the pinned genome, file hashes, tool identities and index readability. It writes chunks smaller than 2 GiB plus the manifest; the output directory must not exist.

An arbitrary folder labeled “hg38” is not a validated bundle: genome sequence, contig names, index settings and tool identities must agree with OncoTracer.