Skip to content

Implementation details

This reference is for developers and administrators checking packaging, installation ownership, recovery, and runtime isolation. For a short explanation of the analysis, read How OncoTracer works.

Stage graph

OncoTracer is an orchestration application, not a reimplementation of BWA, samtools, Picard, qDNAseq, HMMcopy, ichorCNA, or GISTIC2. Each native stage has explicit inputs, outputs, argument-array command, tool environment, validation, trace record, and resume signature. The Python engine coordinates:

  • BWA and Picard for Illumina;
  • minimap2 for ONT;
  • qDNAseq in its pinned R 4.1 environment;
  • HMMcopy and ichorCNA 0.5.1 in their pinned R 4.4 environment;
  • the existing BAM boundary-refinement, CNA notation, and plotting implementations;
  • the optional variant callers, FFPE processing, evidence filtering and ANNOVAR annotation;
  • the optional CNA classifier, GISTIC2, knowledge enrichment, pathology concordance, HTML/PDF reports, and clinician summaries.

The optional classifier is a direct native stage graph over its versioned Python scripts and assets. The bundled GLIMPSE2 workflow includes .nf files in the release payload; the current Dockerfile also supplies its pinned Nextflow runtime. Complete Illumina short-variant analysis uses that workflow, while CNA stages remain native. See the variant migration guide. The separate ONT methylation branch requires additional, user-supplied tools and models.

Single-file executable

scripts/build_native_binary.py creates a compressed Python zipapp containing the CLI and the complete versioned payload (bin, examples, parameter templates, environment definitions, and stable source-input provenance). The build embeds the exact Git commit and deterministic git archive SHA-256. At first use, payload files are staged and published atomically under $XDG_CACHE_HOME/oncotracer/2.1.0/<executable-sha256>/payload. The complete executable SHA-256 gives two different v2.1.0 binaries separate cache roots and locks.

On every reuse, OncoTracer independently derives the expected payload inventory from the executable and verifies each canonical path, file type, normalized mode, size, and SHA-256. Symlinks, special files, missing paths, and extras cannot be accepted as a valid cache. An explicit --root or ONCOTRACER_ROOT containing bin/scripts takes precedence over extraction.

All standalone --dry-run commands use a process-scoped temporary payload and remove it after success, validation failure, or interruption. They do not populate the persistent XDG cache or create installation configuration, result directories, Conda environments, container images, or SIF files.

See the build instructions and standalone installation reference.

Installer ownership boundary

Ownership checks and replacement

Managed Conda/Poetry installations use a strict root marker plus one strict marker for each fixed child. The records bind canonical paths, a random install ID, the environment or lock-file digest, exact file inventory, and OncoTracer source identity. An ownership-checked sibling lock serializes operations.

For each changed child, an authenticated rollback journal records the prior state, the exact-owned old child moves to a same-filesystem backup, and the replacement is created directly at its final canonical prefix. Semantic probes and the inventory run at that final path before commit; unrelated siblings are outside the transaction. Populated unowned paths, marker mismatches, symlinked paths, foreign entries, and active prefixes fail before replacement. A shared lock is held for the complete lifetime of each managed analysis, so the exclusive installer cannot publish beneath a consumer.

All final-prefix and backup publications use no-replacement renames. Metadata updates atomically exchange names and retain the prior bytes under a unique name.

Interrupted installs and retained records

A durable staging journal exists before transaction creation or package-manager execution. If SIGKILL leaves an unsealed final-prefix tree, recovery restores the prior state and moves the unknown tree to a reported sibling preservation path instead of deleting it.

After any failed publication, recovery records the restored target in a durable rolled_back journal and seals the exact transaction inventory before retaining the transaction root, so interruption between root and journal retention resumes.

After commit, the exact inventory-authenticated transaction root is atomically renamed with no replacement to a transaction-ID-bound retained rollback path. The canonical journal is then atomically renamed to a target-bound retained audit record. Recovery accepts either the original transaction name or its exact retained name, verifies the complete sealed inventory, and resumes this order idempotently.

No installer path recursively deletes transaction contents, rollback roots, journals, or temporary metadata; later transactions ignore the retained names. Their exact paths are reported for deliberate administrator archival or removal after active-use checks.

Poetry runtime

Poetry's launcher is a fixed isolated poetry-runtime child built from an exact clean checkout in isolated state and installed only by the final target interpreter; it is never an ambient or checkout-local virtual environment.

Runtime isolation

Native child processes receive invocation-private HOME, XDG, Matplotlib, and Fontconfig state below the analysis output's .oncotracer-native/runtime-cache. OncoTracer audits the selected Fontconfig include graph, exposes one private cache destination, and checks the containment rules before and after each child.

While the shared managed-runtime lock is still held, it also rechecks every managed child inventory and ownership identity after analysis or doctor use. These rules cover commands launched by OncoTracer; invoking a managed prefix's binaries directly is outside that runtime containment boundary.

Validation must also preserve managed environments. Run Python tests with python -B -m unittest; the test package disables bytecode writes in the test process and ordinary Python subprocesses, and native CI applies the same setting to standalone checks. This prevents imports from regenerating sealed NumPy or SciPy caches. Installer inventory checks still reject altered files, including bytecode. If a cache was already changed, restore an original copy only after its bytes match the existing inventory; do not rewrite the inventory to accept the changed file.

Singularity/Apptainer images

A managed SIF has an adjacent strict sidecar binding its canonical path, image reference, bytes, install ID, and source identity. Every reuse verifies the file SHA-256 and executes container doctor and provenance. Replacement pulls into a same-directory owned transaction and verifies the candidate before an atomic, rollback-protected swap. Neither --force path adopts or pre-deletes an unowned destination.

Install dry-runs

Install dry-runs perform the same read-only target checks but create no locks, markers, transactions, environments, images, or saved configuration.

Native invariant

Native CNA execution does not invoke Nextflow. Its managed Conda and container backends include the Java runtime used by Picard for Illumina duplicate marking. The complete Illumina variant branch additionally requires Nextflow and a compatible Java runtime; the current Dockerfile bundles both. CI retains the CNA trace check that excludes Nextflow and separately tests the bundled variant workflow with offline startup as an unprivileged container user.

See release validation for the independent comparison used during release testing.