Command Reference#

Global Options#

cftk [--version] [--config PATH] <command> ...

--version: Print the installed CFTK package version and exit.
--config PATH: Path to cftk_init.json. Defaults to ./cftk_init.json.

Commands#

init

Create a missing schema-v2 project interactively, or validate an existing schema-v2/legacy config and prepare bwa-meth, samtools, and Picard reference companions.

cftk init

For managed batch setup, pass --non-interactive and --sample-sheet PATH. The default profile installs under CFTK_REFERENCE_ROOT or ~/.cache/cftk/references. Local mode also requires --reference-mode local --reference-root PATH. --profile and --profile-version select a non-default or ambiguous version. The expert CFTK_REFERENCE_REGISTRY override remains available. Pass --skip-reference-prep for validation only.

doctor

Check whether selected core processing and optional downstream-analysis stages can start. The default checks steps 1 through 4. Diagnostics continue after failures and do not download, index, repair, or process data.

cftk --config cftk_init.json doctor
cftk --config cftk_init.json doctor --step 4 --json
cftk --config cftk_init.json doctor --analysis-preset comparative

--step {1,2,3,4} accepts one or more space-separated values. --target-bed PATH validates an expert Picard covered-target override; --skip-picard-metrics removes Picard metrics from the required step-3 tool checks. --parallel N validates a parallel-sample override against the configured total CPU budget and detected scheduler allocation. --json writes only machine-readable JSON to stdout. --analysis-preset or --analysis-stage adds read-only checks for the downstream stage dependencies, inputs, references, roles, and CPU plan. --fragmentomics-scope applies the same assay-aware scope resolution to selected WPS, occupancy, and DELFI checks.

Human checks use PASS, WARN, and FAIL. No required failures returns exit status 0; one or more failures returns 1; invalid arguments return 2.

run

Run the beginner-safe schema-v2 core workflow. The default order is process steps 1-4, then QC steps 2, 0, and 1. FASTQ projects run every stage; BAM projects record process steps 1-2 as skipped. Mixed inputs, legacy configs, missing locks, alternative processing tools, and failed doctor checks stop before computation.

cftk run
cftk run --dry-run
cftk run --parallel 2 --qc-dinucleotide
cftk run --downstream auto
cftk run --downstream differential --modality cpg

By default, run stops after core processing and QC. --downstream explicitly continues into a role-aware downstream preset, reusing valid artifacts and linking the downstream manifest and HTML summary from the core run summary. --downstream all is an advanced option; per-sample job plans and Slurm helpers remain separate advanced commands and are never submitted automatically.

--adopt-existing is required to validate complete outputs that predate a run manifest. Partial untrusted outputs are preserved in a timestamped quarantine before retry. Automatic resume requires a matching config, lock, options hash, exact CFTK software identity, trusted prior stage state, and currently valid artifacts. A code or installed-build change therefore reruns affected stages instead of silently reusing old results. --target-bed PATH is an expert one-run Picard target override. --parallel N sets concurrent samples. CFTK divides the configured total process.cores budget across those sample commands and records the result in resource-plan.json. --fragmentomics-scope is an advanced downstream-only override and must be used together with --downstream; it applies the same panel/genome scope contract as cftk analyze. --modality NAME [NAME ...] selects differential matrices for this run without rewriting the config and requires a downstream preset containing differential analysis.

Every attempt writes run.json, event and command JSONL files, doctor and tool-version JSON, output/figure TSVs, and run-summary.html under results/provenance/runs/<run-id>/. It also generates an evidence/ directory containing stage/artifact/command tables and sanitized figure previews. If analysis completes but evidence reporting fails, the manifest status is complete_with_reporting_error and the command returns a distinct nonzero status; a subsequent run can rebuild evidence without rerunning valid stages. See Beginner Workflow for the full contract.

plan

Resolve a role-aware downstream preset and record its dependencies, resources, expected outputs, and read-only doctor result without launching a stage.

cftk plan
cftk plan --preset all
cftk plan --preset differential --modality cpg occupancy wps
cftk plan --stage diff report --json
cftk plan --stage delfi --execution per-sample --slurm

auto plans occupancy, WPS, and reporting for one group; it additionally plans differential analysis for an explicit two-group control/case project. comparative and all require roles, not group-name inference. Plans are recorded under results/provenance/analysis-plans/. Use --fragmentomics-scope panel|genome only when overriding the assay-aware default. If configured differential or MESA modalities require occupancy or WPS matrices, their producer stages are added ahead of the dependent stage. A differential modality override is recorded in the plan and cannot be combined with --execution per-sample because the comparison is a cohort-level stage.

--execution local is the default read-only plan and the recommended beginner path. Normal execution uses cftk run --parallel N (or one ordinary institutional batch job containing that command). For expensive fragmentomics or core stages, the advanced --execution per-sample writes one independent sample task plus a dependent cohort finalizer. CFTK never submits jobs or requires a scheduler. Fragmentomics sample tasks run the established cftk frag implementation with --parallel 1 --no-finalize; core and QC tasks use their corresponding cftk process/cftk qc boundaries. The finalizer verifies all per-sample outputs, creates cohort matrices/figures where applicable, and records successful completion. --slurm additionally writes an optional Slurm-array helper and success-gated finalizer helper under results/provenance/job-plans/.

Core processing and fragment-length QC can use the same one-sample boundary when a scheduler should own the sample jobs:

cftk plan --workflow core --execution per-sample --slurm
cftk plan --workflow process --stage 3 --execution per-sample --slurm
cftk plan --workflow qc --stage 2 --execution per-sample --slurm

core includes process steps 1-4 and QC step 2. BAM-only projects mark FASTQ-only process steps 1-2 as skipped. QC steps 0, 1, and 3 remain cohort-level because they assemble cohort tables or figures; run them once after the generated sample tasks finish. The generated process step-3 finalizer also requires the covered-target CollectHsMetrics and CollectMultipleMetrics outputs unless --skip-picard-metrics was explicitly used for the sample tasks. CFTK never submits the helper; submit it under the user’s institutional account.

status

Advanced, read-only inspection of a generated per-sample job plan. It reports observed sample artifacts and whether each finalizer is pending, ready, complete, or stale (a recorded finalizer whose artifacts no longer satisfy the plan). It does not infer scheduler queue or failure state.

cftk status
cftk status --workflow core --json
cftk status --plan results/provenance/job-plans/core-*/job-plan.json

analyze

Run downstream stages with fail-fast preflight, artifact contracts, provenance, evidence, and resume behavior. It requires a schema-v2 project and its matching lock file.

cftk analyze --dry-run
cftk analyze --preset differential --modality cpg
cftk analyze --preset fragmentomics
cftk analyze --preset comparative
cftk analyze --preset all

Explicit presets are descriptive, differential, dmr, fragmentomics, mesa, comparative, all, and report. Use --stage for a precise stage or alias such as diff, wps, or report. --fragmentomics-scope controls targeted WPS, occupancy, and DELFI inputs; it defaults to panel scope for Twist. Complete stages recorded by a compatible prior analysis manifest are validated and reused automatically, even when the current preset selects additional stages. The differential and dmr presets also refresh results/report/report.html. DMR stage reuse additionally requires matching SHA-256 signatures for the selected CpG bedGraphs. --modality NAME [NAME ...] overrides only the selected run and is stored in the plan, manifest, and reconstructed command. Differential stage reuse additionally requires matching SHA-256 signatures for all selected input matrices; changing a matrix reruns the comparison while unchanged matrices remain reusable. --adopt-existing remains available for complete outputs produced before any trusted analysis manifest; untrusted partial outputs are quarantined before a retry. See Downstream Workflow.

job-plan

Compatibility alias for cftk plan --execution per-sample. New scripts should use plan so all workflow planning starts from one command.

process

Run raw processing steps 1 through 4. Step 3 uses the schema-v2 profile’s covered-target BED for Picard metrics, with a bundled source-checkout fallback for legacy configs.

cftk --config cftk_init.json process -s 1 2 3 4

Use --target-bed PATH to override the covered targets, or --skip-picard-metrics for a workflow that does not need them.

qc

Run QC steps 0 through 3. Step 0 assembles QC tables, step 1 plots methylation distributions, step 2 measures fragment lengths, and step 3 computes dinucleotide frequencies.

cftk --config cftk_init.json qc -s 1 2 3

power

Run the legacy CpG-level analytical power workflow. It uses the pickled reference_data.cpg_std table from cftk_init.json to evaluate methylation effect-size and sample-size scenarios.

cftk --config cftk_init.json power -s 100 -e 0.1

This command is separate from the model-development power calculator. The latter simulates cross-validated feature selection and classification, performs matched null calibration, and is available through the Python API and Streamlit app described in Model-Development Power.

diff

Advanced compatibility command that runs PCA, differential testing, and visualizations directly. It does not provide managed preflight, matrix-sensitive resume, evidence generation, or immutable analysis-run provenance. Prefer cftk analyze --preset differential for new work.

cftk --config cftk_init.json diff --modality cpg
cftk --config cftk_init.json diff --modality cpg occupancy wps

dmr

Advanced direct DMR compatibility command. It bypasses managed preflight, input-sensitive resume, evidence, and immutable analysis-run provenance. Prefer cftk analyze --preset dmr for new projects; it uses all samples in the role-defined groups unless analysis.dmr.samples selects a subset, records the selection, and refreshes the final report.

cftk --config cftk_init.json dmr

frag

Run fragmentomics workflows. Occupancy and WPS are valid for a one-group descriptive project; comparison figures are produced only when two groups are available. The default Twist profile scopes WPS, occupancy, and DELFI to panel-overlapping reads and regions. Use --fragmentomics-scope genome only for validated whole-genome inputs.

cftk --config cftk_init.json frag --wps
cftk --config cftk_init.json frag --delfi --fragmentomics-scope panel

--sample NAME --no-finalize is the internal/public per-sample task boundary used by plan --execution per-sample. Use --finalize only after every sample task for that stage is successful; it validates the expected per-sample outputs and creates cohort matrices and figures without recalculating the sample measurements.

mesa

Run MESA modality performance, model construction, and LOOCV. It requires explicit control/case roles; CFTK does not infer labels from group names.

cftk --config cftk_init.json mesa --performance --mesa-model --loocv

merge

Build feature matrices from user-specified files in the config merge block.

cftk --config cftk_init.json merge --modality cpg

vis

Regenerate visualizations from existing results.

cftk --config cftk_init.json vis --mode all

report

Generate a self-contained HTML report.

cftk --config cftk_init.json report

run-all

Run the expert compatibility pipeline. It may continue after a failed step and does not provide the validated run-state/resume contract of cftk run.

cftk --config cftk_init.json run-all --parallel 4