CRAM → FASTQ: efficiency & cost

All 4 approaches convert the same CRAM inputs to paired FASTQ (12 array tasks each) — so every difference below is pure overhead, not different work.

CPU efficiency at a glance

CPU time used ÷ (cores × wall time). Low values mean allocated cores sat idle; ~1/N means a single-threaded job that asked for N cores.

Summary

What the numbers show

• Approach A requests 8 CPUs but reaches only 12% CPU efficiency — effectively 1.0 core(s). It runs single-threaded, so the other allocated cores sit idle while it also pins ~90 GB RAM for the whole run.
• Approach B reaches 68% efficiency across 6 CPUs (~4.1 cores) with 6 GB, finishing in 00:05:28 (mean) at a cost of 0.071.
• Approach C reaches 93% efficiency across 3 CPUs (~2.8 cores) with 3 GB, finishing in 00:07:40 (mean) at a cost of 0.050.
• Approach D reaches 89% efficiency across 4 CPUs (~3.5 cores) with 4 GB, finishing in 00:05:58 (mean) at a cost of 0.051.
• The leanest approach, Approach C, costs 100x less than Approach A (0.050 vs 4.96) for identical output, using 33x less memory and 18x less wall time. Because the gap is per-sample, it grows in absolute terms with bigger datasets.

Cost vs speed

Same output, different resource shapes. Core-hours per sample = cores × wall time; lower packs more work into a fixed cluster allocation.

• Approach B has the fastest per-sample turnaround (00:05:28) — prefer it when single-sample latency matters (e.g. one urgent sample).
• Approach C is the cheapest per sample (0.004) and packs the most work into a core budget (261 samples per 100 core-hours) — best for large batches where total cost dominates.
• Approach D is the best all-rounder: within ~4% of Approach C's cost but ~22% faster (00:05:58 vs 00:07:40).
• See Notes for how each approach threads samtools fastq.

Notes

• Approach A runs samtools fastq without -@ (single-threaded), so the 8 requested cores cannot be used — hence ~1/8 CPU efficiency.
• Approach B threads samtools fastq with -@ (SLURM_CPUS_PER_TASK - 1): one worker per allocated core, minus one for overhead.
• Approach C overprovisions samtools fastq threads with -@ (SLURM_CPUS_PER_TASK * 2 - 1), about twice the allocated cores, so the CPUs stay busy through I/O waits — this is why efficiency stays high at a lower core count.
• Approach D overprovisions samtools fastq threads with -@ (SLURM_CPUS_PER_TASK * 2 - 1), about twice the allocated cores, so the CPUs stay busy through I/O waits — this is why efficiency stays high at a lower core count.

Scaling projection

Per-sample cost and wasted compute are essentially fixed (each CRAM is similar), so the differences scale linearly with cohort size. Each approach column is projected total cost; Idle CPU-hours are Approach A's.

• A 1,000-sample cohort would cost ~413 with Approach A versus ~4 with the leanest Approach C (~100x more, ~409 wasted), while Approach A alone burns ~15,823 CPU-hours on idle cores that never do any work.
• A 10,000-sample cohort would cost ~4,134 with Approach A versus ~41 with the leanest Approach C (~100x more, ~4,093 wasted), while Approach A alone burns ~158,232 CPU-hours on idle cores that never do any work.

Per-task detail