cis: power by replication of known pairs¶
Question: how many known cis-xQTLs does an analysis setup recover in my data, and which of several setups recovers the most?
A setup is one set of inputs: genotypes, phenotypes (and their normalization), covariates,
the phenotype transform (as given, or RINT with --rint) and the reference of known pairs. Each reference pair (trait, variant) is re-tested in the
study data. The number of pairs that replicate, at several strengths, is a cheap and direct
measure of cis-mapping power. You evaluate each setup in its own run and then compare the
runs, before any genome-wide scan.
For a worked example on the MAGE data, see cis evaluation: comparing analysis setups.
How it works¶
flowchart TB
A["cis run<br/>setup A"] --> C["cis compare"]
B["cis run<br/>setup B"] --> C
D["cis run<br/>setup C"] --> C
C --> T["comparison tables"]
C --> H["report/cis.html"]
| Command | What it does | HTML |
|---|---|---|
anqchor cis run |
tests the reference pairs for one setup and writes the summary tables of that run | no |
anqchor cis compare RUN_DIR... --out DIR |
compares any finished runs, writes comparison tables | yes (--no-report to skip) |
anqchor cis report --out DIR |
re-renders report/cis.html of a comparison or of a single run from its tables (about 1 s, never runs qpgen) |
yes |
anqchor cis all |
cis run and then cis report on the same run directory |
yes |
Runs are independent: start them in parallel, each into its own --out directory.
Runs may differ in any input, not only in covariates. A run tests one phenotype
transform: the phenotypes as given (default), or with --rint a rank-based inverse normal
transform after covariate adjustment. To see what RINT changes, make a second run with
--rint and compare the two; phenotypes that are already inverse-normal transformed need no
RINT run.
A run is described by flags (a configuration file is optional):
anqchor cis run \
--geno-list examples/mage/data/geno/mage.pgen_list.tsv \
--pheno examples/mage/data/pheno/mage_expr.chr21_22.regenie.tsv.gz \
--cov examples/mage/data/pheno/mage_covariates.regenie.tsv.gz \
--cov-include sex,PC1-5,PEER1-10 \
--pairs examples/mage/data/reference/gtex_lcl_eqtl.chr21_22.tsv.gz \
--out results/cis/peer10
| Flag | Meaning |
|---|---|
--geno-list or --pfile |
genotypes: a qpgen pgen list, or one PLINK2 prefix with .pgen/.psam/.pvar.idx.gz |
--pheno, --pheno-format |
phenotype matrix (default format regenie) |
--cov, --cov-format |
covariate matrix. Omit --cov to run without covariates |
--cat-cov-names |
categorical covariates coded as numbers (e.g. a batch number); non-numeric columns such as sex are categorical automatically |
--cov-include |
use only these covariates of --cov: comma-separated names; ranges such as PEER1-10 expand to PEER1 … PEER10. Default: every covariate in --cov |
--pairs |
reference pairs: trait variant beta se [pval] |
--trait-annot |
trait loci (BED-like). Needed only to split the rect engine's controls by chromosome |
--trait-id-rule |
how reference trait IDs map onto phenotype columns (default as_is) |
--rint |
apply a rank-based inverse normal transform (RINT) to the phenotypes after covariate adjustment. Default: the phenotypes as given |
--assoc-engine |
pair (default, pair-assoc) or rect (rect-assoc) |
The same run can be written as a configuration file, for example
examples/mage/cis.yaml
(the all-covariate run): anqchor cis run --config examples/mage/cis.yaml --out results/cis_config.
See Configuration for every key.
Steps of a run¶
With flags only, the dataset is called dataset, the reference reference and the run's
single strategy default (set --dataset-name / --reference-name to change the first
two). Step names below use <ds>, <ref> and <s> for them.
| Step | Kind | What it does |
|---|---|---|
prepare.genotypes |
Python | genotype samples, normalized pgen list |
prepare.reference.<ref> |
Python | reads the reference pairs, checks which variants are in the genotypes |
prepare.dataset.<ds> |
Python | normalizes phenotypes and covariates (categorical covariates such as sex keep their text levels XX/XY; qpgen encodes them) |
prepare.mapping.<ds>.<ref> |
Python | maps reference traits onto phenotype columns |
cis.<ds>.<s>.inputs |
Python | the run's covariate file (all of --cov, the --cov-include subset, or none) and strategy.json |
cis.<ds>.<s>.assoc (pair engine) |
qpgen | one pair-assoc run over the mapped pairs (--skip-rint unless --rint) |
cis.<ds>.<s>.rect.<t> (rect engine) |
qpgen | one rect-assoc run over the reference's present variants × mapped traits; <t> is raw, or rint with --rint |
cis.<ds>.summarize |
Python | replication levels, per-trait table, power curve, summary |
report.cis |
Python | report/cis.html; only with cis all or cis report |
The qpgen command of the all-covariate MAGE run (from the run's Makefile; paths are
relative to the run directory):
qpgentools pair-assoc --pgen-list prepare/genotypes/pgen_list.tsv \
--pheno prepare/datasets/dataset/pheno.regenie.tsv.gz \
--pairs prepare/datasets/dataset/ref.reference/pairs.tsv.gz \
--out cis/dataset/strategies/default/pair_assoc.tsv.gz \
--cov cis/dataset/strategies/default/cov.regenie.tsv.gz --cat-cov-names sex --skip-rint
With --rint the command has no --skip-rint, and the summary reads qpgen's *_RINT
columns. With --assoc-engine rect and --rint:
qpgentools rect-assoc --pgen-list prepare/genotypes/pgen_list.tsv \
--pheno prepare/datasets/dataset/pheno.regenie.tsv.gz \
--var-list prepare/datasets/dataset/ref.reference/variants.txt \
--out cis/dataset/strategies/default/rect_assoc.rint.tsv.gz \
--cov cis/dataset/strategies/default/cov.regenie.tsv.gz \
--pheno-list prepare/datasets/dataset/ref.reference/traits.txt --cat-cov-names sex --rint-after-adj
Without covariates --cov is left out. --cat-cov-names lists the categorical covariates
among the run's covariates: non-numeric columns such as sex plus any named with
--cat-cov-names (for categories coded as numbers, e.g. a batch number). qpgen encodes each
as L − 1 indicator columns. --cov-include sex,PC1-5 passes --cat-cov-names sex;
--cov-include PC1-5,PEER1-10 passes none. anqchor needs qpgen 0.2.0 or later
(qpgen integration).
Model and transform¶
- Phenotypes as given (default): the phenotype and the genotype are both residualized on
the covariates (with an intercept; Frisch-Waugh-Lovell), then regressed on each other.
betais the partial effect of the genotype, as iny ~ 1 + covariates + genotype. --rint: the covariate-adjusted phenotype is rank-based inverse-normal transformed (Blom offsets, average ranks for ties) and tested the same way.pair-assoc's*_RINTcolumns correspond torect-assoc --rint-after-adj. RINT rescales the phenotype, so its effect sizes are not on the reference's scale; compare z-scores or −log10 p instead.- Degrees of freedom:
df = N − 2 − k, where N is the number of samples and k the rank of the covariate matrix;N − 2without covariates.
Engines¶
pair (default) tests exactly the reference pairs. rect tests the whole rectangle of
reference variants × reference traits (346 × 367 = 126,982 cells on MAGE) and keeps the
non-reference cells as local controls: their λ_GC and p < 1e-5 counts appear in
strategies.tsv (*_control* columns), overall and for cross-chromosome cells (these need
--trait-annot). Same-chromosome control cells are in LD with neighbouring eQTLs, so they are
not a null; the cross-chromosome cells are much closer to one.
On MAGE both engines give identical replication counts at every level for the example runs
(with and without --rint), and identical per-pair statistics.
Replication levels¶
Each tested pair gets one 0/1 flag per level. Every level requires sign concordance
(sign(beta_ref) == sign(beta)); alleles agree by construction because the reference
variant ID (CHROM:POS:REF:ALT) is the tested variant and beta is the effect of ALT.
| Level | Default rule | Cut-off on MAGE (367 pairs) |
|---|---|---|
sign |
sign concordance only | none |
nominal |
sign and p < 0.05 | −log10 p > 1.30 |
fdr |
sign and Benjamini-Hochberg q ≤ 0.05 over the run's tested pairs | data-dependent |
bonferroni |
sign and p < 0.05 / n_tested | 0.05/367 → −log10 p > 3.87 |
strict |
sign and p < 1e-5 | −log10 p > 5 |
The levels are configurable in a configuration file (pipelines.cis.levels; null switches
one off).
Level order. Levels are ranked by their actual −log10 p cut-off, not by name: with
367 pairs Bonferroni (p < 1.36e-4) is weaker than strict, but with tens of thousands of
pairs it is stronger. summary.json level_order records the order used.
Status. Every mapped pair has a status: tested, variant_absent (variant not in the
genotypes) or not_tested (present but no qpgen result or no finite statistic, for example a
variant with no variation among the phenotyped samples). The summary step fails when more
than max_unresolved_fraction (20%) of the mapped pairs are unresolved, because the result
would then say more about variant matching than about power. The error names absent
examples; a different genome build, chr21 vs 21, swapped REF/ALT or a different indel
representation are the usual causes.
Recommendation (comparison). The run with the most pairs at the strongest level (by
default strict), ties broken by the next weaker levels, then by order. summary.json and
the report state the reason in words.
z-scores and effective sample size. Every tested pair has a signed z = beta / se
and the reference's z_ref = beta_ref / se_ref; both refer to the same allele. For a
pair, E[z²] ≈ 1 + N·β²·Var(g)/σ²ₑ (the noncentrality), with σ²ₑ the residual variance, so
on the same pairs (mean z² − 1) is proportional to the effective sample size: removing
residual variance with covariates shrinks se and raises z² as more samples would. (The
normal quantile of the p-value would understate strong pairs, whose p-values come from a t
distribution.) rel_neff in strategies.tsv is (mean z² − 1) / (baseline mean z² − 1)
over the pairs tested in every run, the baseline being the first run. It is a ratio of
means, so pairs count in proportion to their z²: the strongest pairs dominate (in the MAGE
example the top 10 of 367 pairs carry about a third of Σz², and the median per-pair ratio
is larger than rel_neff for the stronger runs). The reference pairs are selected known
signals, not a random set of variants, so it is a relative diagnostic, not a sample
size. cis compare re-derives z from each run's beta and
se, so every compared run uses the same definition.
π1 is Storey's 1 − π0 with a single λ = 0.5: approximate, only indicative for a few hundred pairs.
Outputs of a run¶
results/cis/peer10/
├── cis/dataset/
│ ├── strategies.tsv one row: counts per level, rint, mean z², ...
│ ├── pairs.long.tsv.gz one row per reference pair (beta, z, −log10 p, reference values)
│ ├── traits.tsv one row per trait
│ ├── power_curve.tsv replicated count vs −log10 p threshold
│ ├── summary.json everything above, levels, recommendation, warnings
│ └── strategies/default/ cov.regenie.tsv.gz, strategy.json, pair_assoc.tsv.gz
├── prepare/ normalized inputs
├── logs/<step>.log output of each step
├── steps/ step stamps and run times (<step>.seconds)
├── anqchor.resolved.yaml the complete settings of the run, flags included
└── run.json provenance: anqchor version, qpgen binary and commit, commands, host
A run writes no HTML; anqchor cis report --out results/cis/peer10 renders
report/cis.html for it. Rows are labelled (run column) with the run directory name, here
peer10. Column definitions: Output files.
pval_ref is the reference's pval column (NA when the file has none; another column
name needs columns: {pval: NAME} in a configuration file).
Reproducing a run. anqchor.resolved.yaml holds every setting, including the flags:
gives identical tables (labelled peer10_again).
Outputs of a comparison¶
anqchor cis compare RUN_DIR... --out DIR [--label L ...] [--no-report] reads finished runs
(it never runs qpgen; about 1 s). Each run gets a label: the run directory name, or the
--label values in order (one per run). The comparison directory holds:
| File | Content |
|---|---|
runs.tsv |
one row per run: label, run directory, genotypes and sample count, phenotypes and sample counts, covariate file, cov_include (all when every covariate was used), n_cov, covariate columns, rint, reference, engine, mapped pairs |
strategies.tsv |
one row per run (run = label = the run label): the run's own counts (n_tested, n_rep_<level>, frac_rep_<level>, median_log10p, mean_z2, beta_corr, pi1, ...), plus n_common_tested, n_rep_<level>_common, mean_z2_common and rel_neff (effective sample size relative to the first run) |
pairs.long.tsv.gz |
the runs' long tables stacked (run = label), with levels re-derived using the first run's level settings |
traits.tsv |
one row per trait, best_log10p:/level:/delta_log10p:<run> (delta against the first run), best_run |
power_curve.tsv |
per run: replicated pairs at −log10 p thresholds 0, 0.5, 1, … |
summary.json |
as for a run, plus runs, comparability (same_reference, a hash of each run's reference, rint per run) and recommended |
compare.json |
manifest of the comparison (runs, labels, command); cis report uses it to recognise a comparison directory |
settings/<label>.anqchor.yaml |
the exact settings of each compared run (a copy of its anqchor.resolved.yaml) |
report/cis.html |
the report (unless --no-report) |
Common pairs. Runs can test different pairs (a different reference, or a genotype set
missing some variants). n_common_tested is the number of pairs (matched by reference trait
and variant) tested in every compared run, and n_rep_<level>_common counts replication
over those pairs only, with FDR and Bonferroni recomputed on them. When the runs tested the
same pairs, the _common columns equal the plain ones. Runs may differ in the phenotype
transform: each run's rows carry its own statistics, and runs.tsv records rint. anqchor warns when the runs used
different reference files or tested different pairs; then compare the _common columns.
The recommendation and the report's charts use each run's own counts.
Reading the report¶
The same page shows one run or a comparison (title Comparing N runs); the first run of a comparison is the baseline. From top to bottom:
- Replicated pairs by p-value threshold: the power curve. Pairs replicated (same sign, p < 10^−t, as the levels) against t on a log(1 + t) axis labelled in −log10 p, one line per run; dotted lines at the nominal and strict cut-offs and at the number of tested pairs. Count or percentage.
- Signal strength: for a comparison first the effective sample size relative to the
baseline (
rel_neff, a dot per run on a log axis around ×1, with a 95% paired bootstrap interval), then |z| or −log10 p per run as boxes (middle half, 5th–95th percentile, median) with the reference study as a hollow box, all over the pairs tested in every run. - Significant hits: per run (rows), the pairs of the same sign, at the nominal and at the strict cut-off, with percentages and bars. FDR and Bonferroni counts are in the run details.
- Agreement with the reference: for one run at a time, effect size, z and −log10 p against the reference (each axis on its own scale; a dashed least-squares line through 0 with its slope for effect size and z), and (comparison) z against the baseline run with the line y = x. Open diamonds mark pairs with the opposite sign to the reference (against the baseline: pairs where the two runs disagree in sign).
- All pairs: −log10 p in the reference and in each run, with the number of runs of opposite sign; a run's value is red with a minus sign where its effect has the opposite sign (100 rows a page, sortable, searchable, header and pair column pinned).
- Run details (collapsed): covariates, RINT, samples, FDR and Bonferroni counts and
saved settings per run; the FDR and Bonferroni rules; local controls (rect engine);
untested pairs; a link to
strategies.tsvwith every per-run number.
Interpretation¶
- Compare the
strictandbonferronicounts.signandnominalsaturate when the reference eQTLs are strong, so they show that the data are sound but cannot rank setups. - A difference of a few pairs out of hundreds is within noise; prefer the simpler setup unless the power curves separate over a wide range of thresholds.
- More replications is not always better: a setup that ignores population structure inflates every statistic and also "replicates" more. Check calibration with the rect engine's cross-chromosome controls.
- RINT matters little for expression that is already normalized and more for skewed
phenotypes (proteins, metabolites). Test it as a separate
--rintrun. - A low
beta_corror many sign-discordant pairs with an otherwise good setup point to a trait-mapping problem or a reference from a different tissue or population. - If the reference was discovered in the same samples with the same covariates, the setup that reproduces the discovery model is favoured by construction. Use an independent reference (for example GTEx for the same tissue, as in the MAGE example) for real decisions.
Several strategies in one configuration file¶
A configuration file may still list several entries under pipelines.cis.strategies; they
are then evaluated in one run directory, and cis compare labels them <label>:<strategy>.
Separate runs are simpler and can differ in any input, so this page and the example use one
setup per run.