The review rules

This page is the repository's own document docs/rules.md, shown as it is, so that it never drifts from the product.

Made by node tools/rules_doc.ts > docs/rules.md from the registry (src/rules.ts, the modules of src/rules/) and the knowledge base (knowledge/dist/knowledge.json): do not edit by hand. A rule runs on every sapc check, build, json and review, in the editor as you type, and in the MCP server. Each finding says what is missing or wrong, the fix in Sapling, and the source (the guideline and its section).

• Severity comes from the knowledge item (or the specify entry of a method page): error (the SAP cannot be executed or is ambiguous for the primary analysis), warning (a guideline expects it), info (good practice). Where the item's check has clauses of different levels (ERROR when ... WARNING when ...), the finding takes the level of its clause. Without a knowledge base the rules still run, as warnings, without sources.
• The primary estimand is the estimand with level="primary"; when no estimand says its level, the first estimand of the plan. Its analysis (analysis="...") is the primary analysis.
• A plan silences a rule only with a reason: ::: {.waive rule="rule-id" block="ID"} with the reason in its body. The finding stays as information with the reason; the PDF and the preview show the waiver. A waiver of method.mmrm covers its keys (method.mmrm/df_method, ...). A waiver of a rule that finds nothing is a warning; of an unknown rule an error.
• The proposal of the knowledge base (docs/knowledge.md, "Lint rules for the next phase") is implemented as the 20 rules of the families below; the two built-in rules of phase 1 stay.

The built-in rules (phase 1)#

What each rule asks, in the words of the knowledge base#

• gamble.16 (16. Level of statistical significance): Every test has its significance level. The knowledge item gamble.16 (knowledge/checklists/gamble-2017.md:406).
• gamble.20 (20. Analysis populations): Every analysis has a population, every population a definition in words, safety analyses a safety population. The knowledge item gamble.20 (knowledge/checklists/gamble-2017.md:527).
• gamble.17 (17. Multiplicity): Two or more confirmatory hypotheses need a multiplicity strategy. The knowledge item gamble.17 (knowledge/checklists/gamble-2017.md:423).
• gamble.18 (18. Confidence intervals): A comparison of treatments reports an estimate with its confidence interval, not a p-value only. The knowledge item gamble.18 (knowledge/checklists/gamble-2017.md:445).
• gamble.28 (28. Missing data): The primary analysis says how it handles missing data, and under which assumption. The knowledge item gamble.28 (knowledge/checklists/gamble-2017.md:839).
• e9r1.attributes (The five attributes of an estimand): An estimand has its five attributes: population, treatment, variable, intercurrent events with strategies, population-level summary. The knowledge item e9r1.attributes (knowledge/guidelines/ich-e9r1.md:18).
• e9r1.strategies (The five strategies): Each intercurrent event has one of the five strategies of ICH E9(R1). The knowledge item e9r1.strategies (knowledge/guidelines/ich-e9r1.md:62).
• e9r1.treatment-policy-terminal (Treatment policy cannot handle terminal events): Treatment policy cannot handle a terminal event (death): the variable does not exist after it. The knowledge item e9r1.treatment-policy-terminal (knowledge/guidelines/ich-e9r1.md:82).
• e9r1.main-estimator (The main estimator is aligned with the estimand): A primary estimand names its main estimator, and the estimator uses the estimand's population. The knowledge item e9r1.main-estimator (knowledge/guidelines/ich-e9r1.md:184).
• practice.estimand-analysis-consistency (The analysis matches the estimand): The records of the main analysis match the strategies of its estimand. The knowledge item practice.estimand-analysis-consistency (knowledge/practice/conventions.md:11).
• practice.as-randomised-as-treated (As randomised for efficacy, as treated for safety): Efficacy by the randomised (planned) treatment, safety by the actual treatment. The knowledge item practice.as-randomised-as-treated (knowledge/practice/data-handling.md:37).
• method.mmrm/covariance_structure (The covariance structure of the within-patient errors (usually unstructured), and whether it is the same in each treatment group.): An MMRM states its covariance structure. The knowledge item method.mmrm (knowledge/methods/mmrm.md:14).
• method.mmrm/df_method (The method for the denominator degrees of freedom (Kenward-Roger is usual; Satterthwaite is an alternative).): An MMRM states its method for the denominator degrees of freedom. The knowledge item method.mmrm (knowledge/methods/mmrm.md:14).
• method.mmrm/fallback_order (The ordered list of structures to use if the unstructured model does not converge (for example heterogeneous Toeplitz, heterogeneous AR(1), Toeplitz, AR(1), compound symmetry) and the convergence criterion.): An MMRM with an unstructured covariance gives the order of structures to try when it does not converge. The knowledge item method.mmrm (knowledge/methods/mmrm.md:14).
• method.cox/ties (The method for tied event times (Efron or Breslow or exact).): A Cox model states how it handles tied event times. The knowledge item method.cox (knowledge/methods/cox.md:13).
• method.km-logrank/censoring_rules (Each censoring reason with its censoring date (last known alive, last adequate assessment, data cut-off, new anticancer therapy).): A time-to-event analysis has the derivation of its censoring. The knowledge item method.km-logrank (knowledge/methods/km-logrank.md:14).
• fda-mult.co-primary (Co-primary endpoints: no alpha adjustment, but less power): Co-primary endpoints (all must be significant) are each tested at the overall alpha, and the sample size gives their joint power. The knowledge item fda-mult.co-primary (knowledge/guidelines/multiplicity.md:41).
• interim.alpha-spending (Efficacy looks: boundaries from an alpha spending function): An interim analysis that may stop for efficacy has boundaries that keep the overall type I error: a spending function or fixed boundaries, the information fraction of each look, and an alpha that its family accounts for. The knowledge item interim.alpha-spending (knowledge/guidelines/interim-adaptive.md:37).
• e9.covariates-subgroups (Covariates, subgroups and interactions): Subgroup analyses test the interaction of the treatment and each factor first (or say why they do not), and are read with care. The knowledge item e9.covariates-subgroups (knowledge/guidelines/ich-e9.md:356).
• e9.multiplicity (Adjustment of significance and confidence levels): The p-values of subgroup analyses (of each level, of the interaction) come with a statement on multiplicity: they are nominal, or a procedure controls them. The knowledge item e9.multiplicity (knowledge/guidelines/ich-e9.md:339).
• ema-subgroups.prespecify (Plan key and exploratory subgroups): A subgroup analysis says whether it is key or exploratory, and its factors have their levels (cut-points or categories) before unblinding. The knowledge item ema-subgroups.prespecify (knowledge/guidelines/subgroups.md:15).
• safety.teae-definition (Definition of treatment-emergent): The plan defines treatment-emergent: the start and the end of the period, and how an event with a missing start date counts. The knowledge item safety.teae-definition (knowledge/safety/adverse-events.md:16).
• safety.ae-by-soc-pt (Adverse events by system organ class and preferred term): An adverse event table by system organ class and preferred term states the order of its rows. The knowledge item safety.ae-by-soc-pt (knowledge/safety/adverse-events.md:74).
• safety.ae-severity-relatedness (Severity, relationship, and events leading to discontinuation): A table by severity or toxicity grade counts a subject with several events of a term once, at the worst; a toxicity grade names its scale and version. The knowledge item safety.ae-severity-relatedness (knowledge/safety/adverse-events.md:92).
• safety.lab-shift (Shift tables): A shift table says which value after baseline it uses: the worst, the last, or each visit. The knowledge item safety.lab-shift (knowledge/safety/labs-vitals-ecg.md:34).
• safety.lab-pcsa (Potentially clinically significant values): Laboratory values counted by a toxicity grade name the grading scale and its version. The knowledge item safety.lab-pcsa (knowledge/safety/labs-vitals-ecg.md:50).
• safety.descriptive (Safety analyses are descriptive): A test in an adverse event table is said to be descriptive (a flag, not a claim), in a footnote of the table or in a multiplicity block that says the other tests are nominal. The knowledge item safety.descriptive (knowledge/safety/exposure-disposition.md:74).
• safety.aesi (Grouped terms and adverse events of special interest): An adverse event of special interest names its definition: a query (an SMQ or a custom query) or its list of terms. The knowledge item safety.aesi (knowledge/safety/adverse-events.md:144).
• missing.single-imputation (Single imputation (LOCF, BOCF) needs a justification): A primary analysis on single-imputed data (LOCF, BOCF, WOCF) justifies it. The knowledge item missing.single-imputation (knowledge/guidelines/missing-data.md:72).
• fda-cov.stratification-factors (Adjust for the stratification factors): The primary model adjusts for the stratification factors of the randomisation. The knowledge item fda-cov.stratification-factors (knowledge/guidelines/covariates.md:35).
• ema-cov.post-randomisation (Do not adjust for post-randomisation variables): A primary model adjusts only for values measured before randomisation. The knowledge item ema-cov.post-randomisation (knowledge/guidelines/covariates.md:53).

Writing a rule#

A family module of src/rules/ exports rules: KRule[]: id (a knowledge item id, or item/key of a method page's specify), describe, condition, fix, and check(c, kb), which returns kb.find(id, span, message, fix, words): the severity and the source come from the knowledge base (words choose the clause of the item's check). Register the module in src/rules.ts, add a test that makes it fire and one that passes (test/rules.test.ts), and run node tools/rules_doc.ts > docs/rules.md. A company's own rules: sapc check --rules module.mjs (the Rule interface of src/rules.ts).