This page is the repository's own document docs/markdown.md, shown as it is, so that it never drifts from the product.
A statistical analysis plan (SAP) in Sapling is a Quarto Markdown file (sap.qmd). Quarto and Pandoc read it as
ordinary Markdown: front matter in YAML, headings, fenced divs, bracketed spans, $...$ math and HTML comments. About
thirty kinds of block give the plan its meaning: a population, a test, a table, a model, an estimand, a citation of the
protocol. There are no LaTeX commands in a plan (only the LaTeX math inside $...$ and $$...$$, as Markdown has it).
This page is the reference of the language: how each block is written, what it means, what the compiler checks, and what it makes. To start, read the one page quickstart.md: from a protocol PDF to a first checked part. The same file goes two ways:
| Command | Product | For |
|---|---|---|
sapling sap pdf, sapling sap html (Quarto with the extension of Sapling) | the typeset SAP: each block as a box with its kind, id and attributes, each citation with the protocol's words, the open decisions marked (The Quarto extension) | people |
node src/cli.ts build plan.qmd | out/sap.md: the structured SAP (sas-macro-agentic/docs/structured_sap.md), from which the chain makes the analysis contract (CDISC ARS), the shells, the QC specification, the ADaM specification draft and the programs; out/sap.json (the plan as data for agents); out/sap.html (a live preview) | the chain, agents, the preview |
The compiler reads a plan (a .qmd or .md file) with the Markdown reader src/qmd.ts. The messages of the compiler
name the blocks in their Markdown form: a fix shows the Markdown to write.
| Command | Result |
|---|---|
node src/cli.ts check plan.qmd (and build, json, review, ...) | the diagnostics, the products (What the compiler checks) |
node tools/qmd_doc.ts | writes the table of commands at the end of this page from src/commands.ts |
node tools/quarto_check.ts [plan.qmd ...] | Pandoc and Quarto on the plans (the example plans by default): QUARTO CHECK PASS (Pandoc and Quarto) |
sapling new <study> --protocol usdm.json --sdtm DIR | a study folder whose plan is sap.qmd (A study folder) |
sapling sap pdf, sapling sap html | the plan typeset by Quarto with the extension of Sapling: out/sap/sap.pdf (Typst), out/sap/quarto/sap.html (The Quarto extension) |
node src/cli.ts pdf plan.qmd [--out DIR] | the same PDF for one plan, without a study folder: wrote out/plan.pdf (DIR/plan.pdf), the copy Quarto renders in DIR/quarto/ |
zsh tools/pilot_qmd.sh | the CDISC pilot from sapling new to the dashboard's gate with sap.qmd, compared with the stored digest of a proven run (tools/pilot_digest.json): PILOT QMD PASS |
Contents. Write: Ask the agent · A small plan · The rules of the form · The front matter · Sections · Blocks: fenced divs · Parts and inline commands: spans · Attributes · Citations · Prose · Models · Comments · Macros · What the reader also accepts · The exceptions. Meaning: Values and conditions · The settings · Citations: the protocol is the source · Populations, subsets, groupings · Tests and statistics · Tables and analyses · Models: a formula or an equation · Estimands · Decisions · The plan around the analyses · Adverse event tables · The options of a model · The review and waivers · The SAP library · The kinds of blocks · What the compiler checks · The ADaM from the SAP · Start a plan · The machine view · Reference card. Tools: Pandoc and Quarto · A study folder · The Quarto extension · All commands
You never need to remember the syntax: ask the agent#
Say what you want in your own words. The agent writes the block, compiles, and shows you the sentence and the table. These ten blocks are 90% of a SAP:
| You say | The agent writes |
|---|---|
| "The efficacy population is the patients with EFFFL = Y." | ::: {.population #EFF condition="ADSL.EFFFL"} with the name Efficacy population in its body |
| "That comes from protocol section 4.1." | []{.cite source="protocol:4.1"} |
| "Compare the groups by ANOVA, as section 4.2 says." | [anova]{.test}[]{.cite source="protocol:4.2" words="The groups will be compared"} in the sentence |
| "A demographics table in the enrolled patients, with a Total column." | ::: {.tlf #T_DEMOG population="ENR" total="true"}, the title Demographics in bold, then its rows |
| "Age: n, mean, SD, median, min, max, with the ANOVA." | ::: {.summarize #AGE variable="ADSL.AGE" test="anova"} with the title Age (years) |
| "Sex: counts and percents, chi-square." | ::: {.tabulate #SEX variable="ADSL.SEX" test="chisq"} with the title Sex |
| "Adverse events by SOC and PT." | ::: {.incidence #AE condition='ADAE.TRTEMFL = "Y"' test="chisq"} with the title Treatment-emergent adverse events |
| "Treatment-emergent: from the first dose to 30 days after the last; a missing start date counts." | ::: {.treatmentemergent #TEAE end="TRTEDT + 30" missing="emergent"} |
| "The terms in 5% or more of a group, most frequent first, with the risk difference and its CI." | ::: {.incidence #AE_RD condition='ADAE.TRTEMFL = "Y"' rows="pt" threshold="5" order="frequency" test="riskdiff" ci="newcombe" pvalue="false"} |
| "By maximum severity: each patient once, at the worst." | ::: {.incidence #AE_SEV condition='ADAE.TRTEMFL = "Y"' worst="SEV" order="alphabetical"}, with ::: {.grouping #SEV variable="ADAE.AESEV" groups="MILD; MODERATE; SEVERE"} |
| "Vital signs by visit, observed and change." | ::: {.byvisit #VS dataset="ADVS" timepoint="true"} with the title Vital signs |
| "ANCOVA of the change with baseline and pooled site, linear dose response." | []{.formula formula="CHG ~ BASE + SITEGR1 + TRTPN"} (in an analysis) |
| "The protocol leaves the pooling rule to me; note it." | ::: {.decision} with Confirm that SITEGR1 is the pooling rule., and []{.cite source="protocol:4.3.3" words="For all ANOVA and ANCOVA models"} after it |
And for the rest of a SAP, which the review asks for (sapc review):
| You say | The agent writes |
|---|---|
| "Parallel, 1:1:1, double-blind, stratified by site." | ::: {.design type="parallel" allocation="1:1:1" blinding="double" strata="ADSL.SITEGR1"} |
| "300 patients: 90% power for a difference of 3 with SD 6.5." | ::: {.samplesize #SS1 n="300" power="0.90" effect="3.0" sd="6.5" test="ttest"} and []{.cite source="protocol:3.4.4"} after it |
| "Both primary endpoints must be significant." | ::: {.multiplicity #MULT procedure="co-primary"} with [...]{.hypothesis #H1 analysis="P_ADAS"} and [...]{.hypothesis #H2 analysis="EST2"} |
| "Missing data under MAR, handled by the MMRM, because ..." | ::: {.missingdata #MD1 analysis="P_ADAS" assumption="MAR" handling="mmrm"} with the reason in its body |
| "A tipping point analysis under MNAR." | ::: {.sensitivity #S1 estimand="EST1" method="tipping-point" assumption="MNAR" seed="20261109"} with the title Tipping point |
| "Unstructured covariance, Kenward-Roger." | []{.formula formula="CHG ~ BASE + TRTP*AVISIT" covariance="us" fallback="toeph; ar1" df="kenward-roger"} |
| "We analyse in R 4.4.1; AEs coded in MedDRA 27.1." | ::: {.software name="R" version="4.4.1"} and ::: {.dictionary name="MedDRA" version="27.1"} |
| "The protocol is right here; ignore that rule, because ..." | ::: {.waive rule="gamble.18" block="P_ADAS"} with the reason in its body (you write the reason; the PDF shows it) |
The agent never writes approved: in the front matter, status: APPROVED, or by= and on= on a decision: only you
approve a plan or decide an open decision.
A small plan#
---
title: Statistical Analysis Plan
date: 11 November 2026
toc: true
sapling:
study: CDISCPILOT01
version: 0.1 (draft)
protocol:
file: protocol/protocol.json
name: Protocol H2Q-MC-LZZT(c)
usdm: protocol/CDISC_Pilot_Study_v4.json
adam: adam
status: DRAFT
drafted-by: agent
alpha-level:
level: 0.05
sided: 2
cite:
source: usdm:objectives
words: To determine if ... erroralpha=0.05)
treatments:
groups: [Placebo, Xanomeline Low Dose, Xanomeline High Dose]
variable: ADSL.TRT01A
cite: usdm:arms
---
<!-- drafted by: agent (docs/markdown.md); the statistician reviews it -->
# Analysis populations []{.cite source="protocol:4.1"}
::: {.population #EFF condition="ADSL.EFFFL" definition="Patients with a baseline and at least one postrandomization measurement"}
Efficacy population
:::
# The primary estimand
::: {.estimand #EST1 level="primary" population="EFF" analysis="P_ADAS"}
**Group mean changes from baseline in the primary efficacy parameters**
[Xanomeline TTS at the low dose (54 mg) or the high dose (81 mg), or placebo, once daily for 24 weeks]{.treatment}
[Change from baseline to Week 24 in the ADAS-Cog (11) total score]{.endpoint}
[Temporary treatment interruption]{.intercurrent strategy="treatment policy"}
[]{.cite source="usdm:estimands" words="Estimand EST1 ... regardless of interruption."}
:::
# Efficacy []{.cite source="protocol:4.3.3" words="Effects in the ANCOVA model"}
The change from baseline is analysed by analysis of covariance with the baseline score, the pooled site and the dose
$d_i \in \{0, 54, 81\}$ as effects ([ADQSADAS.CHG]{.var}).
::::: {.tlf #P_EFF population="EFF"}
**Primary Endpoint Analysis: ADAS-Cog (11), Change from Baseline to Week 24**
:::: {.analysis #P_ADAS}
**ADAS-Cog (11), change from baseline to Week 24**
[]{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "Week 24" and ANL01FL = "Y"'}
::: {.model}
$$
Y_{i} = \mu + \beta\,\mathrm{BASE}_{i} + \gamma_{s(i)} + \tau\, d_{i} + \varepsilon_{i}
$$
[]{.where bindings="Y = CHG, s = SITEGR1, d = TRTPN"}
:::
::::
^[The test for linear dose response: the treatment as a number (0, 54, 81 mg).]
:::::
::: {.decision}
The protocol's investigator effect is the pooled site SITEGR1 of the ADaM data. The statistician confirms
that its pooling is this rule.
:::
[]{.cite source="protocol:4.3.3" words="For all ANOVA and ANCOVA models"}
The plans of the repository are in examples/: examples/cdiscpilot01/pilot.qmd (the CDISC pilot study),
pilot-full.qmd (every analysis of its study report), pilot-adam.qmd (the ADaM from the SAP), examples/customize/
(the programmer's hooks) and examples/errors/typos.qmd (the mistakes people make).
The rules#
One rule for each role of a command (src/commands.ts); the forms are in src/qmd_forms.ts:
| Role | Markdown form |
|---|---|
| preamble (a setting) | a key of the front matter: title, subtitle, date, author at the top (Quarto's own keys); every other setting under sapling: |
| heading | a Markdown heading: # section, ## subsection, ### subsubsection, #### paragraph |
| block, and every environment | a fenced div: ::: {.name #ID key="value" ...}, the body, ::: |
| part | a bracketed span on a line of its own: [prose]{.name key="value"} |
| inline | a bracketed span in the text: [first argument]{.name key="value"}; some have a form of Markdown itself (below) |
| prose (lists, quotes, equations) | Markdown: - item, 1. item, definition lists, > quote, $$...$$ |
The arguments of a command. The argument named id is the identifier #ID. The prose argument (an argument a
person reads as words: title, name, text, task, reason, justification, what, event) is the body of the
div or the content of the span. Every other argument and every option is an attribute with its name in
src/commands.ts. name as the first argument names the block (dataset, software): it is an attribute
(name="ADSL").
Two prose parts in one div. An environment has a title and a body; change has two prose arguments. Then the first
is the first paragraph of the div, all in bold (**...**), and the second follows:
::: {.change}
**What the SAP does differently**
Why it does so.
:::
The front matter#
---
title: Statistical Analysis Plan # Quarto's keys: title, subtitle, date, author
toc: true # the table of contents
sapling: # every other setting
study: CDISCPILOT01 # one argument, no option: a scalar
protocol: # options or more arguments: a map with the names of commands.ts
file: protocol/protocol.json
name: Protocol H2Q-MC-LZZT(c)
alpha-level: {level: 0.05, sided: 2} # a flow map is the same
treatments:
groups: [Placebo, Xanomeline Low Dose, Xanomeline High Dose] # a Sapling list (a; b; c) is a YAML list
variable: ADSL.TRT01A
cite: usdm:arms # the citation of the setting: a source ...
visits:
visits: [Baseline, Week 2, Week 4]
cite: {source: "usdm:encounters", words: "the words of the sentence"} # ... or a source and its words
---
- A setting with one argument and no option is a scalar. With options, more arguments or a citation it is a map whose
keys are the names of its arguments and options. A setting written more than once (
approved, one per role) is a list of maps. - The keys of two settings are in kebab case:
drafted-byandalpha-level. toc: truegives the table of contents. Quarto shows the title block when there is atitle:. Other keys of Quarto (format:,lang:,number-sections:, ...) are Quarto's: the compiler does not read them.- The reader reads a subset of YAML: block maps and lists by indentation, plain, single-quoted and double-quoted
values, flow maps
{a: b}and flow lists[a, b], comments. It does not read anchors, tags or block scalars (|,>): an error says so, at the line and column. A value with:or#, or one that starts with a character YAML reserves, goes in quotes; the writer uses single quotes, in which a backslash is a backslash.
Sections#
# Introduction []{.cite source="protocol:3.1"}
## Demographics []{.cite source="protocol:4.2" words="The groups will be compared"}
#### Vital signs
# Unnumbered section {.unnumbered}
# Table shells {.appendix}
The citation of a section is a cite span at the end of its heading line. {.unnumbered} (or {-}) makes a heading
without a number. A level-1 heading with {.appendix} starts the appendix: its sections are
numbered A, B, ...; the writer puts {.appendix} on every section of the appendix, as Quarto expects.
Blocks: fenced divs#
::: {.population #ENR condition="ADSL.ITTFL" definition="All patients enrolled (randomized) in the study"}
All enrolled patients
:::
[]{.cite source="protocol:3.10.2"}
::: {.software name="R" version="4.4.1"}
:::
- A div opens with three or more colons and its attributes, and closes with a line of colons only. A closing line
closes the innermost open div (Pandoc's rule). The writer gives an outer div more colons (
::::,:::::) so that the eye sees the nesting. - A block without a prose argument has an empty div.
- Pandoc needs an empty line between a paragraph and a div (or a heading) after it. Without it Pandoc reads the fence
as text; the reader of Sapling reports an error at that line (
a div needs an empty line before it) and reads the div.
Parts and inline commands: spans#
[Discontinuation of study drug]{.intercurrent strategy="treatment policy" reason="the patient stops the patch"}
[]{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "Week 24"'}
[]{.formula formula="CHG ~ BASE + SITEGR1 + TRTPN" covariance="us"}
[Word Recall Task]{.param paramcd="ACITM01" n="1"}
The groups are compared by [anova]{.test} and [ADSL.AGE]{.var} is described in table [P_DEMOG]{.ref}.
- A part is a span on a line of its own. Its content is its prose argument, or empty.
- An inline command is a span in the text. Its content is its prose argument, or else its first argument
(
[ADSL.AGE]{.var},[P_DEMOG]{.ref},[anova]{.test}). - Parts on lines that follow each other make one Markdown paragraph; the compiler reads each part on its own.
Attributes#
key="value", always in quotes in the plans the writer makes. A value with a double quote is written in single quotes (records='PARAMCD = "ACTOT"'); a value with both is in double quotes with\".- A value is the text as written: a list is separated by
;(fallback="toeph; ar1h"), a flag is"true"(total="true"). #IDis the argumentid; when a command has no argumentidand has an optionid(test, a model),#IDis the option.id="ID"is the same as#ID(Pandoc's rule).- The attributes may go over more lines inside the braces (no empty line), of a div and of a span alike. The writers
(
sapling new,sapling generate, the skeleton; the assistant is asked to) put a head longer than 120 characters over more lines: the next lines of a div's attributes at four spaces, of a span's at two:
[]{.formula formula="CHG ~ BASE + SITEGR1 + TRTP*AVISIT + BASE*AVISIT" covariance="us" fallback="toeph; ar1"
df="kenward-roger" estimation="reml" visit="Week 24"
stats="LS mean (SE) = xx.x (xx.xx); Diff (SE) = xx.x (xx.xx); 95% CI = (xx.x, xx.x); p-value = x.xxx"}
- Some values of the example plans have a backslash before
%or_(timing="approximately 50\% of the patients"): the compiler keeps a value as it is written. In a new plan, write%and_as they are.
Citations#
[]{.cite source="protocol:4.2" words="the words of the sentence"}, []{.cite source="protocol:4.1" sentence="3"},
[]{.cite source="usdm:arms"}. The span is empty: the words find the sentence in the protocol, they are not text of
the plan. (Quarto shows the content of a span: words in the content would show in the sentence, in a heading, in the
anchor of the heading and in the table of contents.) The reader also takes the words as the content, [the words]{.cite source="protocol:4.2"}: the citation is the same.
A citation belongs to what comes before it, by the order of the blocks:
| Where | The citation of |
|---|---|
| at the end of a heading line | the section |
on the line right after the closing ::: of a block (no empty line between) | that block |
| at the start of the body of an environment, or after its parts | the environment |
| in a paragraph | the paragraph (and the tests the paragraph declares) |
cite: of a setting in the front matter | the setting |
An empty line between a block and a citation makes the citation a paragraph of its own (the paragraph's).
Prose#
The text of a plan is Markdown:
| Text | Markdown |
|---|---|
| a new paragraph | an empty line |
| emphasis | *x* |
| bold | **x** |
| code | `x` |
| small capitals | [x]{.smallcaps} |
| a web address | <https://...> |
| a link | [text](url) |
| a footnote (in a table: a footnote of the shell) | ^[x] |
| a line break | a backslash at the end of the line |
| math | $x$ |
| display math | $$x$$ |
| a numbered equation | $$x$$ {#eq-1} |
| equations aligned | $$\begin{aligned} ... \end{aligned}$$ |
| a list | - item, 1. item |
| a definition list | the term on its line, then : the definition |
| a quote | > text |
| a table of text | a pipe table (| a | b |, then |---|---|): the cells are text |
| a special character | the character (Markdown escapes \*, \_, \[, \$, ... where Markdown needs them) |
Models#
::: {.model contrast="pairwise"}
$$
Y_{i} = \mu + \beta\,\mathrm{BASE}_{i} + \gamma_{s(i)} + \tau_{k(i)} + \varepsilon_{i}
$$
[]{.where bindings="Y = CHG, s = SITEGR1, k = TRTP"}
:::
The equation is the display math of the div, as written; the where and formula spans come after it. A model as a
formula only is a span in the analysis: []{.formula formula="CHG ~ BASE + SITEGR1 + TRTPN" df="kenward-roger"}.
Comments#
A comment is an HTML comment <!-- ... -->: Quarto does not show it, and the reader of Sapling skips it. A comment
that a tool of Sapling reads starts a comment of its own, above the block or paragraph it is about:
<!-- drafted by: agent (...) -->, <!-- default: omakase (...) -->, <!-- review (Name, date): text -->,
<!-- synced from ... -->. A comment inside the arguments of a command goes above the block; a comment inside a
paragraph goes above the paragraph.
Macros#
Markdown has no macros: a family of tables is written out, one tlf div each (or made by sapling generate table).
What the reader also accepts#
The reader does not depend on the layout of the writer:
- attributes in any order, with more blanks, over more lines; values in double quotes, single quotes or without quotes
(a value without blanks);
id="X"for#X; - a div with one class and no braces:
::: decision; - empty lines (or none) between the parts of an environment;
- a block without a prose argument as a span on a line of its own (
[]{.derive variable="ADSL.AGEGR1N" rule="..."}), and a prose argument as an attribute (::: {.population #EFF name="Efficacy population" condition="ADSL.EFFFL"}); - the front matter in block or flow YAML; a list as
[a, b]or as- alines; - a class one letter off a Sapling command (
.summarise) is read as that command's typo: the compiler saysdid you mean. Other classes (.callout-note,.column-margin) are Markdown's own: the reader reads their content.
The exceptions#
The forms that are not the rule of their role:
| What | Form | Why |
|---|---|---|
| the title, the subtitle, the date, the author | top-level keys of the front matter | they are Quarto's own keys |
the settings drafted-by and alpha-level | drafted-by:, alpha-level: | kebab case of two words, as Quarto's keys |
| the title page, the table of contents | the title block of Quarto, toc: true | Quarto's own |
| an item of a list | a list item | Markdown's own |
| a citation | []{.cite source="..." words="..."} | an empty span: the words locate the sentence, they are not text (Quarto shows the content of a span) |
| emphasis, bold, code, small capitals, a web address, a footnote | Markdown's own forms (Prose) | Markdown has them |
| the appendix | {.appendix} on the first section of the appendix | Quarto's own (an empty ::: {.appendix} div when no section follows) |
model (a part) | a div | its equation is display math |
the option class of dataset | the attribute adam-class | Pandoc reads class= as the classes of the div |
Values and conditions#
| Convention | Example |
|---|---|
an attribute is key="value"; a flag is "true" | population="ENR" total="true" |
a list is separated by ; | groups="Placebo; Low Dose; High Dose"; in the front matter a YAML list [Placebo, Low Dose, High Dose] |
an id has letters, digits and _ | #P_EFF, ANOVA_4_2 |
| a value with a double quote goes in single quotes | records='PARAMCD = "ACTOT"' |
Conditions are written on ADaM variables the way you say them; sapc writes the SAS WHERE text of the chain:
| You write | The structured SAP gets |
|---|---|
ADSL.EFFFL (a flag alone) | ADSL.EFFFL = 'Y' |
ADSL.SAFFL = "Y" | ADSL.SAFFL = 'Y' |
in []{.data dataset="ADQSADAS" records='...'}: PARAMCD = "ACTOT" and AVISIT = "Week 24" | ADQSADAS.PARAMCD = 'ACTOT' AND ADQSADAS.AVISIT = 'Week 24' |
ADAE.AESEV in ("MILD", "MODERATE") | ADAE.AESEV IN ('MILD', 'MODERATE') |
ADSL.AGE >= 65 | ADSL.AGE >= 65 |
ADVS.ABLFL != "Y" or ADVS.ABLFL ne "Y" | ADVS.ABLFL NE 'Y' |
ADLB.ATOXGR is not missing, ADAE.AESEV is missing (a text variable; the words of the ADaM rule language) | ADLB.ATOXGR NE '', ADAE.AESEV = '' (a number: an error, the chain's conditions have no missing number) |
A variable without a data set takes the data set of its analysis (the data span), else ADSL. sapc checks every
variable against the ADaM specification (adam: in the front matter), the quotes against the type (text in quotes,
numbers without), and every quoted value against the variable's codelist: AVISIT = "WEEK 24" is an error with the
fix write AVISIT = "Week 24".
The settings#
The settings of the plan are keys of the front matter, under sapling: (Quarto's own keys title, subtitle,
date and author are at the top).
| Key | Meaning | Structured SAP |
|---|---|---|
study: CDISCPILOT01 | the study id (required) | study.id |
title:, subtitle:, date:, author: (top level); version: | the title page | |
protocol: {file: protocol/protocol.json, name: Protocol H2Q-MC-LZZT(c)} | the protocol: the source of every []{.cite source="protocol:..."}. A page index ({pages: [{page, text}]}, as the chain's docs/_index/protocol.json) or a text file with numbered headings | study.source (the chain's path docs/_index/protocol.json; another with chain:) |
usdm: protocol/study_usdm.json | the protocol as CDISC USDM: adds usdm:objectives, usdm:endpoints, usdm:arms, usdm:encounters, usdm:estimands | |
adam: adam | the ADaM specification (<dataset>.json, codelists.json); without it, the ADaM IG (ADSL, BDS) | |
sdtm: sdtm | the SDTM data a plan derives its ADaM from (a folder of .xpt files, or its sdtm.json): The ADaM from the SAP | |
status: DRAFT | DRAFT or APPROVED | study.status |
drafted-by: agent | who drafted the plan | study.drafted_by |
approved: {by: Name, on: 2026-12-01, role: statistician} | a person only: the compiler never writes it, an agent never writes it (the MCP server and the assistant refuse it). One per role (statistician, senior statistician, sponsor, chief investigator, ...): a YAML list of maps; the title page has a row for each | study.approved_by, approved_on (the statistician's); the JSON study.approvals |
alpha-level: {level: 0.05, sided: 2, cite: ...} | the default significance level and sidedness of every test, with its source | alpha, sided, alpha_source |
treatments: {groups: [A, B, C], reference: A, variable: ADSL.TRT01A, id: TRT, cite: ...} | the treatment groups, in the order of the columns of every table; reference: the control group of every comparison (The direction of a comparison) | the grouping TRT |
visits: {visits: [Baseline, Week 2, ...], variable: AVISIT} | the analysis visits of the summaries by visit | the visit grouping of byvisit |
library: path | another SAP library (default: standards/sap_library) |
toc: true at the top level gives the table of contents. Quarto shows the title block when there is a title:.
Citations: the protocol is the source#
Every block of the structured SAP has a section and a quote: the protocol's words the decision rests on, word for
word; the chain's checker finds each quote in the protocol. You do not copy the quote: you point at it with a citation
span, and the compiler copies it.
| You write | The quote is |
|---|---|
[]{.cite source="protocol:4.1"} | the first sentence of section 4.1 |
[]{.cite source="protocol:4.3.3" words="Effects in the ANCOVA"} | the one sentence of 4.3.3 that has these words |
[]{.cite source="protocol:3.9.3.1" words="Safety measures ... readings)"} | the exact words from Safety measures to readings) (...) |
[]{.cite source="protocol:4.3.3" sentence="2"} | the second sentence of 4.3.3 |
[]{.cite source="usdm:arms"} | the first sentence of the USDM part arms |
The section is the protocol's own number (4.3.3 is the page Protocol 4.3.3. of the index) or a name
(usdm:arms, protocol:title). When the words are not in the section, the error shows the closest sentences; when
they are in two, it shows both. In the PDF, a citation in a paragraph is a footnote with the protocol's words; a
citation after a block or a heading is a source line under it.
Where a citation belongs (the forms: Citations):
- on the line right after the closing
:::of a block: the quote of that block; - at the end of a heading line: the default quote of every block of that section (and its subsections);
- inside a paragraph: a citation of the paragraph, and the quote of the tests (
[anova]{.test}) of the paragraph; - at the top of the body of a
tlf,analysisorestimand: the quote of that block.
A table passes its quote to its analyses, an analysis to its subset and its model. The section of a block in the
structured SAP is the section of its quote; the HTML and the JSON also give the section of your SAP.
A protocol PDF whose text is not its words. sapling new --protocol protocol.pdf reads the text layer of the PDF
(protocol/protocol.txt). On some PDFs that layer is not the words the page shows, and sapling new says which pages:
| The PDF gives | Example | What Sapling does |
|---|---|---|
| letter-spaced text | T his is a P h as e 2 b | a line is rebuilt when every word it then has is a word of the readable pages of the same protocol (only spaces are removed; nothing is guessed); a line that cannot be rebuilt stays, and a citation finds its words there by their letters (the quote is then your words, every letter of them the protocol's) |
| cipher text (a font without the map of its letters) | *5B;=B;+HI8MC: | never guessed: the pages are listed; a citation of words there is the error cite-unreadable, which names the page of the PDF and shows the readable sentences of the section; a citation of a sentence there (sentence="n", or the first sentence) is the warning cite-unreadable |
| a word broken at the end of a line | Last-Observation- / Carried-Forward | one word again (Last-Observation-Carried-Forward); imput- / ation also matches the words imputation |
When the words of a citation are not in the section, the error shows the nearest sentences (by the words they share, else by their letters), from the place where they share a word.
Populations, subsets, groupings, derivations#
::: {.population #ENR condition="ADSL.ITTFL"}
All enrolled patients
:::
[]{.cite source="protocol:3.10.2"}
::: {.population #EFF condition="ADSL.EFFFL"
definition="Patients with a baseline and at least one postrandomization measurement"}
Efficacy population
:::
[]{.cite source="protocol:4.1"}
::: {.datasubset #SER condition='ADAE.TRTEMFL = "Y" and ADAE.AESER = "Y"'}
Serious treatment-emergent events
:::
<!-- the groups of the values in the data -->
::: {.grouping #AGEGRP variable="ADSL.AGEGR1"}
Age group
:::
<!-- these groups, in this order -->
::: {.grouping #AGEGRP3 variable="ADSL.AGEGR1" groups="<65; 65-80; >80"}
Age group
:::
::: {.derivation #LOCF variable="ADQSADAS.AVAL" rule="the last observation carried forward"
condition='ADQSADAS.ANL01FL = "Y"'}
:::
| Block | Id, attributes, body | Structured SAP |
|---|---|---|
population | #ID, condition=, definition= (the definition in words); the name in the body | population |
datasubset | #ID, condition=; the name in the body | subset |
grouping | #ID, variable=, groups=; the name in the body | grouping (groups, or data_driven: true) |
derivation | #ID, variable=, rule= (the rule in words), condition= | derivation |
Tests and statistics#
A test is declared once, where the SAP names it, and used by name:
The groups are compared by [anova]{.test} for continuous variables and by [chisq]{.test} for categorical
variables[]{.cite source="protocol:4.2" words="The groups will be compared"}.
The text reads "analysis of variance (ANOVA)" and "Pearson's chi-square test". The structured SAP has two method
blocks, ANOVA_4_2 and CHISQ_4_2 (the test and the section of its quote), with the cited sentence as quote and
sided, alpha from alpha-level:. An analysis then says test="anova": the test of that kind declared in its
section (or test="ANOVA_4_2" by id); if none is declared there, the compiler declares it with the analysis's quote.
The attributes of a test span: #ID (or id=), sided=, alpha=, pattern= (default x.xxx), text= (other
words in the prose).
Tests: anova, ancova, chisq, fisher, cmh, logrank, ttest, wilcoxon, mmrm, logistic, cox,
riskdiff, riskratio, oddsratio, poisson, negbin. Which
engine computes each, and what it can report: The methods and what they report.
The statistics of a summary are the rows of the shell, each with its cell picture:
::: {.statistics #CONT_SUMM rows="n = xx; Mean (SD) = xx.x (xx.xx); Median = xx.x; Min, Max = xx, xx"}
:::
::: {.statistics #CAT_SUMM rows="n (%) = xx (xx.x%)"}
:::
::: {.statistics #KM_SUMM rows="n = xx; Events = xx; Median = xx.x; 95% CI = (xx.x, xx.x)" estimator="kaplan-meier"}:
the statistics are the estimates of a Kaplan-Meier curve (the median time to the event and its CI), not of the values
of the variable. The method block gets estimator: kaplan-meier, and the engine of sapling computes them
(ars_e_km: the subjects, the events, the median and its 95% CI by Brookmeyer and Crowley; the censoring of CNSR).
conftype= is the transformation of the interval: plain (the default, written into the method: what both engines
did before it was an option; R survfit's own default is log, SAS LIFETEST's own log-log), log, log-log.
time="24" with the row Survival = xx.x gives the survival at that time in percent and its CI (one block per
time; its CI is then of the survival, not of the median). unit= (days, weeks, months, years) says the unit
of the time; without it a warning says that the time is read in the unit of the analysis variable, and with it the
compiler checks it against the parameter where the plan gives its unit (a time-to-event step of the plan gives AVAL in
days; a parameter named "... (Months)"):
::: {.statistics #PFS12 rows="Survival = xx.x; 95% CI = (xx.x, xx.x)" estimator="kaplan-meier" time="12" conftype="log-log"}.
The rendered plan states the options of each method that change a number (the ties of a Cox model, the interval of a proportion, the transformation of a Kaplan-Meier interval, ...): the plan's value, or the registry's default marked "(the default)".
::: {.statistics #RESP rows="n (%) = xx (xx.x%); 95% CI = (xx.x, xx.x)" ci="clopper-pearson"}: the confidence
interval of each percent (of the subjects of the column, as the percent), by the method of ci=: clopper-pearson
(the default: the exact interval), wilson (the score interval, without continuity correction), wald,
agresti-coull (both kept within 0 and 100). The method block gets ci: (the default written when the plan says
none), and the engines compute it (ars_e_propci; SAS %proportion, PROC FREQ BINOMIAL).
::: {.statistics #EAIR rows="n = xx; Patient-years = xx.x; Rate = xx.xx; 95% CI = (xx.xx, xx.xx)" exposure="ADSL.TRTDUR / 365.25" per="100"}:
the exposure-adjusted incidence rate of each group (as the statistics of an incidence): the subjects with the event,
the sum of the exposure of the subjects of the group (exposure=: a variable of ADSL, or a variable divided by a
number: days to years), the rate per per= units (default 100: per 100 patient-years) and its exact Poisson interval.
The exposure of a group is that of all its subjects, not the time to the first event (ars_e_eair; SAS %rate).
count="events" (with the row Events = xx) makes the rate of the events, every record of the group, with the exact
Poisson interval of the number of events (method eventrate; SAS %rate(count=events)); a block with a rate and the
row Events and no count= is an error (the rate of the subjects or of the events?).
Row labels: n, Mean, SD, Median, Q1, Q3, Min, Max, %, Events, p-value, LS mean, SE,
Diff, 95% CI, Estimate, OR, HR, RD (risk difference), RR (relative risk), Rate ratio, Rate,
Patient-years, Survival. One pattern per statistic, in order; the parentheses of a percent belong to
its pattern ((xx.x%)). Without =, the library's pattern. A p-value that its pattern rounds to 0 or 1 prints as
<0.001 or >0.999 (x.xxx; its number is unchanged), and a category of a ::: {.codelist} prints its label
(F = Female). summarize, tabulate and incidence use the
library's CONT_SUMM, CAT_SUMM, AE_INCID; declare a statistics block with the same id to change the study's
decimals, or a new id and stats="ID".
The methods and what they report#
One registry says what each method computes (src/methods.ts): its inputs, its options with their defaults, the rows
it can report, the engine that computes it in R (r-agentic) and in SAS (the macro library), what it cannot do, and a
small known answer. The compiler reads it: a statistic a method makes is never warned about; a statistic it does not
make, an input its engine cannot take (an interaction, strata, a censoring variable other than CNSR) and a test no
engine computes are each an error chain-engine at the place of the plan, which says which method can, and the output
of that analysis is not made (only that output: sapling build and sapling run make the others). The table below
is made from the registry (node tools/methods_doc.ts); test/methods-r.test.ts runs every row of every method
through r-agentic's contract maker and R engines on the known answer.
| Method | The plan asks for it with | It can report (rows of stats=) | R engine (r-agentic) | SAS macro | It cannot |
|---|---|---|---|---|---|
| count: the number of subjects of each group | the N of each column (every table); a summary whose statistics are n only | n = xx (the subjects with a record, each once; a summary with a mean gives the number of values) | ars_e_count: cards::ard_tabulate | %freq (PROC FREQ) | count the values of a variable (n with a mean does: descriptive statistics) |
| catsumm: counts and percents of the subjects in each category | ::: {.tabulate}, ::: {.incidence}, or statistics n (%), Events | n (%) = xx (xx.x%); Events = xxx | ars_e_catsumm: cards::ard_tabulate; the records at the worst group of each subject (ars_worst), the rows of the threshold; the order of the rows in the table (ars_table) | %freq (PROC FREQ) | a percent of another denominator than the subjects of the column; a confidence interval of a proportion |
| contsumm: descriptive statistics of a number | ::: {.summarize} (the library's CONT_SUMM), ::: {.byvisit}, or statistics Mean (SD), Median, Q1, Q3, Min, Max | n = xx (the subjects with a value); Mean (SD) = xx.x (xx.xx); Median = xx.x; Q1, Q3 = xx.x, xx.x; Min, Max = xx, xx | ars_e_contsumm: cards::ard_summary, stats::quantile(type = 2) | %describe (PROC MEANS) | the standard error of a mean (SE) in R (the SAS macro makes it: the two engines differ); a confidence interval of a mean; a geometric mean or a coefficient of variation |
| anova: one-way analysis of variance (ANOVA): the F test of the groups | test="anova" on a summary; a model of a number on the treatment groups only (CHG ~ TRTP) | p-value = x.xxx | ars_e_anova: cardx::ard_stats_oneway_test(var.equal = TRUE) | %anova (PROC GLM) | factors or covariates (write them in a model: the dose response, or contrast="pairwise"); an estimate or a confidence interval |
| ancova: analysis of covariance (ANCOVA): the test for linear dose response, with the estimate of the dose | a model with the treatment as a number (CHG ~ BASE + SITEGR1 + TRTPN); also an ANOVA with the dose (CHG ~ SITEGR1 + TRTPN) | p-value = x.xxx; Estimate = xx.xx (the slope per unit of the dose); SE = xx.xx; Estimate (SE) = xx.xx (xx.xx); 95% CI = (xx.xx, xx.xx) (of the estimate); each with the p-value of the test | ars_e_ancova: stats::lm; cardx::ard_stats_anova of the models with and without the dose; stats::confint | %ancova (PROC GLM) | interaction terms (the engine fits the main effects only); strata; the LS mean of each group (write contrast="pairwise" with TRTP); a confidence level other than 95% |
| lsmeans: the LS means of a linear model and the pairwise differences, from one fit of all groups | a model of the groups with contrast="pairwise" (CHG ~ BASE + SITEGR1 + TRTP); ANCOVA or ANOVA | LS mean (SE) = xx.x (xx.xx); LS mean = xx.x; Diff (SE) = xx.x (xx.xx); Diff = xx.x; 95% CI = (xx.x, xx.x) (of the difference); p-value = x.xxx (of the difference); each with the p-value of the test | ars_e_lsmeans: stats::lm; emmeans::emmeans, emmeans::contrast(adjust = "none") | %lsmeans (PROC MIXED) | the confidence interval of an LS mean (only of a difference); an SE row on its own (write LS mean (SE) or Diff (SE)); interaction terms; strata; a multiplicity adjustment (Dunnett, Holm, ...) |
| mmrm: a mixed model for repeated measures (MMRM): the LS means at one visit and their pairwise differences | a model with a covariance term and contrast="pairwise": CHG ~ BASE + TRTP*AVISIT + us(AVISIT | USUBJID) with visit="Week 24" | LS mean (SE) = xx.x (xx.xx); LS mean = xx.x; Diff (SE) = xx.x (xx.xx); Diff = xx.x; 95% CI = (xx.x, xx.x) (of the difference); p-value = x.xxx (of the difference); each with the p-value of the test | ars_e_mmrm: mmrm::mmrm; emmeans::emmeans at the visit, emmeans::contrast(adjust = "none") | %mmrm (PROC MIXED) | a treatment as a number (a dose response) or the overall test of the groups: only the pairwise comparisons at one visit; the LS means averaged over visits; the confidence interval of an LS mean; an SE row on its own; the non-linear Kenward-Roger adjustment of mmrm (its default "Kenward-Roger"): kenward-roger is the linear one, as SAS DDFM=KR |
| chisq: Pearson's chi-square test of the groups | test="chisq" on a tabulate (one table) or an incidence (one test per row) | p-value = x.xxx | ars_e_chisq (one table); ars_e_fisher test=CHISQ (a test per row of occurrence data): cardx::ard_stats_chisq_test(correct = FALSE) | %chisq (PROC FREQ) | a continuity correction; an estimate |
| fisher: Fisher's exact test of the groups | test="fisher" on a tabulate (the r x c table), or comparisons= on an incidence (each pair, per row) | p-value = x.xxx | ars_e_chisq test=FISHER (one table); ars_e_fisher (per row of occurrence data): cardx::ard_stats_fisher_test (stats::fisher.test) | %fisher (PROC FREQ) | a one-sided test; an odds ratio |
| cmh: the Cochran-Mantel-Haenszel test (general association; row mean scores when the categories are numbers) | a model of a category with strata: AVAL ~ TRTP + strata(SITEGR1) with test="cmh" | p-value = x.xxx | ars_e_cmh: written by hand: Q = G' V^-1 G (SAS PROC FREQ CMH) | %cmh (PROC FREQ) | the common odds ratio; the Breslow-Day test |
| km: the Kaplan-Meier estimates of each group: the subjects, the events, the median time and its CI | a statistics block with estimator="kaplan-meier", as the statistics of an analysis of a time to event (Surv(AVAL, CNSR) ~ TRTA) | n = xx; Events = xx; Median = xx.x; 95% CI = (xx.x, xx.x) (of the median) | ars_e_km: survival::survfit(conf.type = conftype), quantile() | %km (PROC LIFETEST) | the survival rate at a time in the same statistics block as the CI of the median (one interval per block: kmrate); the number at risk in a table (the figure has it); another censoring variable than CNSR; the mean or the restricted mean survival time |
| kmrate: the Kaplan-Meier estimate of the survival function at a named time in each group (in percent), with its CI | a statistics block with estimator="kaplan-meier", time= and the survival: ::: {.statistics #PFS12 rows="Survival = xx.x; 95% CI = (xx.x, xx.x)" estimator="kaplan-meier" time="12" conftype="log-log"}, one block per time | Survival = xx.x (in percent); 95% CI = (xx.x, xx.x) (of the survival) | ars_e_km: survival::survfit(conf.type = conftype); summary(times = time, extend = TRUE) | %km (PROC LIFETEST) | several times in one statistics block (write one block per time); the CI of the median in the same block; the difference of two survival rates |
| logrank: the log-rank test of the groups, stratified when the model has strata | a model of a time to event: Surv(AVAL, CNSR) ~ TRTA; stratified: Surv(AVAL, CNSR) ~ TRTA + strata(STRAT1) + strata(STRAT2) (or strata="STRAT1; STRAT2") | p-value = x.xxx | ars_e_logrank: survival::survdiff (with strata()) | %logrank (PROC LIFETEST) | a hazard ratio (a Cox model: test="cox"); the test of each pair of groups (a Cox model with pvalue="logrank"); other weights (Wilcoxon, Fleming-Harrington); another censoring variable than CNSR |
| cox: a Cox proportional hazards model: the hazard ratio of each pair of groups from one fit, its CI and p-value | a model of a time to event with test="cox" (or with covariates): Surv(AVAL, CNSR) ~ TRTP + strata(STRAT1) + strata(STRAT2); the pairs of comparisons= | HR = xx.xx; 95% CI = (xx.xx, xx.xx) (of the hazard ratio); p-value = x.xxx (of the pair); each with the p-value of the test | ars_e_cox: survival::coxph(ties = ...); the contrast of the coefficients of the pair, exp(); survival::survdiff of the pair (pvalue=logrank) | %cox (PROC PHREG) | a treatment as a number (a hazard ratio per unit of a dose); time-dependent covariates or an interaction with time (a test of proportional hazards); a profile likelihood interval; the score or likelihood ratio test of all groups together; another censoring variable than CNSR |
| logistic: logistic regression: the odds ratio of each pair of groups from one fit, its CI and p-value | a model of a response with two values with test="logistic" (a text response with covariates is one by itself): CRIT1FL ~ TRTP + AGE + SEX with event="Y"; the pairs of comparisons= | OR = xx.xx; 95% CI = (xx.xx, xx.xx) (of the odds ratio); p-value = x.xxx (the Wald test of the pair); each with the p-value of the test | ars_e_logistic: stats::glm(family = binomial); the contrast of the coefficients of the pair, exp() | %logistic (PROC LOGISTIC) | a treatment as a number; interaction terms; a response with more than two values (an ordinal or multinomial model); Firth's penalized likelihood or an exact test when the data separate; a profile likelihood interval |
| propci: counts and percents of the subjects in each category, with the confidence interval of each percent by a named method | a statistics block with the CI and ci=: ::: {.statistics #RESP rows="n (%) = xx (xx.x%); 95% CI = (xx.x, xx.x)" ci="clopper-pearson"}, as the statistics of a tabulate or an analysis of a category | 95% CI = (xx.x, xx.x) (of the percent, in percent) | ars_e_propci: cards::ard_tabulate (the counts, as ars_e_catsumm); the interval written out (stats::qbeta for Clopper-Pearson) | %proportion (PROC FREQ) | a continuity correction of the Wilson interval; the Jeffreys or mid-p interval; a stratified proportion; the CI of a count of events (records) |
| riskdiff: the risk difference of each pair of groups (the difference of two proportions) with its interval by a named method; stratified: Mantel-Haenszel | a model of a response with two values: CRIT1FL ~ TRTP with test="riskdiff" ci="newcombe" event="Y"; stratified: CRIT1FL ~ TRTP + strata(STRAT1) | RD = x.xxx (a proportion: 0.25 is 25 percentage points); 95% CI = (x.xxx, x.xxx) (of the difference); p-value = x.xxx (the chi-square test of the pair); each with the p-value of the test | ars_e_binary: the interval written out (ars_diff_ci; Newcombe's from two Wilson intervals; Miettinen-Nurminen by stats::uniroot of the score); stats::mantelhaen.test with strata | %risk (PROC FREQ) | a non-inferiority test against a margin (the interval is given; the margin is the plan's text); the stratified Newcombe or Miettinen-Nurminen interval (strata: the Mantel-Haenszel interval only); a continuity correction; an exact interval |
| riskratio: the relative risk of each pair of groups (the ratio of two proportions) with the Wald interval of its logarithm | a model of a response with two values: CRIT1FL ~ TRTP with test="riskratio" event="Y" | RR = xx.xx; 95% CI = (xx.xx, xx.xx) (of the relative risk); p-value = x.xxx (the chi-square test of the pair); each with the p-value of the test | ars_e_binary: the ratio and its interval written out | %risk (PROC FREQ) | a stratified (Mantel-Haenszel) relative risk; a score interval; a ratio where a group has no event (no number) |
| oddsratio: the odds ratio of the 2 x 2 table of each pair of groups with the Wald interval of its logarithm | a model of a response with two values: CRIT1FL ~ TRTP with test="oddsratio" event="Y" (with covariates: a logistic regression) | OR = xx.xx; 95% CI = (xx.xx, xx.xx) (of the odds ratio); p-value = x.xxx (the chi-square test of the pair); each with the p-value of the test | ars_e_binary: the ratio and its interval written out | %risk (PROC FREQ) | a stratified (Mantel-Haenszel) odds ratio; an exact interval; an odds ratio where a cell of the table is 0 (no number) |
| poisson: a Poisson regression of a count with the logarithm of its exposure as the offset: the rate ratio of each pair of groups from one fit, its CI and p-value | a model of a count with test="poisson" and its exposure: AVAL ~ TRTP + AGE with exposure="TRTDUR / 365.25" (a variable, or a variable divided by a number: days to years) | Rate ratio = xx.xx; 95% CI = (xx.xx, xx.xx) (of the rate ratio); p-value = x.xxx (the Wald test of the pair); each with the p-value of the test | ars_e_rates: stats::glm(family = poisson, offset); the contrast of the coefficients of the pair, exp() | %rateratio (PROC GENMOD) | a treatment as a number; interaction terms; zero-inflated or hurdle models; a scale for overdispersion (Pearson or deviance): use negbin; the rate of each group from the model (the observed rates: an incidence rate summary) |
| negbin: a negative binomial regression of a count with the logarithm of its exposure as the offset: the rate ratio of each pair of groups from one fit, its CI and p-value | a model of a count with test="negbin" and its exposure: AVAL ~ TRTP + AGE with exposure="TRTDUR / 365.25" (a variable, or a variable divided by a number: days to years) | Rate ratio = xx.xx; 95% CI = (xx.xx, xx.xx) (of the rate ratio); p-value = x.xxx (the Wald test of the pair); each with the p-value of the test | ars_e_rates: MASS::glm.nb (offset); the observed information of the coefficients and k; the contrast of the pair, exp() | %rateratio (PROC GENMOD) | a treatment as a number; interaction terms; zero-inflated or hurdle models; another dispersion (NB1, a fixed k); the rate of each group from the model (the observed rates: an incidence rate summary) |
| eair: the exposure-adjusted incidence rate of each group: the subjects with an event per 100 patient-years, with its exact Poisson CI | a statistics block with the rate and exposure=: ::: {.statistics #EAIR rows="n = xx; Patient-years = xx.x; Rate = xx.xx; 95% CI = (xx.xx, xx.xx)" exposure="ADSL.TRTDUR / 365.25" per="100"}, as the statistics of an incidence (stats="EAIR") | Rate = xx.xx (per 100 patient-years); Patient-years = xx.x; n = xx (the subjects with the event); 95% CI = (xx.xx, xx.xx) (of the rate) | ars_e_eair: cards::ard_tabulate (the subjects, as ars_e_catsumm); the sum of the exposure; stats::qchisq | %rate (PROC FREQ) | the exposure up to the first event (the plan computes it into a variable of ADSL); the rate of the events (records): eventrate (count="events"); the difference or ratio of two incidence rates |
| eventrate: the exposure-adjusted rate of the events of each group: the events (every record) per 100 patient-years, with its exact Poisson CI | a statistics block with the rate, exposure= and count="events": ::: {.statistics #EVR rows="Events = xx; Patient-years = xx.x; Rate = xx.xx; 95% CI = (xx.xx, xx.xx)" exposure="ADSL.TRTDUR / 365.25" per="100" count="events"}, as the statistics of an incidence (stats="EVR") | Rate = xx.xx (the events per 100 patient-years); Events = xx (the records of the group); Patient-years = xx.x; 95% CI = (xx.xx, xx.xx) (of the rate) | ars_e_eair: cards::ard_tabulate (the records, as ars_e_catsumm events=Y); the sum of the exposure; stats::qchisq | %rate (PROC FREQ) | a confidence interval that allows for subjects with many events (overdispersion: a negative binomial model, test="negbin"); the difference or ratio of two event rates (a Poisson or negative binomial model of the count of each subject) |
| perterm: the comparison of two proportions of each pair of groups for each row of an incidence (each term): the risk difference (relative risk, odds ratio) of the subjects with the term against those without | an incidence with a test of two proportions: ::: {.incidence #AE_RD condition='ADAE.TRTEMFL = "Y"' rows="pt" test="riskdiff" ci="newcombe" threshold="5"} (test="riskratio", "oddsratio"; pvalue="false": the estimate and its CI only) | RD = x.xxx (a proportion: 0.25 is 25 percentage points); 95% CI = (x.xxx, x.xxx) (of the difference); p-value = x.xxx (the chi-square test of the 2 x 2 table of the row (nominal)); each with the p-value of the test | ars_e_perterm: the flag of each row for every subject of the population, then ars_e_binary on it (the same engine as the riskdiff, riskratio and oddsratio methods: no second computation) | %risk (PROC FREQ) | a comparison of each row by the worst group (worst=); an exact test or interval of each row; an adjustment of the p-values of the rows for multiplicity (they are nominal, as the safety guidelines read them); a stratified comparison of each row |
| ttest: the two-sample t-test of each pair of groups: the difference of the means, its SE, CI and p-value | a model of a number on the groups with test="ttest": CHG ~ TRTP (variances="unequal": Welch's); the pairs of comparisons= | Diff = xx.x; Diff (SE) = xx.x (xx.xx); 95% CI = (xx.x, xx.x) (of the difference); p-value = x.xxx (of the pair); each with the p-value of the test | ars_e_ttest: stats::t.test(var.equal = variances == "equal") | %ttest (PROC TTEST) | the variance of all groups pooled (that is the LS means of a one-way model: contrast="pairwise"); a paired t-test; a one-sided test |
| wilcoxon: the Wilcoxon rank-sum test of each pair of groups, with the Hodges-Lehmann estimate of the shift and its CI | a model of a number on the groups with test="wilcoxon": CHG ~ TRTP; the pairs of comparisons= | Estimate = xx.xx (the Hodges-Lehmann estimate of the shift); 95% CI = (xx.xx, xx.xx) (of the shift); p-value = x.xxx (of the pair); each with the p-value of the test | ars_e_wilcoxon: stats::wilcox.test(exact = FALSE, correct =); the median of the differences and Moses's order statistics written out | %wilcoxon (PROC NPAR1WAY) | the exact test or the exact interval; a stratified (van Elteren) test; the signed-rank test of paired values |
| interaction-lm: the F test of the interaction of the treatment and a factor in an ANCOVA (a linear model of the groups, contrast="pairwise") (a subgroup block) | a subgroup block of an analysis with an ANCOVA (a linear model of the groups, contrast="pairwise"): ::: {.subgroup analyses="A1" factors="SG_SEX" interaction="0.10"}; one test per factor | p-value = x.xxx (of the interaction, one per factor) | ars_e_interaction: stats::lm twice (with and without the treatment-by-factor terms); stats::anova of the two (the F test) | %interaction_glm (PROC GLM) | the Wald test of the interaction; the interaction of the treatment and a continuous covariate (a factor has levels); the estimate of each level from one model with the interaction terms (the levels are fitted one by one: the analysis in each level); a level without events or values in a group: the fit does not converge, and no p-value is made |
| interaction-logistic: the likelihood-ratio test of the interaction of the treatment and a factor in a logistic regression (a subgroup block) | a subgroup block of an analysis with a logistic regression: ::: {.subgroup analyses="A1" factors="SG_SEX" interaction="0.10"}; one test per factor | p-value = x.xxx (of the interaction, one per factor) | ars_e_interaction: stats::glm(family = binomial) twice; the difference of the deviances, stats::pchisq | %interaction_logistic (PROC LOGISTIC) | the Wald test of the interaction; the interaction of the treatment and a continuous covariate (a factor has levels); the estimate of each level from one model with the interaction terms (the levels are fitted one by one: the analysis in each level); a level without events or values in a group: the fit does not converge, and no p-value is made |
| interaction-poisson: the likelihood-ratio test of the interaction of the treatment and a factor in a Poisson regression with an exposure (a subgroup block) | a subgroup block of an analysis with a Poisson regression with an exposure: ::: {.subgroup analyses="A1" factors="SG_SEX" interaction="0.10"}; one test per factor | p-value = x.xxx (of the interaction, one per factor) | ars_e_interaction: stats::glm(family = poisson, offset) twice; the difference of the deviances, stats::pchisq | %interaction_genmod (PROC GENMOD) | the Wald test of the interaction; the interaction of the treatment and a continuous covariate (a factor has levels); the estimate of each level from one model with the interaction terms (the levels are fitted one by one: the analysis in each level); a level without events or values in a group: the fit does not converge, and no p-value is made |
| interaction-negbin: the likelihood-ratio test of the interaction of the treatment and a factor in a negative binomial regression with an exposure (a subgroup block) | a subgroup block of an analysis with a negative binomial regression with an exposure: ::: {.subgroup analyses="A1" factors="SG_SEX" interaction="0.10"}; one test per factor | p-value = x.xxx (of the interaction, one per factor) | ars_e_interaction: MASS::glm.nb twice (k estimated in each); 2 (log L1 - log L0), stats::pchisq | %interaction_genmod (PROC GENMOD) | the Wald test of the interaction; the interaction of the treatment and a continuous covariate (a factor has levels); the estimate of each level from one model with the interaction terms (the levels are fitted one by one: the analysis in each level); a level without events or values in a group: the fit does not converge, and no p-value is made |
| interaction-cox: the likelihood-ratio test of the interaction of the treatment and a factor in a Cox model (a subgroup block) | a subgroup block of an analysis with a Cox model: ::: {.subgroup analyses="A1" factors="SG_SEX" interaction="0.10"}; one test per factor | p-value = x.xxx (of the interaction, one per factor) | ars_e_interaction: survival::coxph twice; 2 (log L1 - log L0) of the partial likelihoods, stats::pchisq | %interaction_phreg (PROC PHREG) | the Wald test of the interaction; the interaction of the treatment and a continuous covariate (a factor has levels); the estimate of each level from one model with the interaction terms (the levels are fitted one by one: the analysis in each level); a level without events or values in a group: the fit does not converge, and no p-value is made |
The test words of the plan that no engine computes yet (an analysis that uses one is an error, and its output is not made):
| Test | Why |
|---|
The options and their defaults (what a plan does not say, the engine does as written here):
| Method | Option | Values | Default | How the plan says it |
|---|---|---|---|---|
| catsumm | percent | of the subjects of the column | the population N of the column (header N) | fixed |
| catsumm | events | Y, N | N: Y when the plan asks for Events | the row Events |
| catsumm | worst | a grouping of a variable of the records whose groups are in order, least to most severe | none: every record counts (a subject is in every group of its records) | worst= on the incidence: each subject once in each row, at its worst group (the maximum severity or toxicity grade of a term) |
| catsumm | threshold | a percent from 0 to 100 | none: every row | threshold= on the incidence: the rows with this percent of the subjects of a group or more, in at least one group (the percent itself, not its rounded text) |
| catsumm | order | alphabetical, frequency, frequency in a group | alphabetical (the engines' order of the values) | order= on the incidence: by the subjects of the row, the most first (of all the groups of the columns, or of one group); the rows above it (the system organ classes) keep their order |
| contsumm | quartiles | SAS QNTLDEF 1 to 5 | 5 (the SAS default; R quantile type 2) | fixed |
| anova | variances | equal | equal (the classical F test, SAS PROC GLM) | fixed |
| ancova | estimate | Y, N | N: Y when the plan asks for Estimate, SE or a CI | the rows |
| ancova | level | 0.95 | 0.95 | fixed (alpha= of the plan does not change it) |
| lsmeans | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): "A vs B" is A against B | comparisons= |
| lsmeans | weights | equal | equal weights over the levels of a factor, the covariates at their mean (SAS LSMEANS) | fixed |
| lsmeans | adjust | none | none: no multiplicity adjustment | fixed |
| lsmeans | level | 0.95 | 0.95 | fixed |
| lsmeans | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| mmrm | covariance | us, toeph, toep, ar1h, ar1, csh, cs, adh, ad | the formula's term (required) | covariance= or the term us(AVISIT | USUBJID) |
| mmrm | fallback | structures | none | fallback= |
| mmrm | df | kenward-roger, satterthwaite, residual, between-within | between-within (the engine's default; say it: R and SAS differ). kenward-roger is SAS PROC MIXED DDFM=KR: in R mmrm's "Kenward-Roger" with vcov "Kenward-Roger-Linear" (the covariance parameters taken linearly, as SAS does for UN and CS: the same SE on SAS 9.4 M9, 2026-11-12); mmrm's own "Kenward-Roger" (a Cholesky parameterization) gives 0.607 on the fixture, not the 0.645 of SAS | df= |
| mmrm | estimation | reml, ml | reml | estimation= |
| mmrm | visit | a value of AVISIT | none: required | visit= |
| mmrm | adjust | none | none | fixed |
| mmrm | level | 0.95 | 0.95 | fixed |
| mmrm | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| chisq | correction | none | no continuity correction (SAS "Chi-Square") | fixed |
| fisher | sided | two-sided | two-sided | fixed |
| cmh | statistic | general association; row mean scores | general association; row mean scores when the groups of the category are numbers (SAS CMH2, SCORES=TABLE) | fixed: by the category |
| km | conftype | plain, log, log-log | plain (linear; SAS CONFTYPE=LINEAR), in both engines: what they did before it was an option. Not R survfit's own default (log), not SAS LIFETEST's own (log-log): a plan that wants log-log writes conftype="log-log" | conftype= on the statistics block |
| km | median interval | Brookmeyer-Crowley | Brookmeyer and Crowley (1982): the times whose interval of the survival function (with the transformation of conftype) contains 0.5 (R quantile.survfit; SAS LIFETEST quartile limits) | fixed |
| km | level | 0.95 | 0.95 | fixed |
| km | cnsr | CNSR | CNSR | fixed: the censoring variable is CNSR |
| kmrate | time | a time, in the unit of the analysis variable | none: required (the survival at that time: the estimate of the last event time before it) | time= on the statistics block |
| kmrate | conftype | plain, log, log-log | plain (linear, the same default as the median's; SAS CONFTYPE=LINEAR, kept within 0 and 100). log-log: S ^ exp(-/+ z sigma / log S) (Kalbfleisch-Prentice; SAS LOGLOG, R "log-log"); the standard error of Greenwood | conftype= on the statistics block |
| kmrate | level | 0.95 | 0.95 | fixed |
| kmrate | cnsr | CNSR | CNSR | fixed |
| logrank | weights | rho = 0 | the log-rank weights (SAS TEST=LOGRANK) | fixed |
| logrank | strata | variables | none: the test of all subjects together | strata(X) or strata= |
| logrank | cnsr | CNSR | CNSR | fixed |
| cox | ties | efron, breslow, exact | efron (R coxph's default; SAS PHREG's own default is BRESLOW: the macro writes TIES=EFRON). exact is the discrete likelihood: R coxph "exact", SAS TIES=DISCRETE | ties= |
| cox | pvalue | wald, logrank | wald: the Wald test of the pair in the model. logrank: the log-rank test of the two groups of the pair, stratified by the same strata (the hazard ratio and its CI stay the model's) | pvalue= |
| cox | interval | Wald | Wald: exp(log HR +/- z SE) (SAS PHREG RISKLIMITS and LSMEANS / EXP CL; not the profile likelihood) | fixed |
| cox | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): "A vs B" is A against B | comparisons= |
| cox | level | 0.95 | 0.95 | fixed |
| cox | cnsr | CNSR | CNSR | fixed |
| cox | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| logistic | event | a value of the response | Y for a text response, 1 for a number (the model is of the probability of the event). SAS PROC LOGISTIC alone models the first value in sort order: the macro writes EVENT= | event= |
| logistic | interval | Wald | Wald: exp(log OR +/- z SE) (SAS LSMEANS / DIFF CL, CLODDS=WALD; R confint.default; not the profile likelihood of R confint) | fixed |
| logistic | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): "A vs B" is A against B | comparisons= |
| logistic | level | 0.95 | 0.95 | fixed |
| logistic | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| propci | ci | clopper-pearson, wilson, wald, agresti-coull | clopper-pearson (the exact interval of the binomial distribution; SAS BINOMIAL CL=EXACT). wilson: the score interval without continuity correction; wald and agresti-coull are kept within 0 and 100 | ci= on the statistics block |
| propci | percent | of the subjects of the column | the population N of the column (header N), the same as n (%) | fixed |
| propci | level | 0.95 | 0.95 | fixed |
| riskdiff | ci | wald, newcombe, miettinen-nurminen, agresti-caffo; mh (stratified) | wald (SAS RISKDIFF's own default). newcombe: the hybrid score interval of two Wilson intervals (Newcombe 1998, method 10); miettinen-nurminen: the score interval with the factor N / (N - 1) (SAS CL=MN); agresti-caffo: Wald's with one event and one non-event added to each group. With strata: mh, the Mantel-Haenszel estimate with the variance of Sato (1989; SAS COMMONRISKDIFF(CL=MH)) | ci= |
| riskdiff | event | a value of the response | Y for a text response, 1 for a number | event= |
| riskdiff | pvalue | chi-square | Pearson's chi-square test of the 2 x 2 table without continuity correction (with strata: the CMH test), whatever the interval | fixed |
| riskdiff | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): the difference of "A vs B" is A minus B | comparisons= |
| riskdiff | level | 0.95 | 0.95 | fixed |
| riskdiff | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| riskratio | interval | Wald of the log | exp(log RR +/- z sqrt(1/x1 - 1/n1 + 1/x2 - 1/n2)) (SAS RELRISK CL=WALD, its default) | fixed |
| riskratio | event | a value of the response | Y for a text response, 1 for a number | event= |
| riskratio | pvalue | chi-square | Pearson's chi-square test of the 2 x 2 table without continuity correction | fixed |
| riskratio | level | 0.95 | 0.95 | fixed |
| riskratio | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| oddsratio | interval | Wald of the log | exp(log OR +/- z sqrt(1/a + 1/b + 1/c + 1/d)) (SAS OR CL=WALD, its default) | fixed |
| oddsratio | event | a value of the response | Y for a text response, 1 for a number | event= |
| oddsratio | pvalue | chi-square | Pearson's chi-square test of the 2 x 2 table without continuity correction | fixed |
| oddsratio | level | 0.95 | 0.95 | fixed |
| oddsratio | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| poisson | exposure | a variable, or a variable / a number | none: required (the offset is log(exposure)) | exposure= |
| poisson | interval | Wald | Wald: exp(log RR +/- z SE) (SAS GENMOD LSMEANS / DIFF EXP CL) | fixed |
| poisson | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): "A vs B" is A against B | comparisons= |
| poisson | level | 0.95 | 0.95 | fixed |
| poisson | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| negbin | exposure | a variable, or a variable / a number | none: required (the offset is log(exposure)) | exposure= |
| negbin | dispersion | k by maximum likelihood | k (var = mu + k mu^2) by maximum likelihood (SAS GENMOD DIST=NB; MASS::glm.nb, theta = 1 / k); the standard errors from the observed information of the coefficients and k together (SAS GENMOD's default; glm.nb's own is the expected information at a fixed k) | fixed |
| negbin | interval | Wald | Wald: exp(log RR +/- z SE) (SAS GENMOD LSMEANS / DIFF EXP CL) | fixed |
| negbin | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): "A vs B" is A against B | comparisons= |
| negbin | level | 0.95 | 0.95 | fixed |
| negbin | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| eair | exposure | a variable of ADSL, or a variable / a number | none: required. The exposure of a group is the sum of the exposure of all its subjects (not the time to the first event) | exposure= |
| eair | per | a number | 100: per 100 patient-years when the exposure is in years | per= |
| eair | interval | exact Poisson | the exact (Garwood) interval: the chi-square quantiles of 2n and 2(n + 1) degrees of freedom, halved, divided by the exposure (SAS CINV) | fixed |
| eair | level | 0.95 | 0.95 | fixed |
| eventrate | count | subjects, events | subjects (the incidence rate: eair); events: every record of the group | count= on the statistics block |
| eventrate | exposure | a variable of ADSL, or a variable / a number | none: required. The exposure of a group is the sum of the exposure of all its subjects | exposure= |
| eventrate | per | a number | 100: per 100 patient-years when the exposure is in years | per= |
| eventrate | interval | exact Poisson | the exact (Garwood) interval of the number of events: the chi-square quantiles of 2e and 2(e + 1) degrees of freedom, halved, divided by the exposure (SAS CINV); it takes the events as independent (no overdispersion) | fixed |
| eventrate | level | 0.95 | 0.95 | fixed |
| perterm | ci | wald, newcombe, miettinen-nurminen, agresti-caffo (riskdiff) | wald (as the riskdiff method); a relative risk or an odds ratio: the Wald interval of its logarithm | ci= on the incidence |
| perterm | pvalue | chi-square, none | the chi-square test of the 2 x 2 table of each row without continuity correction (nominal); pvalue="false": none | pvalue= on the incidence |
| perterm | threshold | a percent | none: every row; the incidence's threshold= also selects the rows of the comparison | threshold= on the incidence |
| perterm | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:): the difference of "A vs B" is A minus B | comparisons= on the incidence |
| perterm | level | 0.95 | 0.95 | fixed |
| perterm | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| ttest | variances | equal, unequal | equal: Student's t-test with the pooled variance of the two groups (n1 + n2 - 2 df; SAS TTEST "Pooled"). unequal: Welch's (Satterthwaite's df; SAS "Satterthwaite"). R t.test alone uses unequal | variances= |
| ttest | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): the difference of "A vs B" is A minus B | comparisons= |
| ttest | level | 0.95 | 0.95 | fixed |
| ttest | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| wilcoxon | test | normal approximation | the normal approximation, the variance corrected for ties (SAS NPAR1WAY; R wilcox.test(exact = FALSE)); not the exact test | fixed |
| wilcoxon | correction | yes, no | yes: the continuity correction 0.5 (SAS CORRECT=YES, its default; R correct = TRUE) | correction= |
| wilcoxon | interval | Moses | the asymptotic interval of Moses: the differences of order C and n1 n2 + 1 - C, C = n1 n2 / 2 - z sqrt(n1 n2 (n1 + n2 + 1) / 12) to the nearest integer (SAS NPAR1WAY HL). R wilcox.test's own interval inverts the corrected test and can differ | fixed |
| wilcoxon | comparisons | pairs "A vs B" of the groups | each other group against the reference group (reference: of treatments:, required as soon as a model compares pairs): the shift of "A vs B" is A minus B | comparisons= |
| wilcoxon | level | 0.95 | 0.95 | fixed |
| wilcoxon | adjust | none, bonferroni, holm, hochberg, hierarchy | none: the p-value of each pair unadjusted. A multiplicity block whose hypotheses name pairs of this analysis (comparison=) adds the adjusted p-value of each (Adjusted p-value), by its procedure (R: stats::p.adjust; SAS: PROC MULTTEST INPVALUES=) | a multiplicity block |
| interaction-lm | test | F | the F test of the treatment-by-factor terms: the model of the analysis with the factor against the same with the terms (the residual SS of the two nested models; SAS PROC GLM, the Type III F test of the interaction, the same for the term of the highest order) | fixed |
| interaction-lm | level | a number, none, open | none: the plan states it (interaction="0.10"), or says that no test is made (interaction="none", with the reason) | interaction= on the subgroup block |
| interaction-logistic | test | likelihood ratio | the likelihood-ratio test: 2 (log L1 - log L0) of the model of the analysis with the factor (L0) and the same with the treatment-by-factor terms (L1), chi-square with the number of terms the data estimate (not the Wald test) | fixed |
| interaction-logistic | level | a number, none, open | none: the plan states it (interaction="0.10"), or says that no test is made (interaction="none", with the reason) | interaction= on the subgroup block |
| interaction-poisson | test | likelihood ratio | the likelihood-ratio test: 2 (log L1 - log L0) of the model of the analysis with the factor (L0) and the same with the treatment-by-factor terms (L1), chi-square with the number of terms the data estimate (not the Wald test) | fixed |
| interaction-poisson | level | a number, none, open | none: the plan states it (interaction="0.10"), or says that no test is made (interaction="none", with the reason) | interaction= on the subgroup block |
| interaction-negbin | test | likelihood ratio | the likelihood-ratio test: 2 (log L1 - log L0) of the model of the analysis with the factor (L0) and the same with the treatment-by-factor terms (L1), chi-square with the number of terms the data estimate; the dispersion k is estimated in each model (not the Wald test) | fixed |
| interaction-negbin | level | a number, none, open | none: the plan states it (interaction="0.10"), or says that no test is made (interaction="none", with the reason) | interaction= on the subgroup block |
| interaction-cox | test | likelihood ratio | the likelihood-ratio test: 2 (log L1 - log L0) of the model of the analysis with the factor (L0) and the same with the treatment-by-factor terms (L1), chi-square with the number of terms the data estimate (not the Wald test) | fixed |
| interaction-cox | level | a number, none, open | none: the plan states it (interaction="0.10"), or says that no test is made (interaction="none", with the reason) | interaction= on the subgroup block |
| interaction-cox | ties | efron, breslow, exact | those of the analysis (efron) | ties= of the analysis |
The known answer of each method (a small data set of 12 subjects, FIXTURE_STUDY of src/methods.ts; the numbers by tools/method_answers.R; test/methods-r.test.ts runs every row through the real contract maker and R engine):
| Method | Where the expected numbers come from |
|---|---|
| count | counted by hand: 4 subjects in each group |
| catsumm | counted by hand: the subjects with an adverse event of each system organ class, their percent of 4, the events |
| contsumm | by hand: the ages 70, 72, 75, 81 (Placebo), 65, 68, 77, 79 (Low), 60, 71, 74, 85 (High); the quartiles of SAS definition 5 |
| anova | by hand: SSB 32 (2 df), SSW 15 (9 df), F = 9.6; p = pf(9.6, 2, 9, lower.tail = FALSE) |
| ancova | by hand: (X'X)^-1 X'y of CHG ~ BASE + TRTPN at Week 24, its t interval and t test (9 df); tools/method_answers.R |
| lsmeans | the textbook one-way layout (CHG ~ TRTP at Week 24): the LS means are the means; SE = sqrt(MSE (1/4 + 1/4)), t with 9 df; tools/method_answers.R |
| mmrm | the textbook result for complete balanced data with an unstructured covariance and the treatment by visit means: the LS means at Week 24 are the means, the SE is that of the one-way layout at Week 24 (REML: the pooled covariance, 9 df; df=kenward-roger here, which SAS DDFM=KR gives without a change; see the option df); tools/method_answers.R |
| chisq | by hand: the 3 x 2 table of sex (2/2, 1/3, 3/1), every expected count 2, X2 = 2 with 2 df, p = exp(-1) |
| fisher | stats::fisher.test of the 3 x 2 table of sex, computed once (tools/method_answers.R) |
| cmh | stats::mantelhaen.test (an independent program) of sex by group in the strata of SITEGR1 and RACE (each combination a stratum), computed once (tools/method_answers.R); in the strata of SITEGR1 alone: 0.3916 |
| km | the medians by hand (30, 35, 18 days); the confidence limits from survival::survfit(conf.type = "plain"), computed once (tools/method_answers.R): with 4 subjects the upper limit is not reached (NaN: not estimable) |
| kmrate | by hand: the product-limit estimate at 24 days written out (66.7, 75.0, 37.5%), Greenwood's sum and the log-log interval (tools/method_answers.R); survival::survfit gives the same |
| logrank | by hand: the log-rank statistic written out (observed minus expected events and their hypergeometric covariance, summed over the times and the strata of SEX and AGEGR1; tools/method_answers.R); survival::survdiff gives the same. Unstratified: survival::survdiff of the three groups, computed once |
| cox | by hand: the Efron partial likelihood of TRTDUR (tied times) on the groups in the strata of SEX, its score and information written out, Newton to the maximum; each pair a contrast of the coefficients with its Wald interval (tools/method_answers.R; survival::coxph gives the same) |
| logistic | by hand: the binomial likelihood of COMP24FL = "Y" on the groups and AGE written out, Newton (X'WX) to the maximum; each pair a contrast of the coefficients with its Wald interval (tools/method_answers.R; stats::glm gives the same) |
| propci | stats::binom.test (the Clopper-Pearson interval) of 1, 2 and 3 of 4 subjects with COMP24FL = "Y" (tools/method_answers.R); the methods against published examples: r-agentic tests/testthat/test-ars_engines_methods.R (Newcombe 1998, 81 of 263) |
| riskdiff | by hand: the Wilson intervals of stats::prop.test (without correction) of 1, 2 and 3 of 4 combined by Newcombe's method 10; Pearson's chi-square by stats::chisq.test (tools/method_answers.R). The four intervals against Newcombe (1998, Stat Med 17:873, Table II): r-agentic tests/testthat/test-ars_engines_methods.R |
| riskratio | by hand: p1 / p2 of 1, 2 and 3 of 4 and the Wald interval of its logarithm; Pearson's chi-square by stats::chisq.test (tools/method_answers.R) |
| oddsratio | by hand: (x1 (n2 - x2)) / (x2 (n1 - x1)) of 1, 2 and 3 of 4 and the Wald interval of its logarithm; Pearson's chi-square by stats::chisq.test (tools/method_answers.R) |
| poisson | by hand: the rate ratio of two groups in a model of the groups alone is the ratio of their rates (the sum of the counts of VISNUMEN over the sum of the exposures TRTDUR / 365.25), the SE of its logarithm sqrt(1 / y1 + 1 / y2) (tools/method_answers.R) |
| negbin | MASS::glm.nb of VISNUMEN with the offset log(TRTDUR / 365.25) for the estimate (k = 0.682); the standard errors from the numerical second derivatives of the negative binomial log likelihood in the coefficients and k (tools/method_answers.R) |
| eair | by hand: the subjects with an adverse event (2, 3, 4) over the sum of TRTDUR / 365.25 of the 4 subjects of each group, times 100; the interval from stats::qchisq of 2n and 2(n + 1) degrees of freedom (tools/method_answers.R) |
| eventrate | by hand: the adverse events (3, 3, 4 records) over the sum of TRTDUR / 365.25 of the 4 subjects of each group, times 100; the interval from stats::qchisq of 2e and 2(e + 1) degrees of freedom (tools/method_answers.R) |
| perterm | counted by hand: the subjects with each preferred term of the 4 of each group; Newcombe's interval (method 10) from the two Wilson score intervals written out; Pearson's X2 of the 2 x 2 table by hand (tools/method_answers.R) |
| ttest | by hand: the difference of the means of CHG at Week 24 of each pair, the pooled SD of the two groups, t with 6 df (tools/method_answers.R) |
| wilcoxon | by hand: the midranks of CHG at Week 24 of each pair (ties between the groups), the rank sum, its variance corrected for ties, z with the continuity correction; the median of the 16 differences and the differences of order C = 2 and 15 (tools/method_answers.R; stats::wilcox.test gives the same p-value) |
| interaction-lm | by hand: the residual sums of squares of CHG at Week 24 on BASE, the groups and RACE, and with the group-by-RACE terms, from the normal equations; F with 2 and 5 df; the factor RACE (two subjects of each group in each level) (tools/method_answers.R) |
| interaction-logistic | by hand: the binomial likelihood of COMP16FL = "Y" on the groups, AGE and RACE, and with the group-by-RACE terms, Newton's method to the maximum of each; 2 df; the factor RACE (two subjects of each group in each level) (tools/method_answers.R) |
| interaction-poisson | by hand: the Poisson likelihood of VISNUMEN with the offset log(TRTDUR / 365.25) on the groups and RACE, and with the group-by-RACE terms, Newton's method; 2 df; the factor RACE (two subjects of each group in each level) (tools/method_answers.R) |
| interaction-negbin | by hand: the negative binomial likelihood of VISNUMEN (the offset log(TRTDUR / 365.25)) on the groups and RACE, and with the group-by-RACE terms, its coefficients and k by stats::optim (no glm.nb); 2 df; the factor RACE (two subjects of each group in each level) (tools/method_answers.R) |
| interaction-cox | by hand: the Efron partial likelihood of TRTDUR (tied times) on the groups and RACE, and with the group-by-RACE terms, Newton's method to the maximum of each; 2 df (Breslow: 0.6883); the factor RACE (two subjects of each group in each level) (tools/method_answers.R) |
Tables and analyses#
A table of the plan is the block tlf (table, listing or figure). Its analyses are its rows, in order:
:::: {.tlf #P_DEMOG population="ENR" total="true"}
**Demographics and Patient Characteristics Measured at Baseline**
::: {.summarize #P_AGE variable="ADSL.AGE" test="anova"}
Age (years)
:::
::: {.tabulate #P_SEX variable="ADSL.SEX" test="chisq"}
Gender
:::
::: {.tabulate #P_RACE variable="ADSL.RACE" test="chisq"}
Origin
:::
^[The p-values are descriptive: the treatment groups were randomized.]
::::
The footnote ^[...] of a table is a footnote of its shell.
Attribute of tlf | Meaning | Default |
|---|---|---|
population= | the population of the table and its analyses | required |
total="true" | a Total column | no |
columns= | the column grouping; it is also the first grouping (by:) of every analysis of the table, as the chain wants | TRT (treatments:) |
pvalue="true" / pvalue="false" | p-value columns | yes when an analysis has a test |
template= | the company's table template (or the SAP's: template="SAP Template 5") | |
figure="true" | the output is a figure: type: figure in the structured SAP; its template (Appendix A) is headed FIGURE, with a frame that says what it draws (for a Surv() model: the Kaplan-Meier curves by group, the number at risk) above the table of its estimates | a table |
| Row block | For | Library pattern |
|---|---|---|
::: {.summarize #ID variable="DS.VAR" test="..." stats="..."}, the title in the body | a continuous variable by treatment | cont_by_trt |
::: {.tabulate #ID variable="DS.VAR" test="..." categories="a; b"}, the title in the body | a categorical variable by treatment | cat_by_trt |
::: {.incidence #ID condition="..." test="..." rows="soc;pt" comparisons="..." stats="..."}, the title in the body | subjects with events by SOC and PT; rows="soc" by SOC only, rows="pt" by PT only, rows="any" one row of all events (an overview row); stats="CAT_SUMM" n (%) without the number of events; comparisons="A vs Placebo; B vs Placebo" Fisher's exact test of each pair; worst=, threshold=, order=, test="riskdiff" with ci= and pvalue=: Adverse event tables | ae_incidence |
::: {.byvisit #ID dataset="DS" timepoint="true" visits="..." records="..."}, the title in the body | a BDS parameter by visit, observed and change | cfb_by_visit |
::: {.analysis #ID}, the title in bold, then its parts | anything else |
The parts of an analysis: []{.data dataset="DS" records="..."} (the data set, and its records as a subset
<id>_REC), []{.variable variable="CHG"}, []{.by groupings="G1; G2"}, []{.stats method="METHOD"}, and the model
(a formula span or a model div).
Models: a formula or an equation#
::::: {.tlf #T_MODEL population="EFF"}
**Two forms of one model**
:::: {.analysis #A_FORMULA}
**ADAS-Cog (11), change from baseline to Week 24: the formula (R, Wilkinson)**
[]{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "Week 24" and ANL01FL = "Y"'}
[]{.formula formula="CHG ~ BASE + SITEGR1 + TRTPN"}
::::
:::: {.analysis #A_EQUATION}
**ADAS-Cog (11), change from baseline to Week 24: the equation, with the meaning of its symbols**
[]{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "Week 24" and ANL01FL = "Y"'}
::: {.model}
$$
Y_{i} = \mu + \beta\,\mathrm{BASE}_{i} + \gamma_{s(i)} + \tau\, d_{i} + \varepsilon_{i}
$$
[]{.where bindings="Y = CHG, s = SITEGR1, d = TRTPN"}
:::
::::
:::::
The model div is a numbered equation; the where span binds each symbol to an ADaM variable ($\mathrm{BASE}$ needs
no binding). The compiler reads the terms: $\mu$ the intercept; a coefficient times a variable
($\beta\,\mathrm{BASE}_i$, $\tau\, d_i$) a covariate, or the dose when the variable is the treatment; an effect with an
index ($\gamma_{s(i)}$, bound by $\gamma$ = ... or s = ...) a class factor; $\varepsilon_i$ the error. A formula
span inside the model is checked against the equation. Round trip from the command line:
node src/cli.ts math "CHG ~ BASE + SITEGR1 + TRTPN" prints the equation;
node src/cli.ts math --tex "<equation>" --where "Y = CHG, d = TRTPN" prints the formula.
| Model (types from the ADaM specification) | Method |
|---|---|
| numeric response, a numeric covariate, the treatment | ancova |
| numeric response, factors and the treatment only | anova; test="ttest" or test="wilcoxon" (treatment only): the t-test, or the Wilcoxon test with the Hodges-Lehmann shift and its CI, of each pair of groups |
a count with test="poisson" or test="negbin" and exposure= | the rate ratio of each pair of groups (the offset log(exposure)) |
| text response ~ treatment | chisq (test="fisher"); test="riskdiff", "riskratio", "oddsratio": the risk difference, the relative risk, the odds ratio of each pair of groups with its CI |
| text response ~ treatment | strata | cmh |
| text response ~ treatment + covariates | logistic (the odds ratio of each pair of groups; test="logistic" without covariates) |
Surv(AVAL, CNSR) ~ treatment | logrank; with covariates, or test="cox": cox (the hazard ratio of each pair of groups) |
a covariance term us(AVISIT | USUBJID) | mmrm |
The treatment term is the variable of treatments: or its analysis twin (TRTP, TRTPN, TRTA, TRTAN, TRT01P,
...). Numeric (TRTPN: 0, 54, 81): the test for linear dose response, method id <analysis>_DR. Text (TRTP):
the overall test of the groups, <analysis>_CMP; contrast="pairwise" asks for the pairwise comparisons. The ANCOVA
above compiles to:
id: A_FORMULA_DR
name: "ANCOVA, test for linear dose response"
test: ancova
statistics:
- {op: pvalue, pattern: "x.xxx"}
model: {dose: ADQSADAS.TRTPN, covariates: [ADQSADAS.BASE], factors: [ADQSADAS.SITEGR1]}
sided: 2
alpha: 0.05
formula: "CHG ~ BASE + SITEGR1 + TRTPN"
Estimands#
::: {.estimand #EST1 level="primary" population="EFF" analysis="P_ADAS"}
**Group mean changes from baseline in the primary efficacy parameters**
[Xanomeline TTS at 54 mg or 81 mg, or placebo, for 24 weeks]{.treatment}
[Change from baseline to Week 24 in the ADAS-Cog (11) total score]{.endpoint}
[Temporary treatment interruption]{.intercurrent strategy="treatment policy"}
[]{.cite source="usdm:estimands" words="Estimand EST1 ... regardless of interruption."}
:::
The five attributes of ICH E9(R1): population=, the treatment span, the endpoint span (the variable),
[event]{.intercurrent strategy="..."} (one per event; strategies treatment policy, hypothetical, composite,
while on treatment, principal stratum), the summary span (without it, written from the model of analysis=).
level= is primary, key secondary, secondary or exploratory; when no estimand says its level, the first
estimand is the primary one. The rules e9r1.* check the attributes, the strategy, death under treatment policy, and
that a primary estimand names its analysis in its population (rules.md). The structured SAP has an
estimand block (the chain does not read it yet; its source is under source_section/source_quote, which the
chain's checker does not claim to check).
What the newer SAPs add to an estimand (read from ten public SAPs, 2026-11):
| You write | It means |
|---|---|
[Rescue medication]{.intercurrent strategy="hypothetical" reason="why this strategy" data='ADSL.RESCFL = "Y"'} | the event with the reason of its strategy and its rule in the data; both are printed in the estimand box and kept in the JSON (intercurrent_events[].reason, .data) |
::: {.estimand #EST2 level="primary" role="supportive" of="EST1" ...} | a second estimand on the same endpoint (other strategies, its own analysis=). It is not a second hypothesis: the rule gamble.17 does not ask for multiplicity between an estimand and its supportive one. role= is main or supportive; a supportive estimand names its main one with of= |
target="Patients with mild to moderate hypertension" | the target population in words (ICH E9(R1): the population attribute); population= stays the analysis set |
::: {.missingdata #MD2 estimand="EST2" after="Rescue medication" pattern="endpoint missing" assumption="MAR" handling="multiple imputation" imputations="100" seed="20261111"}, the reason in the body | the handling of the missing values of an estimand, after one of its intercurrent events or for a pattern. after= must be an event of the estimand; with imputations= the plan gives its seed= |
What the SAP adds to the protocol#
The protocol is cited for what the protocol says. What it does not say is either open (a decision) or the
statistician's addition: a population the protocol does not define, a crude rate, a rule of the time to event, an
interim look an agency asked for. An addition never borrows the nearest protocol sentence: it is named by a change
block with what the SAP adds and why, and that change is its source.
::: {.population #MITT condition="ADSL.EFFFL" definition="The randomised patients with a centrally confirmed diagnosis"}
Modified intent-to-treat population
:::
::: {.change #CH_MITT adds="MITT"}
**The SAP adds the MITT population.**
The central confirmation of the diagnosis is the SAP's (SAP 5.2); the protocol names the ITT population only.
:::
adds=names blocks of any kind:population,datasubset,grouping,derivation,statistics,estimand,tlf,analysis, a plan block (interim,sensitivity, ...). A block inside an added block (the method and records of an added analysis) is added with it. An added block needs no citation (nono-source); the change withadds=needs none either (its reason is the source). A change that alters what the protocol says still cites it. A block named byadds=that also cites the protocol is a warning (addition-cited): one of the two is not true.- The rendered plan shows each added block as an addition (the row "Added by the SAP" in its box, with its change),
and lists every change in one place before the table shells: "Changes from the protocol", with the blocks each adds
and its reason. The JSON has
changes[]; the review counts the additions (SAPLING REVIEW: ..., 2 addition(s) of the SAP to the protocol) and lists them under "CHANGES FROM THE PROTOCOL". - In the structured SAP an added block has
source: sapandadded_by: CH_MITTinstead of a protocol quote; the checker of r-agentic (sapling build) takes it. The SAS project's checker (sas-macro-agentic, read only) asks a quote of every block:sapc chainreports the added blocks there. - Only the statistician writes
adds=: the MCP server, the assistant and the editor's agent refuse a change withadds=(src/service.tsguard). An agent proposes the addition in adecision.
Decisions#
::: {.decision}
The protocol's investigator effect is the pooled site SITEGR1. The statistician confirms the pooling rule.
:::
[]{.cite source="protocol:4.3.3" words="For all ANOVA and ANCOVA models"}
An open decision is a todo block with its quote (the chain counts it for the approver). When a person has decided,
that person adds answer="...", by="Name" and on="2026-12-01" to the div (sapling decide writes them with
the person's name); it becomes a decision block. The compiler never writes by=; an agent never writes it.
A decision cites the protocol sentence that leaves the question open. When the protocol is silent, the decision cites
nothing and says so in its text: a decision is the one block that needs no citation (a guessed section would send the
reader to the wrong place). The skeleton of sapling new writes "The protocol is silent on it, or its section was not
found" in the decision of each part whose section it did not find.
The design plan and the assumptions block#
A design plan is a plan written before the protocol is final: the statistician's design of the statistics (docs/design.md).
It is the same language and the same compiler as a SAP. The one difference is kind: design under sapling:: with it
the plan has no protocol to cite, and that is said once, as information, instead of a warning. Every other rule applies
(a design plan that names a protocol: is checked against it like a SAP). sapling design to-sap makes a SAP from it.
sapling:
kind: design # sap (the default) or design
study: HTN-201
The one new block is the assumptions block: what the simulation needs and what the protocol's sample size section
states. A scenario overrides some of its numbers. Every number has a basis= (a publication, an earlier study, an
expert, "a guess"); a part without one is warned, because the protocol must justify it.
::: {.assumptions #A_BASE analysis="P_SBP" samplesize="SS_SBP"}
Why these numbers (prose, optional).
[Change in SBP (mmHg)]{.effect difference="-6" sd="12" control="-3" profile="0.5; 0.8; 1" baseline_coef="-0.2" basis="earlier trial"}
[Correlation of the visits]{.correlation rho="0.6" structure="ar1" basis="guess"}
[Dropout by Week 12]{.dropout rate="0.15" mechanism="MAR" slope="0.6" basis="the sample size"}
[Treatment discontinuation]{.ice rate="0.08; 0.20" after="effect-lost" retain="0" collected="yes" basis="guess"}
[Pessimistic effect]{.scenario #PESS difference="-4.5" dropout="0.20" curve="true" basis="three quarters of the effect"}
:::
| Part | Attributes | Meaning |
|---|---|---|
block assumptions | analysis= (the primary analysis), samplesize= (the block whose n= is simulated), n= (when there is no sample size block) | one per design |
effect | difference=, sd=, control=, profile=, baseline_coef= (a mean at the last visit: the SD is that of the change after the baseline); rates="p0; p1" (a response); median=, hr= (time to event); basis= | the true effect (active minus control); profile is the share of it at each visit; the control's change at the last visit is control |
correlation | rho=, structure= (ar1, cs), basis=; between="endpoints": the correlation of the endpoints of a multiplicity family | repeated measures; endpoints |
accrual | months=, followup=, basis= | time to event: uniform accrual, then follow-up |
dropout | rate= (one number, or one per group: the share lost by the end), mechanism= (MCAR; MAR: the hazard depends on the last value, slope=; MNAR: also the values nobody sees after dropout shift by delta=, per group, positive = worse), basis= | time to event and response: MCAR only |
ice | rate= (by the end, per group), after= (effect-lost, rescue with shift=, none), retain= (the share of the effect kept after the event), collected= (yes: the visits after it are collected), basis=; the text is the name of the event | an intercurrent event of the estimands (its strategy is in the estimand block) |
effect hypothesis="H1" | difference=, sd=, basis= | one endpoint of a multiplicity family (one per hypothesis of the multiplicity block, in place of the single effect) |
scenario #ID | any of difference sd control rates median hr rho dropout mechanism delta ice_rate retain n, zero="H2; H3" (the hypotheses whose effect is zero in it: a partial null), curve="true" (also a power curve), basis= | a named scenario; NULL (no effect: the type I error) is always run |
sapling check reads the block: a number out of range, the wrong number of per-group values, MNAR without delta=, a
rescue event without shift=, a scenario named twice or named NULL, an effect without its sd= are errors with their
fix; a part without basis= is a warning.
The plan around the analyses: design, sample size, multiplicity, missing data#
These blocks state what a SAP must say beyond its tables (Gamble et al. 2017, ICH E9(R1)). The structured SAP of the
chain has no place for them yet: they go to the JSON (sapc json, "in_structured_sap": false), the preview and the
PDF. Each cites the protocol like any block; a block the SAP adds (not in the protocol) is named by a change block
(adds=): What the SAP adds to the protocol.
::: {.design type="parallel" allocation="1:1:1" blinding="double" framework="superiority" strata="ADSL.SITEGR1"}
:::
[]{.cite source="protocol:3.1"}
::: {.samplesize #SS_ADAS n="300" power="0.90" alpha="0.05" sided="2" effect="3.0" sd="6.5" dropout="0.10" test="ttest"}
:::
[]{.cite source="protocol:3.4.4" words="Previous experience ... standard deviation of 6.5"}
::: {.multiplicity #MULT procedure="graphical" alpha="0.05"}
**The primary and key secondary hypotheses**
[ADAS-Cog (11) dose response]{.hypothesis #H1 analysis="P_ADAS" weight="0.5"}
[CIBIC+ dose response]{.hypothesis #H2 analysis="EST2" weight="0.5"}
[]{.transition from="H1" to="H2" weight="1"}
[]{.transition from="H2" to="H1" weight="1"}
:::
::: {.missingdata #MD_ADAS analysis="P_ADAS" assumption="MAR" handling="mmrm"}
The MMRM uses all observed values; under MAR a patient who stops is like a patient who stays with the same history.
:::
::: {.sensitivity #SENS_TP estimand="EST1" method="tipping-point" assumption="MNAR" delta="0 to 6 by 0.5"
imputations="100" seed="20261109"}
**Tipping point**
The imputed values of the active groups are made worse by delta until the conclusion changes.
:::
::: {.supplementary #SUP_COMP estimand="EST1" population="COMP24" method="completers"}
**Completers**
The same analysis in the patients who complete Week 24.
:::
::: {.interim #IA1 timing="50% of the patients complete Week 8" purpose="futility" futility="non-binding"
spending="none" committee="the DMC"}
A look for futility only.
:::
::: {.software name="R" version="4.4.1" packages="mmrm 0.3.14; emmeans 1.10.7"}
:::
::: {.dictionary name="MedDRA" version="27.1" for="adverse events"}
:::
::: {.change adds="SENS_TP"}
**The SAP adds a tipping point analysis.**
LOCF is not a sensitivity analysis (EMA 2010, 7).
:::
[]{.cite source="protocol:4.3.2"}
| Block | Attributes, body | What the compiler checks | Where it goes |
|---|---|---|---|
::: {.design type="..."} | type=: parallel, crossover, factorial, cluster, single-arm, adaptive; allocation="1:1:1" (one number per group of treatments:), blinding= (open, single, double, triple), framework= (superiority, non-inferiority with margin=, equivalence), strata="ADSL.X; ..." (ADaM variables), randomisation= (words) | the ratio against the groups, the margin of a non-inferiority design, the strata variables | JSON, PDF |
::: {.samplesize #ID n="N"} | n=: the total number; power=, alpha=, sided=, effect=, sd=, rates="p1; p2", hr=, events=, dropout=, test= (the test it assumes), per_group= (one number, or one per group: "217; 433"), design= (two-sample, the default; paired, one-sample, crossover, cluster), margin=, joint_power= (co-primary endpoints), analysis= | the numbers; for a comparison of the means of equal groups (ttest, ancova, anova) it recomputes N (normal approximation) and warns when N is smaller; a design it cannot model (paired, one-sample, a margin, unequal groups) is not recomputed, and recomputed says why | JSON (with recomputed), PDF |
::: {.multiplicity #ID procedure="..." alpha="..."}, the title in bold | procedure=: hierarchy (the order of the hypotheses), graphical (Bretz et al. 2009: weight= on each hypothesis, transition spans), bonferroni, holm, hochberg, co-primary (all must be rejected), none (no adjustment, with the reason: below) | weights add up to 1 at most, out of each hypothesis too; transitions name hypotheses; a graphical procedure without transitions | JSON, PDF (the graph drawn) |
[words]{.hypothesis #H1 analysis="..." weight="..." alpha="..." comparison="A vs B" final_alpha="..."} | the family of hypotheses, in a multiplicity block; analysis= names an analysis or an estimand; comparison= a pair of that analysis (its comparisons=); final_alpha= the nominal alpha of its final analysis after interim looks | the analysis or estimand exists; the pair is one of the analysis; the alphas of a Bonferroni split add up to the family's at most; final_alpha= against the computed final boundary | |
[]{.transition from="H1" to="H2" weight="..."} | the share of H1's alpha that goes to H2 when H1 is rejected | ||
::: {.missingdata #ID analysis="..." estimand="..." after="..." pattern="..." assumption="..." handling="..." imputations="..." seed="..."}, the justification in the body | assumption= MAR, MNAR, MCAR; handling= mmrm, multiple imputation, reference-based, locf, ... | the analysis exists | JSON, PDF |
::: {.sensitivity #ID estimand="..." method="..." assumption="..." delta="..." reference="..." imputations="..." seed="..." analysis="..."}, the title in bold | an analysis of the same estimand under other assumptions: method= tipping-point, j2r, cr, cir, delta-adjustment, mi-mar, ... | an estimand (required); a reference group for J2R/CR/CIR (a group of treatments:); the range of a tipping point; a seed for multiple imputation | JSON, PDF |
::: {.supplementary #ID estimand="..." population="..." analysis="..." method="..."}, the title in bold | another estimand or analysis set | an estimand, population or analysis | JSON, PDF |
::: {.subgroup #ID analyses="..." factors="..."}, the title in bold | the subgroup analyses of one or more analyses: factors= (groupings), pvalue=, interaction= (a level, none, open), minimum=, forest=, level= (key, exploratory) (below) | the analyses compare pairs of groups; the factors are groupings of ADSL or of the analysis's data set; an interaction test has an engine for the model; interaction="none" has its reason | the structured SAP (its analyses, methods and table), JSON, PDF (a sentence and the table of its analyses) |
::: {.interim #ID}, the words in the body | hypotheses= (the hypotheses of multiplicity blocks it tests), information= (fraction), events=, timing=, purpose= (futility, efficacy, ...), futility= (non-binding, binding), futility_z=, spending= (obrien-fleming, pocock, hwang-shih-decani with gamma=, haybittle-peto, fixed, none), alpha= or z= (a stated boundary), stop= (all, any), committee= | a futility look says whether it binds; an efficacy look has its spending and names its hypotheses; the boundaries of each look are computed and compared with the stated numbers (below) | JSON (with boundaries), PDF (the boundary table) |
::: {.software name="R" version="..." packages="..." validation="..."} | the software, its version, the packages | a version | JSON, PDF |
::: {.dictionary name="MedDRA" version="..." for="..."} | MedDRA, WHODrug, CTCAE | a version | JSON, PDF |
::: {.change #ID adds="..." version="..." date="..."}, what in bold, then the reason | a change from the protocol; adds="ID; ...": the blocks the SAP adds, of any kind (their source is the change) | a reason; each id of adds= a block of the plan | JSON, PDF, the list "Changes from the protocol" |
The adjusted p-values. When every hypothesis of a multiplicity block names the same analysis and one of its pairs
(comparison=), and the procedure is bonferroni (weighted by weight= when the hypotheses have weights), holm,
hochberg or hierarchy (the fixed sequence: the adjusted p-value of a hypothesis is the largest p-value of it and of
those before it; it is rejected when that is at most alpha), the engines make the adjusted p-value of each pair (the
row Adjusted p-value; R stats::p.adjust, SAS %adjust, PROC MULTTEST INPVALUES=). The method block of the analysis
gets adjust: {procedure, family} and the statistic padj. "Hochberg over two doses":
::: {.multiplicity #MULT procedure="hochberg" alpha="0.05"}
**The two doses against placebo**
[High dose]{.hypothesis #H1 analysis="A_RESP" comparison="High Dose vs Placebo"}
[Low dose]{.hypothesis #H2 analysis="A_RESP" comparison="Low Dose vs Placebo"}
:::
A family over several analyses, a graphical procedure, Dunnett's test and the co-primary criterion are stated in the plan; the engines do not adjust them (an error says so when a hypothesis names a pair).
None, as data. A plan says "no adjustment for multiplicity", "no interim analysis is planned" or "no sensitivity
analysis is planned" with the value none of the attribute that says what is done, and the reason (in the body) or
the protocol's sentence (a citation after the block, or of the heading of its section):
::: {.multiplicity #MULT procedure="none"}
**No adjustment for multiplicity**
The study is a dose-finding study: each dose is compared with placebo at the two-sided 5% level.
:::
[]{.cite source="protocol:4.3.3" words="Effects in the ANCOVA model"}
::: {.interim #IA purpose="none"}
No interim analysis is planned.
:::
::: {.sensitivity #SENS_NONE estimand="EST1" method="none"}
**No sensitivity analysis**
The primary analysis is descriptive: it tests no assumption about the missing values.
:::
| Form | Means | The review |
|---|---|---|
procedure="none" on multiplicity | no adjustment: the hypotheses (none, or the family it names) are each tested at the full level | gamble.17 asks for no other strategy |
purpose="none" on interim | no interim analysis is planned | gamble.13a, spirit.28b: addressed; gamble.13b, gamble.13c, spirit.28a: not applicable |
method="none" on sensitivity | no sensitivity analysis is planned, of estimand= (or of every estimand without it) | gamble.27e: addressed for that estimand; it is not an additional analysis (spirit.27d) |
A none block without a reason and without a citation is an error. The review no longer says "the protocol plans
interim analyses" when every sentence of the protocol on interim analyses says there is none ("No interim analyses
... are planned").
Subgroup analyses#
One block declares the subgroup analyses of one or more analyses: the factors, what each level reports, the test of
the interaction of the treatment and each factor, the minimum size of a level and the forest plot. Each factor is a
grouping (its variable and its levels, from the plan or from the data).
::: {.grouping #SG_SEX variable="ADSL.SEX" groups="F; M"}
Sex
:::
::: {.grouping #SG_AGE variable="ADSL.AGEGR1" groups="<65; 65-80; >80"}
Age group
:::
::: {.subgroup #SG_PRIM analyses="A_ANCOVA" factors="SG_SEX; SG_AGE" interaction="0.10" minimum="10"
level="exploratory"}
**Subgroup analyses of the primary endpoint**
Exploratory: the difference of the LS means and its 95% CI in each level; no adjustment for multiplicity.
:::
| Attribute | Meaning | Default (said in the rendered plan) |
|---|---|---|
analyses= | the analyses each level repeats: each a model that compares pairs of groups (contrast="pairwise": an ANCOVA, a Cox model, a logistic, Poisson or negative binomial regression, a t-test, ...) | required |
factors= | the groupings of the subgroups; a factor of ADSL or of the analysis's data set | required |
pvalue="true" | the p-value of each pair in each level (nominal) | none: the estimate and its CI only |
interaction="0.10" | the level of the test of the interaction of the treatment and each factor (one test per factor); none: no test, with the reason in the body (or the protocol's sentence); open: the test is made and its level is the statistician's (a warning) | no test (the review says so) |
minimum="10" | the subjects of a group in a level below which the results of a pair in that level are not shown (NE) | every level is estimated |
forest="false" | no forest plot | a forest plot of the levels goes with the table |
level= | key or exploratory | not stated (the review says so) |
What the compiler makes (in the structured SAP, each with the subgroup block's citation as its source): for each analysis A
and factor F an analysis of the levels (<SG>_<F>, or <SG>_<A>_<F> with several analyses: A's data and records,
by the treatment and F, its method <SG>_<A>_LEVEL the method of A: the same model, engine and options, its
statistics without the p-value unless pvalue="true"), the test of the interaction (<SG>_<F>_INT with the method
<SG>_<F>_INTERACTION, test: interaction), the count of the subjects of each group in each level (<SG>_N), and
one table <SG>_T with these analyses. The engines:
- each level is A's own engine on the records of the level (r-agentic makes one fit per value of a further
grouping of the results;
ars_e_subgrouparound it appliesminimum=). No second computation of a method. - the interaction (
ars_e_interaction): A's model with the factor as a main effect, against the same model with the treatment-by-factor terms. A linear model (ANCOVA, ANOVA): the F test of the terms (SAS PROC GLM, the Type III F test of the interaction). A logistic, Poisson, negative binomial (the dispersion estimated in each model) or Cox model: the likelihood-ratio test, chi-square with the number of terms the data estimate. The Wald test is not made. No engine tests the interaction of a t-test, a Wilcoxon test or the comparison of two proportions: an error says so (interaction="none", or write the analysis as a model). Known answers: the methodsinteraction-lm,-logistic,-poisson,-negbin,-coxof the table above (the fits written out intools/method_answers.R). - SAS (not yet run on a SAS server): each level is the macro of A with
within=the factor (%cox,%logistic,%rateratio,%lsmeans); the interaction is%interaction_glm,%interaction_logistic,%interaction_phregor%interaction_genmod(each holds its procedure); the forest plot is%forestplotof the same results.minimum=is applied by R only (the SAS program says so above the call).
What the products show: the table shell has, for each factor, one row per level with the subjects of each group and
the estimate of each pair with its 95% CI across the groups, then the interaction p-value of the factor; the ARD of the
table has the factor's grouping and level in each result; the forest plot (sapling tlf companions: R; SAS %forestplot)
draws the same results; the box of the block in the rendered plan says what is made in one sentence (each default
marked) with the table of its analyses; the dashboard has the table as a page whose source is the block's citation.
The review: e9.covariates-subgroups (no interaction test and no reason), e9.multiplicity (p-values of the levels or
of the interaction, and no multiplicity block that says the other tests are nominal or names the analysis),
ema-subgroups.prespecify (no level=; a factor whose levels come from the data).
Interim analyses tied to their hypotheses#
An interim look that may stop for efficacy names the hypotheses it tests (hypotheses=: hypotheses of multiplicity
blocks). The compiler then makes the group-sequential design of each such hypothesis: its looks (each interim block
that names it, in the order of the information, and its final analysis at information 1), its level (its share of the
alpha of its family: the family's alpha for hierarchy and co-primary; alpha= or weight= times the family's
alpha for bonferroni; the first step for holm, hochberg and graphical), and the efficacy boundary of each look
as data. NCT05505734 (one look at half of the events, O'Brien-Fleming spending, two-sided 0.05; stop only when both
hypotheses are rejected; 0.049 at the final analysis):
::: {.multiplicity #MULT_IA procedure="hierarchy" alpha="0.05"}
**The primary hypothesis, then the first key secondary one**
[Change at Week 24, high dose against placebo]{.hypothesis #H1_IA analysis="A_PRIM" final_alpha="0.049"}
[Change at Week 16, high dose against placebo]{.hypothesis #H2_IA analysis="A_ANCOVA" final_alpha="0.049"}
:::
::: {.interim #IA_EFF2 hypotheses="H1_IA; H2_IA" information="0.5" events="172" purpose="efficacy"
spending="obrien-fleming" alpha="0.003" stop="all" committee="the independent data monitoring committee"}
One look for efficacy at half of the planned events. The study stops only when both hypotheses are rejected.
:::
| Attribute | Meaning |
|---|---|
hypotheses="H1; H2" | the hypotheses this look tests (each of a multiplicity block); a look for futility only may name them too (it spends no alpha) |
information="0.5" | the information fraction of the look (events or patients over those of the final analysis); one number, or one per hypothesis in the order of hypotheses= ("1; 0.34" when the look is the final of one and an interim of the other: then write the look as an interim of the second only) |
events="172" | the planned events at the look (one, or one per hypothesis): shown in the table |
spending= | how the boundaries are made: obrien-fleming (Lan-DeMets, O'Brien-Fleming type: alpha(t) = 2 - 2 Phi(z(1 - alpha/2) / sqrt(t)), gsDesign sfLDOF), pocock (Lan-DeMets, Pocock type: alpha log(1 + (e - 1) t)), hwang-shih-decani with gamma= (default -4, stated), haybittle-peto (a fixed boundary at the interim look: z= or alpha=, default z = 3, stated), fixed (the stated alpha= or z= of the look); one function for all the looks of a hypothesis |
alpha="0.003", z="2.96" | the boundary the plan states at this look (the plan's sides; one, or one per hypothesis): with a spending function it is compared with the computed one; with haybittle-peto or fixed it is the boundary |
futility="non-binding", futility_z="0" | the futility rule and its boundary (stop when z is below it; an HR of 1.0 or more is z of 0 or less) |
stop="all" / "any" | the study stops for efficacy when all (any) of the hypotheses of the look are rejected; the gate between them is the order of their multiplicity block |
final_alpha="0.049" (on hypothesis) | the nominal alpha the plan states for the final analysis of the hypothesis |
What the compiler computes (src/gsd.ts): the recursive numerical integration of Armitage, McPherson and Rowe (1969;
Jennison and Turnbull 2000, chapter 19) of the joint normal distribution of the statistics of the looks (no
simulation). The efficacy boundaries are one-sided at the one-sided level (a two-sided 0.05 has the boundaries of
one-sided 0.025, and the p-values are shown two-sided); a futility boundary is not used in the computation (the
boundaries of a non-binding rule; with a binding rule the type I error is then at most the level, and the rendered plan
says so). With haybittle-peto or fixed the boundaries of the interim looks are the stated ones and the final
boundary is the one that keeps the level. A look with spending="fixed" and no information= (an analysis at a data
cutoff an agency asks for) spends its alpha by Bonferroni: the other looks share the rest. Known answers: Jennison and
Turnbull (2000) Tables 2.1 and 2.3, the gsDesign technical manual, and the two SAPs above (test/gsd.test.ts); the
crossing probabilities again by mvtnorm::pmvnorm (test/gsd-r.test.ts).
What it checks: each hypothesis of hypotheses= is a hypothesis of a multiplicity block; each look has its information
fraction; one spending function per hypothesis; an efficacy look without hypotheses= is a warning (no family accounts
for its alpha). A stated number (alpha=, z= with a spending function; final_alpha=) that is not the computed one
at the precision it is written with is a warning with both numbers. The review (interim.alpha-spending): an
efficacy look without a spending function or a boundary is an error; without its information fraction a warning; the
looks of a hypothesis that spend more than its level with the plan's own numbers is an error with both numbers.
What it shows: the box of the interim analysis in the rendered plan has one row per hypothesis with the design in a
sentence (the function, the level, the final alpha that results, what all looks spend) and the boundary table (look,
information, events, hypothesis, z, p, cumulative alpha, the plan's own number, the futility boundary). The JSON has
the table as boundaries of the interim block and the design as group_sequential of each hypothesis.
What Sapling does not compute: the boundaries after the real information differs from the plan (the plan states the rule in words: recomputed by the same spending function at the real fraction), a sample size re-estimation and conditional power (prose, or an open decision), repeated confidence intervals, boundaries that move when a graph passes alpha on (Maurer and Bretz 2013: the table is at the first level of each hypothesis).
Adverse event tables and the treatment-emergent period#
The safety part of a plan states its definitions and its tables as data (guides/safety.md: how real SAPs state them). Five forms, each made by both engines from the same contract (R: r-agentic; SAS: the macro library, not yet run on a SAS server):
::: {.treatmentemergent #TEAE variable="ADAE.TRTEMFL" start="TRTSDT" end="TRTEDT + 30" missing="emergent"}
:::
[]{.cite source="protocol:4.4" words="Treatment-emergent adverse events (also referred to as"}
::: {.grouping #SEV variable="ADAE.AESEV" groups="MILD; MODERATE; SEVERE"}
Severity
:::
:::: {.tlf #T_AE population="SAF" columns="ACTUAL"}
**Treatment-emergent adverse events**
::: {.incidence #AE_SEV condition='ADAE.TRTEMFL = "Y"' rows="soc;pt" worst="SEV" order="alphabetical"}
By system organ class, preferred term and maximum severity
:::
::: {.incidence #AE_RD condition='ADAE.TRTEMFL = "Y"' rows="pt" threshold="5" order="frequency" test="riskdiff"
ci="newcombe" pvalue="false"}
The preferred terms in 5% or more of a group, with the risk difference of each
:::
^[Descriptive comparisons: not adjusted for multiplicity.]
::::
| Form | It says | What is made |
|---|---|---|
::: {.treatmentemergent #ID ...} | the period of the flag variable= (default ADAE.TRTEMFL): an event whose start date date= (default ASTDT) is on or after start= (default TRTSDT) and on or before end= (a date and days: TRTEDT + 30; none: no end, with the reason or a citation); a subject without the end date (still on treatment) has no end yet; missing= how an event with a missing start date counts: emergent (the worst case), not emergent, open (a warning; the default then: not emergent) | a plan block (JSON, the rendered plan: one sentence, each default said, and the tables that select the flag). In a plan that derives its ADaM, the flag of the data set (a step of kind flag.teae, placed where the dates exist); a second derivation of the flag is an error. Without missing=: a warning |
worst="G" on an incidence or an analysis | each subject once in each row, at its worst group of the grouping G (its groups in order, least to most severe: the maximum severity, the toxicity grade, the worst status of a shift table); a record in no group of G is not counted | the rows by the groups of G under each row (the last grouping of the rows); n (%) (Events with worst= is an error); R: ars_e_catsumm worst=; SAS %worst then %freq |
threshold="5" on an incidence | only the rows with 5% or more of the subjects of a group, in at least one group (the percent itself, not its rounded text) | R: ars_e_catsumm threshold=; SAS %freq(threshold=). With a test of each row other than the comparison of two proportions: an error (the test would be made for the rows it removes) |
order="frequency", "frequency in <group>", "alphabetical" on an incidence | the rows by their subjects (of all the groups of the columns, or of one group), the most first, equal counts by name; the system organ classes keep their alphabetical order; with worst= the terms are ordered, each with its groups | R: ars_table (the method's sort=); SAS %freq(order=frequency) (a twin <V>N that %print orders by). The review: an incidence by SOC or PT without order= is a warning (safety.ae-by-soc-pt) |
test="riskdiff" ("riskratio", "oddsratio") with ci=, comparisons=, pvalue="false" on an incidence | the comparison of two proportions of each pair for each row: the subjects with the term against those without, of the subjects of the group; ci= the interval of the risk difference method (default wald, said); pvalue="false": the estimate and its CI only | the method perterm (R: ars_e_perterm: the flag of each term for every subject, then ars_e_binary; SAS %termflags then %risk(within=)); the table: one column per pair (the estimate and its CI) and one for its p-value, in the row of the term; the incidence's threshold applies; a row of an overview (rows="any") compares the subjects with any record; the incidences of a table share the columns |
count="events" on a statistics block with a rate | the rate of the events (every record) per per= units of exposure | the method eventrate (R: ars_e_eair count=events; SAS %rate(count=events)) |
Known answers: the fixture's made-up adverse events with their severities, counted by hand (tools/method_answers.R:
the Wilson and Newcombe intervals written out, Pearson's X2, the exact Poisson limits; test/methods-r.test.ts), the
same plan through the contract maker, the engines and ars_table (test/safety-r.test.ts: the worst severity of each
subject, the rows of a threshold, the order of the rows, the column of each comparison), and the flag of the period on
SDTM records through R (test/adam_steps-r.test.ts, step kind flag.teae). The review (safety.*,
rules.md): the treatment-emergent definition, the order of the rows, the worst grade and its scale (CTCAE
with its version), the value of a shift table, a p-value said to be descriptive, an event of special interest with its
query or terms.
sapling generate table makes the conventional tables: teae-severity (by the maximum severity), teae-grade (by the
maximum toxicity grade, the terms in descending frequency), teae-frequent (the terms in 5% or more of a group),
teae-riskdiff (the risk difference of each of them, no p-value), teae-rate (the rates of the subjects and of the
events per 100 patient-years); each default (5%, the order, the interval) is said in the plan for the statistician to
change.
What these forms cannot say yet: an existing event that worsens after the first dose; the grades as columns; the system organ classes by frequency; the time at risk to the first event of each term; a threshold with Fisher's exact test or the chi-square test of each row; the denominator of each parameter or cycle of a laboratory table.
The options of a model: MMRM, Cox, estimates#
The method's options are attributes of the model: of the formula span or of the model div. They go into the method
block of the structured SAP (model:, the chain's place for the model), and the hover shows the R call and the SAS
procedure of this model with them.
| Attribute | Values | For |
|---|---|---|
covariance= | us (unstructured), toeph, toep, ar1h, ar1, csh, cs, adh, ad; or a term us(AVISIT | USUBJID) in the formula | MMRM (an MMRM without test="mmrm": covariance= makes it one) |
fallback="toeph; ar1" | the structures to try, in order, when the first does not converge | MMRM |
df= | kenward-roger, satterthwaite, residual, between-within | MMRM |
estimation= | reml, ml | MMRM |
visit= | a value of AVISIT: the visit of the contrast | MMRM |
ties= | efron (the default: R coxph's; the SAS macro writes TIES=EFRON, not PHREG's own BRESLOW), breslow, exact (the discrete likelihood: R coxph "exact", SAS TIES=DISCRETE) | Cox |
pvalue= | wald (the default: the Wald test of the pair in the model), logrank (the log-rank test of the two groups of the pair, in the same strata; the hazard ratio and its CI stay the model's: "the log-rank test and the Cox estimate") | Cox |
ci= | the interval of a risk difference: wald (the default), newcombe (the hybrid score interval of two Wilson intervals), miettinen-nurminen (the score interval, with the factor N / (N - 1)), agresti-caffo; with strata only mh (the Mantel-Haenszel estimate, the variance of Sato), its default. Written into the method | risk difference |
variances= | equal (the default: Student's t-test, the pooled variance of the two groups), unequal (Welch's, Satterthwaite's df; R t.test's own default). Written into the method | t-test |
correction= | yes (the default: the continuity correction 0.5 of the normal approximation, SAS CORRECT=YES), no. Written into the method | Wilcoxon rank-sum test |
exposure= | the exposure of each subject of a count model: a numeric variable, or one divided by a number ("TRTDUR / 365.25": days to years); the offset is its logarithm, and the rate ratio is per unit of it. Required | Poisson and negative binomial regression |
event= | the value of a response with two values that is the event (the model is of its probability); the default Y for a text, 1 for a number, written into the method (SAS PROC LOGISTIC alone models the first value in sort order: the macro writes EVENT=) | logistic regression |
strata="SITEGR1" | the strata terms (the same as strata(SITEGR1) in the formula); several: strata="STRAT1; STRAT2" or strata(STRAT1) + strata(STRAT2), each combination of their values one stratum | Cox, log-rank, CMH |
contrast= | dose-response (a numeric treatment, the default for TRTPN), overall (the F test of the groups), pairwise (the comparisons of the adjusted (LS) means of each pair of groups, from one fit of all groups: SAS LSMEANS / PDIFF CL, R emmeans; the default of a Cox model, whose hazard ratios are of pairs) | ANCOVA, ANOVA, MMRM, Cox |
treatment= | which term is the treatment, when the model has two treatment variables (a re-randomised part that adjusts for the first treatment: CHG ~ TRT01P + TRTP*AVISIT + ... with treatment="TRTP"). The other is then an adjustment term. Without it, two treatment variables in one model are an error: the compiler does not choose | all |
comparisons= | the pairs of contrast="pairwise", "A vs B; ..." of the treatment groups: each the estimate of A against B; by default each other group against the reference group of treatments: (Low vs Placebo, High vs Placebo) | pairwise |
stats="Estimate = xx.xx; 95% CI = (xx.xx, xx.xx); p-value = x.xxx" | the statistics of the method: the estimate and its CI from the same fit (the shell shows them across the columns). A comparison reports its p-value, or, where the SAP plans none, the estimate with its interval only (stats="OR = xx.xx; 95% CI = (xx.xx, xx.xx)": the methods whose rows have the estimate, src/methods.ts) | any model |
ties="open", df="open", ... | an option the SAP does not state, left open (also "not stated"): df, estimation, ties, pvalue, ci, variances, correction, event. It is an open decision of the plan (a warning open-option at its place, and a todo in the structured SAP with the analysis's quote, counted for the approver); the method's stated default applies for the computation, and the rendered plan says ties: OPEN (the plan leaves it to the statistician; default: efron applies). The rules that ask for the option (method.cox/ties, method.mmrm/df_method) take it | the options with a default |
[]{.formula formula="CHG ~ BASE + TRTP*AVISIT" covariance="us" fallback="toeph; ar1h; ar1" df="kenward-roger" estimation="reml" visit="Week 24"}
[]{.formula formula="Surv(AVAL, CNSR) ~ TRTP" test="cox" ties="efron" strata="SITEGR1"}
A Cox model gives the hazard ratio of each pair of comparisons= (by default each other group against the
reference group of treatments:) from one fit of all groups, its 95% CI (Wald) and the p-value
of the pair (stats="HR = xx.xx; 95% CI = (xx.xx, xx.xx); p-value = x.xxx", the default). The p-value is the Wald
test of the pair in the model; pvalue="logrank" makes it the log-rank test of the two groups in the same strata, the
usual "stratified log-rank test and the hazard ratio of the stratified Cox model" in one analysis:
[]{.formula formula="Surv(AVAL, CNSR) ~ TRTP + strata(STRAT1) + strata(STRAT2)" test="cox" ties="efron" pvalue="logrank"}
The direction of a comparison#
A hazard ratio, an odds ratio, a relative risk, a rate ratio and a difference compare two groups in one direction. Sapling never chooses it silently:
treatments:names the control group withreference:(sapling newwrites the arm whose type the USDM gives as a placebo control or no intervention; else the statistician writes it). As soon as a model compares pairs of groups and the treatments have no reference, the check is an error (comparison-reference) whose fix is the line to write.- A pair
"A vs B"is the estimate of A against B: the hazard ratio of A relative to B, the difference A minus B. Withoutcomparisons=a model compares each other group against the reference, in the order of the groups. A comparison of two other groups (High vs Low) is written incomparisons=. - A pair whose first group is the reference (
"Placebo vs Drug") is a warning (comparison-direction) that names the pair and its inverse. - The rendered plan (the box of the analysis, the HTML preview) prints the pairs with the estimate in words, and the
table shell has a row per pair:
Drug vs Placebo: Hazard ratio. The R engines and the SAS macros take the pairs as written (pairs = "Drug" "Placebo"in the SAS call, the pairs separated by#);test/direction-r.test.tsruns both directions of one pair through the contract maker and the R engine on the known answer: the hazard ratio of B vs A is 1 / that of A vs B, the risk difference turns its sign.
treatments:
groups: [BDA MDI 160/180 ug, AS MDI 180 ug]
reference: AS MDI 180 ug
variable: ADSL.TRT01P
Pairwise comparisons of the LS means, and what the chain cannot take yet#
:::: {.tlf #T14_3_01 population="EFF"}
**ADAS-Cog (11): ANCOVA at Week 24**
::: {.analysis #T14_3_01_LSM}
**Pairwise comparisons of the LS means**
[]{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "Week 24" and ANL01FL = "Y"'}
[]{.stats method="MODEL_N"}
[]{.formula formula="CHG ~ BASE + SITEGR1 + TRTP" contrast="pairwise"}
:::
::::
The treatment is a factor (TRTP), and contrast="pairwise" asks for the difference of the adjusted means of each
pair of groups, with its standard error, 95% CI and p-value (the default statistics Diff (SE), 95% CI,
p-value; stats= changes them, for example LS mean (SE) = xx.x (xx.xx) first for the LS mean of each group). The
method block has test, model: {treatment, covariates, factors, contrast: pairwise} and comparisons (the list of
pairs). The template (Appendix A) has a row per pair: the difference (SE) and its CI across the columns, the p-value
on the pair's row (CSR Table 14-3.01). An MMRM takes the same attributes (with visit= the visit of the contrast).
(MODEL_N is a statistics block of the full pilot plan: n = xx, the number of patients in the model.)
The engine of sapling computes them (ars_e_lsmeans; an MMRM: ars_e_mmrm): sapling build writes them into the
structured SAP. Pending in the checker of the SAS project (warning chain-pending, only from sapc check and
sapc build without --chain-pairwise): the unchanged checker of sas-macro-agentic takes an ANCOVA only as the test
for linear dose response (model.dose), and it makes a comparison as a data subset of the two groups. For that
checker the analysis is in the plan, its JSON ("in_structured_sap": false, with chain_pending: why) and its
template, and its structured SAP has an open task (a todo with the analysis's quote) in its place. An ANCOVA that
compares the groups without contrast="pairwise" stays an error (chain-model).
What each model makes is in The methods and what they report: the estimate, its
SE and its CI of an ANCOVA with the dose (stats="Estimate = xx.xx; SE = xx.xx; 95% CI = (xx.xx, xx.xx); p-value = x.xxx"), the estimates of a Kaplan-Meier curve, an ANOVA with a dose and factors (the engine of the dose response). A
model no engine computes (an MMRM with the dose, a Cox model or a logistic regression of a dose) is an error
chain-engine that says which method can.
The review: the rules and waivers#
Every check of the knowledge base runs as you write (sapc check, the editor): a finding says what is missing or
wrong, the fix in the Markdown of the plan, and the guideline with its section, for example
examples/cdiscpilot01/pilot.qmd:142:1: warning[missing.single-imputation]: the primary analysis P_ADAS (ADAS-Cog (11),
change from baseline to Week 24) uses single imputation: its records keep the LOCF rows of ADQSADAS (no condition on
ADQSADAS.DTYPE); the plan gives no justification.
142 | :::: {.analysis #P_ADAS}
| ^^^^^^^^^^^^^^^^^^^^^^^^
fix: write `::: {.missingdata #MD_P_ADAS analysis="P_ADAS" assumption="..." handling="locf"}` with the justification
"why it is acceptable for this estimand" in its body, and a sensitivity analysis under MNAR
(`::: {.sensitivity #ID method="tipping-point"}` with the title in bold)
source: NRC report on missing data 2010, Recommendation 10; EMA missing data guideline 2010 6.3.1
The check and the review give the findings to act on first (the errors, then the warnings that ask for something: a
seed, a version, a reason), and the kinds that need no action one by one as one line each, with the count and the first
three names: the variables of the study the compiler cannot check yet (unknown-variable without an ADaM
specification), the analyses the SAS project's checker cannot take yet (chain-pending), the values and steps of an
ADaM derived in part before the data. sapling check --verbose (sapc check --verbose, sapc review --verbose) lists
each one; the review file out/review.txt always has every one. The counts of the passes are those of every finding.
The rules, their severities and conditions: rules.md. sapc review plan.qmd writes the whole review
(out/review.txt, review.json, review.html): the findings, the checklist items no program can check ("for the
reviewer"), and the coverage of the 55 items of Gamble et al. 2017, the SPIRIT items that touch the SAP and the five
ICH E9(R1) attributes of each estimand.
A plan silences a rule only with a reason, which is part of the plan (the PDF shows it):
::: {.waive rule="gamble.18" block="P_ADAS"}
P_ADAS keeps the layout of the study report; the estimate and its CI are in P_EFF_EST.
:::
::: {.waive rule="gamble.20"}
The protocol includes all enrolled patients in the safety analysis.
:::
[]{.cite source="protocol:3.10.2"}
block="ID" waives the findings on that block only; without it, every finding of the rule. A waiver of a rule that
finds nothing is a warning (remove it); of a rule that does not exist, an error. The reason is the statistician's (or
the protocol's, cited): an agent does not invent it.
The SAP library#
The block use makes the blocks of a pattern of standards/sap_library/ (the attribute answers= holds the
pattern's questions as key=value; a missing key is an error that prints the question):
:::: {.tlf #P_EFF2 population="EFF"}
**Primary endpoint analysis**
::: {.use pattern="ancova_dose_response"
answers="id=P_ADAS2, title={ADAS-Cog (11), change from baseline to Week 24}, dataset=ADQSADAS, param=ACTOT, visit=Week 24, dose=TRTPN, covariates=BASE, factors=SITEGR1"}
:::
::::
It compiles to the same blocks as the analysis with its formula span. Markdown has no macros: a family of tables
is written out, one tlf div each (Macros), or made by sapling generate table <template> from the
company's or the study's table templates (docs/omakase.md).
The kinds of blocks#
| Kind | Classes | sapc |
|---|---|---|
| meaning (blocks) | population, datasubset, grouping, derivation, statistics, summarize, tabulate, incidence, byvisit, use, decision, tlf, analysis, model, estimand; spans test, data, variable, by, stats, formula, where, treatment, endpoint, intercurrent, summary | each becomes blocks of the structured SAP, checked |
| meaning (the plan) | design, samplesize, missingdata, interim, software, dictionary, treatmentemergent, change, waive, multiplicity, sensitivity, supplementary; spans hypothesis, transition | blocks of the JSON and the PDF, checked (not in the structured SAP; the ADaM chapter derives the flag of treatmentemergent) |
| meaning (the ADaM) | dataset, codelist, copyvars, derive, flag, cut, pooling, baseline, changefrombase, impute, rangeindicator, addrecords, relativechange, parameter, parameters, windows, timetoevent; spans param, window, event, censor | the ADaM model of the checks; sapc adam: the specification, the define table, the programs (JSON and PDF; not in the structured SAP) |
| meaning (settings) | the keys of the front matter under sapling:: study, protocol, usdm, adam, sdtm, library, status, drafted-by, approved, alpha-level, treatments, visits | the study block and the defaults |
| citations | []{.cite source="..." words="..."} | the quote, checked |
| prose | headings # to #### ({.appendix}, {.unnumbered}), *emphasis*, **bold**, `code`, [x]{.smallcaps}, [DS.VAR]{.var}, [ID]{.ref}, <url>, ^[footnote], lists, definition lists, > quote, pipe tables, $...$, $$...$$ | the text of the plan (in sap.md, the HTML) |
| other Markdown | a div or span with a class that is not Sapling's (.callout-note, .column-margin) | its content is read as prose; a class one letter off a Sapling class (.summarise) is an error with did you mean |
What the compiler checks#
sapc check plan.qmd: the syntax (every div closed, the front matter, the attributes of each block); every id once and
every reference defined; every variable in the ADaM specification of its data set, the type of each value, the
codelist of each quoted value; every citation (source, section, words, with the closest sentences); every model
(variables, treatment term, method, the options of its method); the plan blocks (ratios, weights, transitions,
references, numbers; the sample size recomputed); the approval (status: APPROVED needs approved:, written by a
person, never "agent"); and the completeness rules of the knowledge base (src/rules/, rules.md: ICH
E9(R1), Gamble et al. 2017, FDA and EMA guidance, the method pages; more with --rules module.mjs), each silenced only
by a waive block with a reason; and what the chain refuses: a quote of a block that only the USDM file has, not the
protocol index the chain checks (error chain-quote), an ANCOVA that compares the groups instead of the dose (error
chain-model: the chain makes an ANCOVA only as the test for linear dose response, or the pairwise comparisons with
contrast="pairwise"), and what no engine computes as the plan asks (error chain-engine, from the method registry:
The methods and what they report). Then the chain's own checker on the
product: tools/sap_build.py <study> --file sap/<name>.md ends SAP BUILD PASS (tools/chain_proof.sh).
examples/errors/typos.qmd:24:60: error[codelist]: "WEEK 24" is not a value of ADQSADAS.AVISIT (codelist AVISIT).
24 | []{.data dataset="ADQSADAS" records='PARAMCD = "ACTOT" and AVISIT = "WEEK 24"'}
| ^^^^^^^^^^^^^^^^^^
fix: write AVISIT = "Week 24"
A message names a block by its class ("there is no block summarise"), and a fix shows the Markdown to write: the
opener of a div with its attributes (::: {.software name="SAS" version="..."}) and what goes in its body, a span
([]{.data dataset="ADQSADAS" records="..."}), an attribute (population="SAF"), a heading (## Missing data) or a
key of the front matter (alpha-level: {level: 0.05, sided: 2} under sapling:). The messages are written once
(src/compile.ts, src/rules/); src/say.ts says them in their Markdown form. The test test/messages.test.ts
checks that every message is in Markdown form and that every block a fix shows is Markdown the reader accepts.
The ADaM from the SAP#
Instead of naming an ADaM specification (adam:), a plan can derive its ADaM itself: a chapter names every variable
the analyses read by the ADaM IG and derives it from SDTM (sdtm: folder under sapling: in the front matter);
sapc adam plan.qmd makes the ADaM specification (r-agentic's format), the Define-ready table and one admiral program
per data set. Every block, the rule language and the proofs: adam-from-sap.md. The full pilot:
examples/cdiscpilot01/pilot-adam.qmd; these blocks are from it:
::: {.dataset name="ADSL" label="Subject-Level Analysis" structure="one record per subject" adam-class="ADSL"
from="DM" where='ARMCD != "Scrnfail"' keys="USUBJID"
order="STUDYID; USUBJID; SUBJID; SITEID; SITEGR1; ARM; TRT01P; TRT01PN; TRT01A; TRT01AN; TRTSDT; TRTEDT; TRTDUR; AVGDD; CUMDOSE; AGE; AGEGR1; AGEGR1N; AGEU; RACE; RACEN; SEX; ETHNIC; SAFFL; ITTFL; EFFFL; COMP8FL; COMP16FL; COMP24FL; DISCONFL; DSRAEFL; DTHFL; BMIBL; BMIBLGR1; HEIGHTBL; WEIGHTBL; EDUCLVL; DISONSDT; DURDIS; DURDSGR1; VISIT1DT; RFSTDTC; RFENDTC; VISNUMEN; RFENDT; DCDECOD; DCREASCD; MMSETOT"}
:::
::: {.derive variable="ADSL.TRT01PN" rule='0 if ARMCD = "Pbo"; 54 if ARMCD = "Xan_Lo"; 81 if ARMCD = "Xan_Hi"'}
:::
::: {.derive variable="ADSL.TRTSDT" rule="date(SVSTDTC)" from="SV" where="VISITNUM = 3"}
:::
::: {.flag variable="ADSL.SAFFL" condition='ITTFL = "Y" and TRTSDT is not missing'}
:::
::: {.cut variable="ADSL.AGEGR1" source="AGE" groups="<65; 65-80; >80" codes="AGEGR1N"}
:::
::: {.flag variable="ADQSADAS.ANL01FL" condition="AVISITN is not missing" mode="first" by="USUBJID; PARAMCD; AVISITN"
order="AWTDIFF; ADY" label="Analysis Record Flag 01"}
:::
::: {.impute dataset="ADQSADAS" method="LOCF" records='PARAMCD = "ACTOT"'}
:::
::: {.baseline variable="ADQSADAS.BASE" source="AVAL"}
:::
::: {.changefrombase variable="ADQSADAS.CHG" rule="AVAL - BASE" records="AVISITN > 0"}
:::
Then every variable of every analysis is checked against these derivations: a variable no derivation makes is an error
whose fix is the derive or copyvars block to write.
Before the data: part of the ADaM. The strict check above is for a plan with its SDTM data (sdtm:): it derives
its whole ADaM. A plan without sdtm: may derive part of it (the time-to-event parameters with their ordered censoring
rules, a window, a baseline, the imputation of a date) while the rest is named by the ADaM IG:
- a data set the plan does not declare (
dataset) is checked against the ADaM IG, as in a plan without ADaM (a variable the IG does not have is a warning,not in the ADaM IG and not derived in the plan); a variable a declared data set does not derive is the same warning; a value of a derived codelist (a parameter not derived yet) a warning; - a pattern writes without SDTM metadata: the columns of a source the plan does not derive are not checked (one warning per data set names them); a column read as a date is taken as a date (a warning names it); the type of a derivation from a domain is the ADaM IG's (a name ending in DT a date, a standard variable its IG type) or text, said in a warning; a copy without its SDTM label takes its name, said;
- with
sdtm:every one of these is checked again, and an undeclared data set an analysis reads is the errorunknown-datasetwhose fix is thedatasetblock to write.
sapling build of a plan that derives no ADaM skips the ADaM specification, programs and data (it says so once:
skipped: adam spec, adam programs: the SAP derives no ADaM ...) and goes on to the table specification, the shells,
the QC specification and the programs: the contract takes a provisional ADaM specification of the variables the plan
names (out/tlf/spec/adam: the derived variables, the ADaM IG's, and the others typed by their use in the plan, labelled
"not specified yet"); sapling sas programs reads it. A study made from a protocol PDF (protocol/protocol.txt) gives
the chain its page index from Sapling's reader of the text. The hooks of custom/ of a data set the plan does not
derive (a companion plan in the same folder) are a warning: the programs of this plan do not run them.
Dates, results with limits, periods, computed parameters, censoring rules, intercurrent events#
These blocks state the patterns that real analysis plans need; each is a step kind with its guarantees and its fixture (adam-from-sap.md, "The commands"), made the same way in R and in SAS. A choice the plan must make has no hidden default: the compiler asks for it.
::: {.derive variable="ADAE.TRTEMFL" rule='"Y" if ASTDT >= TRTSDT or samemonth(AESTDTC, TRTSDT); "N" otherwise'}
:::
::: {.derive variable="ADAE.ASTDT" rule="date(AESTDTC)" impute="first" level="month" not-before="TRTSDT" flag="ASTDTF"}
:::
::: {.numeric variable="ADLB.AVAL" source="LBSTRESC" below="half" above="limit" flag="AVALQFL"}
:::
::: {.period variable="ADAE.APERIOD" date="ASTDT" periods="1: TR01SDT - TR01EDT; 2: TR02SDT - TR02EDT"
text="APERIODC" names="Period 01; Period 02"}
:::
::: {.computed parameter="ADVS.BMI" rule="WEIGHT.AVAL / (HEIGHT.AVAL / 100) ** 2" by="AVISIT; AVISITN"}
:::
::: {.intercurrentevent variable="ADSL.RESCFL" records='CMCAT = "RESCUE"' from="CM" at="date(CMSTDTC)"
start="TRTSDT" end="TRTEDT + 7" date="RESCDT"}
:::
::: {.timetoevent parameter="ADTTE.PFS" start="RANDDT" mode="ordered"}
[]{.censor records="LASTADT is missing" date="RANDDT" censor="1" set='EVNTDESC = "No adequate assessment"'}
[]{.censor records="NACTDT is not missing and (PDDT is missing or NACTDT < PDDT)" date="NACTDT" censor="1"}
[]{.censor records="PDDT is not missing and days(LADTPD, PDDT) > 91" date="LADTPD" censor="1"}
[]{.event records="PDDT is not missing" date="PDDT"}
[]{.censor records="" date="LASTADT" censor="1"}
:::
::: {.timetoevent parameter="ADTTE.DIL3X" start="TRTSDT" end="DCUTDT"}
[]{.event from="ADLB" records='PARAMCD in ("ALT", "AST") and R2A1HI >= 3' date="ADT" priority="PARAMCD" set='EVNTDESC = "ALT/AST >= 3x ULN"'}
[]{.censor date="DCUTDT" censor="1" set='EVNTDESC = "Cutoff date"'}
:::
| Form | It says | Function or pattern |
|---|---|---|
year(x), month(x), samemonth(x, y), months(from, to) | the year and the month of a date or of an ISO 8601 text (a partial date has its year and month); the same month; the months between two dates: days(from, to) / 30.4375 | functions: one value of one record, one meaning, nothing to choose |
impute="first", "middle", "last"; level="day" (the default) or "month"; not-before="TRTSDT", not-after="RFENDT" | the imputation of a partial date, clamped to a date of the record only when that date is among the dates the partial date can be (admiral min_dates, max_dates); the flag D or M | options of the date step: typed, an unknown word is an error |
.numeric | a character result as a number: "<5" by below= (half, limit, missing), ">100" by above= (limit, missing), both stated by the plan; the flag Y on such a value | a pattern: two variables from one text and a stated rule |
.period | the period (or phase) whose start and end (of ADSL) hold the date of the record; both ends in, the first period of the list when two meet | a pattern: a lookup in the dates of the subject |
.computed | a parameter from other parameters of the same subject and visit (PARAMCD.VAR); no record when one is missing | a pattern: several records of a group |
.intercurrentevent | the flag and the date of an intercurrent event in ADSL: the earliest record of a domain that meets the condition with its date in start=..end= | a pattern: one fact, two variables that never disagree; the estimands refer to it (data='ADSL.RESCFL = "Y"') |
mode="ordered" on timetoevent | the rows in their order: the first that holds for the subject (and whose date is not missing) gives ADT, CNSR and its set= | a pattern: the order of a table is its meaning |
end= on timetoevent | an at-risk period from start= to end= (dates of the subject): the first event in it (on one date the earlier row, then priority=), else the latest censoring in it, else its end (end_desc=, default "End of at-risk period"; end_censor=, default 1); an event on the start day counts, with start_time= and the event's time= only when it is not before the start time; a subject without a start or an end date gets no record and is named in the run (R: a message [tte] ...; SAS: WARNING: [tte] ...); EVNTDESC, SRCDOM, SRCVAR, SRCSEQ (ADTF, ATM, ATMF) of the record taken | a pattern: the period is of the subject, the rule of its ends the same for every parameter |
The template and the programmer's hooks#
The generated programs are a template. The programmer's code goes into custom/ next to the plan, which sapc never
writes: hooks the generated programs call at a named step (hook_before, hook_after, hook_override, hook_add,
hook_filter in custom/<ds>.R; hook_layout in custom/tlf/<output>.R). Logic the rule language cannot say is
still stated in the SAP, in words, with its code in a hook (custom=) or in one line of R (r=). These blocks are
from examples/customize/custom-adam.qmd:
::: {.derive variable="ADSL.WTLOSFL"
rule="Y when the mean of the weights of three consecutive visits after baseline (VS, WEIGHT) falls to 95 percent of the baseline weight WEIGHTBL or less at any time, N otherwise"
custom="adsl_wtloss" type="C" length="1" label="Weight Loss (3-Visit Rolling Mean) Flag"}
:::
The compiler binds every hook to its step (an error with the fix when it cannot), lists the custom code, and sapc adam reports the hooks whose generated input changed after an amendment (hook drift). The rules, the custom report
and the proof on the pilot: customize.md.
Start a plan#
A new study starts with sapling new (omakase.md): the study folder with its plan sap.qmd, made from
the protocol (USDM) and the SDTM data.
sapling new <study> --protocol protocol/study_usdm.json --sdtm sdtm
node src/cli.ts init --protocol protocol/study_usdm.json --text protocol/protocol.json --adam adam --study STUDY01 study.qmd
node src/cli.ts import sap.md study.qmd --protocol protocol/protocol.json [--usdm F] [--adam DIR]
sapc init writes the first plan (study.qmd) from the protocol as CDISC USDM. The sections follow Gamble et al. 2017 (administrative information, introduction, study methods, statistical
principles, trial population, analysis, references). What the USDM gives as data becomes the front matter and blocks:
treatments: (the arms), visits: (the Baseline and Week encounters), alpha-level: (from the objective), a
design block (the model and the blinding), a samplesize block (the planned number), the objectives and endpoints,
estimand blocks (with their intercurrent events and strategies). What the protocol names in a sentence becomes a
block with that sentence cited: the populations (their ADSL flag from the labels of the ADaM specification, or the
ADaM IG), the tests (a test span, the chain's ids such as ANOVA_4_2), the software. Everything else is a
decision block that cites the sentence that leaves it open, so sapc review shows what remains. Without --text,
the protocol's page index is made from the USDM (protocol/protocol.json beside the plan, page for page the chain's
protocol_to_sap.py index). sapc init never writes approved:; the one waiver it writes (a primary estimand that
names no analysis yet) says why and asks to be removed.
sapc import reads a structured SAP (the chain's format) and writes the plan that compiles to the same blocks: each
table with its rows (summarize, tabulate, incidence, byvisit, or an analysis with its data, by and the
model as a formula), each test as a test span with its id, each quote as a citation (a sentence, or the words
"from ... to"), each open task as a decision block. An approval or a decided decision of the source is kept as a
comment: a person writes it again.
When the plan has usdm:, the structured SAP also carries the protocol's study definition: a todo with the
visits (section: "USDM encounters") and one per endpoint (section: "USDM endpoints", the pattern of the library that
fits or the gap), without a quote, as the chain's protocol_to_sap.py writes them; the chain's dashboard reads them.
The machine view: sapc json#
sapc json plan.qmd (and out/sap.json) is the whole plan as stable JSON for agents and tools: every heading and
block with its kind, id, fields, protocol source and exact position in the file. Schema: json.md.
Reference card#
---
title: Statistical Analysis Plan # Quarto: title, subtitle, date, author; toc: true
sapling:
study: ID
version: 0.1 (draft)
status: DRAFT
drafted-by: agent
protocol: {file: protocol/protocol.json, name: Protocol X}
usdm: protocol/study_usdm.json
adam: adam # or sdtm: sdtm (the ADaM from the SAP)
alpha-level: {level: 0.05, sided: 2, cite: {source: "usdm:objectives", words: "the words"}}
treatments: {groups: [A, B, C], reference: A, variable: ADSL.TRT01P, cite: "usdm:arms"}
visits: {visits: [Baseline, Week 2], variable: AVISIT}
# approved: {by: Name, on: Date, role: statistician} a person only, one per role
---
# Section []{.cite source="protocol:4.2" words="the words of the sentence"}
## Subsection []{.cite source="protocol:4.2" sentence="2"}
::: {.population #ID condition="ADSL.FLAG" definition="..."}
Name
:::
::: {.datasubset #ID condition="..."}
Name
:::
::: {.grouping #ID variable="DS.VAR" groups="a; b"}
Name
:::
::: {.derivation #ID variable="DS.VAR" rule="..." condition="..."}
:::
::: {.statistics #ID rows="n = xx; Mean (SD) = xx.x (xx.xx)" name="..." estimator="kaplan-meier"}
:::
Text with [anova]{.test}, [DS.VAR]{.var}, [ID]{.ref} and a citation []{.cite source="protocol:4.2" words="..."}.
::::: {.tlf #ID population="..." total="true" columns="..." pvalue="true" template="..." figure="true"}
**Title**
::: {.summarize #ID variable="DS.VAR" test="..." stats="..."}
Title
:::
::: {.tabulate #ID variable="DS.VAR" test="..." categories="a; b"}
Title
:::
::: {.incidence #ID condition="..." test="..." rows="soc;pt" comparisons="..." stats="..." worst="..." threshold="..."
order="..." ci="..." pvalue="false"}
Title
:::
::: {.byvisit #ID dataset="DS" timepoint="true" visits="..." records="..."}
Title
:::
:::: {.analysis #ID population="..." test="..." worst="..."}
**Title**
[]{.data dataset="DS" records="..."}
[]{.variable variable="V"}
[]{.by groupings="G1; G2"}
[]{.stats method="M"}
[]{.formula formula="Y ~ ..." test="..." contrast="..." covariance="..." fallback="..." df="..." visit="..." ties="..." strata="..." stats="..."}
::::
^[a footnote of the shell]
:::::
::: {.model contrast="..." treatment="..."}
$$
Y_i = \mu + \tau\, d_i + \varepsilon_i
$$
[]{.where bindings="Y = CHG, d = TRTPN"}
:::
::: {.estimand #ID level="primary" role="main" of="..." target="..." population="..." analysis="..."}
**Title**
[...]{.treatment}
[...]{.endpoint}
[event]{.intercurrent strategy="treatment policy" reason="..." data="..."}
[...]{.summary}
:::
::: {.decision}
The task; a person adds answer="...", by="..." and on="..." when it is decided.
:::
::: {.design type="parallel" allocation="1:1" blinding="double" framework="superiority" margin="..." strata="..." randomisation="..."}
:::
::: {.samplesize #ID n="N" power="..." alpha="..." sided="..." effect="..." sd="..." rates="..." hr="..." events="..." dropout="..." test="..." per_group="..." joint_power="..." analysis="..."}
:::
::: {.multiplicity #ID procedure="..." alpha="..."}
**Title**
[words]{.hypothesis #H1 analysis="..." weight="..." alpha="..."}
[]{.transition from="H1" to="H2" weight="..."}
:::
::: {.missingdata #ID analysis="..." estimand="..." after="..." pattern="..." assumption="..." handling="..." imputations="..." seed="..."}
Justification.
:::
::: {.sensitivity #ID estimand="..." method="..." assumption="..." delta="..." reference="..." imputations="..." seed="..." analysis="..."}
**Title**
:::
::: {.supplementary #ID estimand="..." population="..." analysis="..." method="..."}
**Title**
:::
::: {.interim #ID information="..." timing="..." purpose="..." futility="..." spending="..." alpha="..." committee="..."}
Words.
:::
::: {.software name="R" version="..." packages="..." validation="..."}
:::
::: {.dictionary name="MedDRA" version="..." for="..."}
:::
::: {.treatmentemergent #ID variable="ADAE.TRTEMFL" start="TRTSDT" end="TRTEDT + 30" missing="..."}
:::
::: {.change adds="..." version="..." date="..."}
**What**
Reason.
:::
::: {.waive rule="rule-id" block="..."}
Reason.
:::
::: {.use pattern="..." answers="key=value, ..."}
:::
<!-- the ADaM from the SAP (docs/adam-from-sap.md; sdtm: in the front matter) -->
::: {.dataset name="AD.." label="..." structure="..." adam-class="..." from="..." where="..." keys="..." order="..."}
:::
::: {.copyvars dataset="DS" variables="VAR; NEW = OLD" from="..."}
:::
::: {.derive variable="DS.VAR" rule="..." from="..." where="..." first="..." last="..." impute="..." flag="..." label="..." length="..." type="..." work="true"}
:::
::: {.derive variable="DS.VAR" rule="the rule in words" custom="hook id" type="..." label="..."}
:::
::: {.flag variable="DS.VAR" condition="..." false="N" from="..." mode="first" by="..." order="..." label="..."}
:::
::: {.cut variable="DS.VAR" source="..." groups="<65; 65-80; >80" codes="..."}
:::
::: {.pooling variable="DS.VAR" source="..." by="..." minimum="..." into="..."}
:::
::: {.baseline variable="DS.BASE" source="AVAL" by="..." flag="..."}
:::
::: {.changefrombase variable="DS.CHG" rule="AVAL - BASE" records="..."}
:::
::: {.impute dataset="DS" method="LOCF" records="..." flag="..."}
:::
::: {.rangeindicator variable="DS.ANRIND" value="AVAL" low="..." high="..." fallback="..."}
:::
::: {.addrecords dataset="DS" records="..." set="V = value; ..."}
:::
::: {.relativechange dataset="DS" value="..." prefix="..." limit="..." separately="..."}
:::
::: {.parameter parameter="DS.PARAMCD" rule="mean(AVAL)" records="..." by="..." set="..."}
:::
::: {.codelist codelist="SEX" values="F = Female; M = Male"}
:::
::: {.parameters dataset="DS" from="..." value="..." select="true"}
[name]{.param paramcd="PARAMCD" n="..." code="..." derived="true"}
:::
::: {.windows datasets="DS; DS" by="day" day="..." as="..."}
[]{.window avisit="AVISIT" avisitn="AVISITN" range="2-84" target="..."}
:::
::: {.timetoevent parameter="DS.PARAMCD" start="..." end="..." start_time="..." end_desc="..." end_censor="..."}
[]{.event records="..." from="..." date="..." time="..." priority="..." set="..."}
[]{.censor records="..." from="..." date="..." censor="..." set="..."}
:::
Pandoc and Quarto#
The constructs are those Quarto documents: YAML front matter, ATX headings with attributes, fenced divs with
attributes, bracketed spans, inline footnotes ^[...], $/$$ math, {#eq-...} on display math, definition lists,
pipe tables, HTML comments.
node tools/quarto_check.ts [plan.qmd ...] [--no-render] checks them with Quarto. It copies each plan to a temporary
folder outside the repository (rendering writes files next to its source), reads it with
quarto pandoc -f markdown -t json, and renders it with quarto render --to html. Every div and span the reader of
Sapling reads must be a Div or Span of Pandoc with the same class, id and attributes, and no fence or attribute may be
left over as text: QUARTO CHECK PASS. npm test runs the Pandoc part on pilot.qmd when quarto is on the PATH.
With Quarto 1.11.9 (Pandoc of Quarto), all eight example plans pass: Pandoc reads every div and span as Sapling does (pilot-adam.qmd: 461 divs and 541 spans), every HTML comment as raw HTML, and Quarto renders each plan to HTML without a warning. What Pandoc reads differently:
- in a quoted attribute value, Pandoc reads a backslash before punctuation as an escape: Pandoc sees
50%and_ALBwhere the compiler keeps50\%and\_ALB(values of the example plans, 40 in pilot-adam.qmd; see Attributes); $$...$$ {#eq-1}is a numbered equation for Quarto's cross-references; plain Pandoc shows{#eq-1}as text;- without a filter, Quarto shows a span's content as text and the attributes of a div or span as
data-attributes. Sapling's blocks show as their prose (a population as its name, a table as its title and rows); the citations, being empty spans, show nothing. The Quarto extension of Sapling (below) shows each block as a box.
A study folder: sap.qmd#
sapling new writes the plan as sap.qmd. Every study command reads it: check, status, find, decide,
generate, sap sync, and every step of build and run. A study folder with a plan of the earlier LaTeX form and
no sap.qmd is refused in one line.
| What | In sap.qmd |
|---|---|
the plan of sapling new | the generators write the plan in Markdown directly (omakase.md). The front matter names the extension: filters: [sapling]; _extensions/sapling/ is copied into the study |
an answer of sapling decide | the attributes of the decision's div, the rest of the file byte for byte: ::: {.decision answer="..." by="A. Statistician" on="2026-11-11"} (a div written ::: decision gets braces). Only a person answers (--by names a person; the demos' answers are labelled DEMO) |
a block of sapling generate | written in Markdown and inserted at the end of the own text of its section (before its first subsection), with one empty line around it: a pure insertion. An open decision the generator answers (generate multiplicity) is removed with its citation; an answered one stays. "Once" holds: a second generate of the same block is refused with the line of the first in the .qmd |
sapling sap sync | the spec's block replaces the SAP's at its place, under <!-- synced from adam-spec.qmd (sapling sap sync, date): a person reviews this rule --> |
| the specs | adam-spec.qmd (the derivation divs of the chapter "The analysis data sets" under their headings, each with the label, type and length of the specification as attributes) and tlf-spec.qmd (the .tlf divs, each under <!-- ID: SAP section "..." -->); front matter with the SAP's settings the specs need. The doctrine (omakase.md): a rule difference is reported and synced, a label, a title or the order of rows is the spec's own, a change of the SAP is merged 3-way |
The Quarto extension#
quarto/_extensions/sapling/ (_extension.yml and the Lua filter sapling.lua, about 250 lines). The plan names it
in its front matter (filters: [sapling]); sapling new copies it into the study, so a Quarto preview in the editor
shows the boxes. sapling sap pdf renders a copy of the plan in .sapling/render/pdf/ with the study's extension (or
Sapling's, when the study has none) by quarto render sap.qmd --to typst (Typst). When Quarto fails, the step fails
with Quarto's first error line (the whole log: .sapling/render/pdf/quarto.log.txt).
| In the plan | In the PDF and the HTML |
|---|---|
| a block (a Sapling div) | a box (Quarto's callout) titled with its kind and its id: "Population EFF", "Estimand EST1", "Table T14_3_1", "Analysis T14_3_1_CHG", "Derive ADSL.TRTSDT". Its body (the name, the bold title, the prose, the equation), then a table of its attributes and parts: population: Condition (the flag) and Definition; estimand: Analysis, Population, Level, Treatment, Variable, each Intercurrent event with its strategy, Summary; analysis: Data (data set where records), Variable, By, Statistics; model: the equation, Where, Formula and its options. Nested blocks (the analyses of a table) are boxes in the box |
| a decision | "Decision (open)" (an orange box), or "Decision (answered)" with its Answer, Decided by, Decided on |
| a citation | its source ("Protocol 4.3.3", "USDM arms") and, when the compiler resolved it, the quoted sentence of the protocol: the Source row of the block it cites, a "Source:" line under a heading, a footnote in a sentence. sapling sap pdf writes the sentences (sapling-cites.json, named by sapling-cites: in the front matter of the copy); without it the source and the words are shown |
[X]{.var}, [ID]{.ref}, [x]{.test}, [x]{.texttt} | code, a link to the block, emphasis, code |
a backslash command in the prose (\visits) | code (Pandoc reads it as raw TeX, which Typst and HTML drop) |
#### | a heading without a number; the other sections are numbered 1, 1.1, 1.1.1 |
| the outputs | the last appendix, "Table shells": one template for each output (its rows, the statistics of each cell, the tests, the notes), from sapling-shells.json, which sapling sap pdf and sapc pdf write from the compiler (named by sapling-shells: in the front matter of the copy) |
The PDF: a block is a box of the extension's own (Typst block(breakable: true)), not Quarto's callout, which cannot
break across pages (it left gaps at the bottom of pages, and a box in a box at the bottom of a page was cut). The title
of a box stays on the page of the first line of its body. HTML: Quarto's callout. Word (quarto render --to docx): a
block is its title in bold and the table of its attributes (Quarto refuses a callout in a callout in Word); for the
Word file of Sapling, with the way back, see docs/word.md.
Without the filter (plain Pandoc, or another tool) a block is plain prose; with plain Pandoc (quarto pandoc --lua-filter sapling.lua) the filter makes a div with the title in bold instead of a callout. The legibility is the
aim. A study made before this extension has its own older copy in _extensions/sapling/:
copy quarto/_extensions/sapling/ over it to get the boxes that break.
All commands#
The form of every command and environment of src/commands.ts. Made by node tools/qmd_doc.ts; npm test fails when
it is stale.
| Name | Role | Markdown form | Attributes |
|---|---|---|---|
study | preamble | sapling: study: ... | - |
title | preamble (Quarto) | title: ... at the top of the front matter | - |
subtitle | preamble (Quarto) | subtitle: ... at the top of the front matter | - |
version | preamble | sapling: version: ... | - |
date | preamble (Quarto) | date: ... at the top of the front matter | - |
author | preamble (Quarto) | author: ... at the top of the front matter | - |
protocol | preamble | sapling: protocol: {file: ..., name: ..., chain: ...} | keys of its map |
usdm | preamble | sapling: usdm: ... | - |
adam | preamble | sapling: adam: ... | - |
sdtm | preamble | sapling: sdtm: ... | - |
library | preamble | sapling: library: ... | - |
status | preamble | sapling: status: ... | - |
kind | preamble | sapling: kind: ... | - |
draftedby | preamble | sapling: drafted-by: ... | - |
approved | preamble | sapling: approved: {by: ..., on: ..., role: ...} | keys of its map |
alphalevel | preamble | sapling: alpha-level: {level: ..., sided: ...} | keys of its map |
treatments | preamble | sapling: treatments: {groups: ..., variable: ..., id: ..., name: ..., reference: ...} | keys of its map |
visits | preamble | sapling: visits: {visits: ..., variable: ...} | keys of its map |
maketitle | prose | the title block of Quarto (shown when the front matter has title:) | - |
tableofcontents | prose | toc: true in the front matter | - |
section | heading | # Title ({.unnumbered} after the title: a heading without a number) | - |
subsection | heading | ## Title ({.unnumbered} after the title: a heading without a number) | - |
subsubsection | heading | ### Title ({.unnumbered} after the title: a heading without a number) | - |
paragraph | heading | #### Title ({.unnumbered} after the title: a heading without a number) | - |
appendix | heading | # Title {.appendix}: the first section of the appendix | - |
cite | inline | []{.cite source="protocol:4.2" words="the words of the sentence"} (or sentence="2") | sentence, quote, words |
ref | inline | [id]{.ref #ID} | - |
var | inline | [variable]{.var} | - |
emph | inline | *text* | - |
textit | inline | *text* | - |
textbf | inline | **text** | - |
texttt | inline | `text` | - |
textsc | inline | [text]{.smallcaps} | - |
url | inline | <url> | - |
footnote | inline | ^[text] | - |
item | prose | a list item (- , 1. , or a term of a definition list) | - |
population | block | ::: {.population #ID condition="..."} / the name / ::: | definition |
datasubset | block | ::: {.datasubset #ID condition="..."} / the name / ::: | - |
grouping | block | ::: {.grouping #ID variable="..."} / the name / ::: | groups |
derivation | block | ::: {.derivation #ID variable="..." rule="..."} / empty / ::: | condition |
test | inline | [test]{.test #ID} | sided, alpha, pattern, text |
statistics | block | ::: {.statistics #ID rows="..."} / empty / ::: | name, estimator, conftype, time, unit, ci, exposure, per, count |
tlf | block | ::: {.tlf #ID} / title, then the body / ::: | population, total, columns, pvalue, template, figure |
summarize | block | ::: {.summarize #ID variable="..."} / the title / ::: | test, stats, population |
tabulate | block | ::: {.tabulate #ID variable="..."} / the title / ::: | test, stats, categories, population |
incidence | block | ::: {.incidence #ID condition="..."} / the title / ::: | test, rows, comparisons, population, stats, worst, threshold, order, ci, pvalue |
byvisit | block | ::: {.byvisit #ID dataset="..."} / the title / ::: | timepoint, visits, records, population |
analysis | block | ::: {.analysis #ID} / title, then the body / ::: | population, test, worst |
data | part | []{.data dataset="..." records="..."} on a line of its own | - |
variable | part | []{.variable variable="..."} on a line of its own | - |
by | part | []{.by groupings="..."} on a line of its own | - |
stats | part | []{.stats method="..."} on a line of its own | - |
model | part | ::: {.model #ID} / the body / ::: | test, contrast, sided, alpha, covariance, fallback, df, estimation, visit, ties, strata, stats, comparisons, treatment, variances, correction, event, ci, exposure, pvalue |
where | part | []{.where bindings="..."} on a line of its own | - |
formula | part | []{.formula #ID formula="..."} on a line of its own | test, contrast, sided, alpha, covariance, fallback, df, estimation, visit, ties, strata, stats, comparisons, treatment, variances, correction, event, ci, exposure, pvalue |
estimand | block | ::: {.estimand #ID} / title, then the body / ::: | population, analysis, level, role, of, target |
treatment | part | [text]{.treatment} on a line of its own | - |
endpoint | part | [text]{.endpoint} on a line of its own | - |
intercurrent | part | [event]{.intercurrent strategy="..."} on a line of its own | reason, data |
summary | part | [text]{.summary} on a line of its own | - |
decision | block | ::: {.decision} / the task / ::: | by, on, answer |
use | block | ::: {.use pattern="..." answers="..."} / empty / ::: | - |
design | block | ::: {.design type="..."} / empty / ::: | allocation, blinding, framework, margin, strata, randomisation |
samplesize | block | ::: {.samplesize #ID n="..."} / empty / ::: | power, alpha, sided, effect, sd, rates, hr, events, dropout, test, analysis, joint_power, per_group, design, margin |
multiplicity | block | ::: {.multiplicity #ID} / title, then the body / ::: | procedure, alpha |
hypothesis | part | [text]{.hypothesis #ID analysis="..."} on a line of its own | weight, alpha, comparison, final_alpha |
transition | part | []{.transition from="..." to="..."} on a line of its own | weight |
missingdata | block | ::: {.missingdata #ID} / the justification / ::: | analysis, estimand, after, pattern, assumption, handling, imputations, seed |
sensitivity | block | ::: {.sensitivity #ID} / title, then the body / ::: | estimand, analysis, method, assumption, delta, reference, imputations, seed |
supplementary | block | ::: {.supplementary #ID} / title, then the body / ::: | estimand, analysis, population, method |
subgroup | block | ::: {.subgroup #ID} / title, then the body / ::: | analyses, factors, pvalue, interaction, minimum, forest, level |
assumptions | block | ::: {.assumptions #ID} / the body / ::: | analysis, samplesize, n |
effect | part | [text]{.effect} on a line of its own | difference, sd, control, profile, baseline_coef, rates, median, hr, hypothesis, basis |
correlation | part | [text]{.correlation} on a line of its own | rho, structure, between, basis |
accrual | part | [text]{.accrual} on a line of its own | months, followup, basis |
dropout | part | [text]{.dropout} on a line of its own | rate, mechanism, delta, slope, basis |
ice | part | [text]{.ice} on a line of its own | rate, after, retain, shift, collected, basis |
scenario | part | [text]{.scenario #ID} on a line of its own | difference, sd, control, rates, median, hr, rho, dropout, mechanism, delta, ice_rate, retain, n, curve, zero, basis |
interim | block | ::: {.interim #ID} / the text / ::: | hypotheses, information, events, timing, purpose, futility, futility_z, spending, gamma, alpha, z, stop, committee |
software | block | ::: {.software name="..."} / empty / ::: | version, packages, validation |
dictionary | block | ::: {.dictionary name="..."} / empty / ::: | version, for |
treatmentemergent | block | ::: {.treatmentemergent #ID} / the text / ::: | variable, date, start, end, missing |
change | block | ::: {.change #ID} / what, then the reason / ::: | version, date, adds |
dataset | block | ::: {.dataset name="..." label="..." structure="..."} / empty / ::: | adam-class (the name class in src/commands.ts), from, where, keys, order |
codelist | block | ::: {.codelist codelist="..." values="..."} / empty / ::: | - |
copyvars | block | ::: {.copyvars dataset="..." variables="..."} / empty / ::: | from |
derive | block | ::: {.derive variable="..." rule="..."} / empty / ::: | from, where, first, last, label, length, type, impute, level, not-before, not-after, flag, work, custom, r, sas |
flag | block | ::: {.flag variable="..." condition="..."} / empty / ::: | false, from, mode, by, order, label, work |
cut | block | ::: {.cut variable="..." source="..." groups="..."} / empty / ::: | codes, label, codes_label |
pooling | block | ::: {.pooling variable="..." source="..."} / empty / ::: | by, minimum, into, label |
baseline | block | ::: {.baseline variable="..." source="..."} / empty / ::: | by, flag, label |
changefrombase | block | ::: {.changefrombase variable="..." rule="..."} / empty / ::: | records, label |
impute | block | ::: {.impute dataset="..." method="..."} / empty / ::: | records, flag |
rangeindicator | block | ::: {.rangeindicator variable="..." value="..."} / empty / ::: | low, high, lower, upper, fallback, label |
addrecords | block | ::: {.addrecords dataset="..." records="..."} / empty / ::: | set |
relativechange | block | ::: {.relativechange dataset="..." value="..."} / empty / ::: | prefix, limit, digits, order, separately |
parameter | block | ::: {.parameter parameter="..." rule="..."} / empty / ::: | records, by, set |
numeric | block | ::: {.numeric variable="..." source="..."} / empty / ::: | below, above, flag, flag_label, label, length |
period | block | ::: {.period variable="..."} / empty / ::: | date, periods, from, text, names, text_label, label |
computed | block | ::: {.computed parameter="..." rule="..."} / empty / ::: | by, records, set |
intercurrentevent | block | ::: {.intercurrentevent variable="..." records="..."} / empty / ::: | from, at, start, end, date, date_label, label |
parameters | block | ::: {.parameters dataset="..."} / the body / ::: | from, value, select |
param | part | [name]{.param paramcd="..."} on a line of its own | n, code, derived |
windows | block | ::: {.windows datasets="..."} / the body / ::: | by, day, as |
window | part | []{.window avisit="..." avisitn="..." range="..."} on a line of its own | target |
timetoevent | block | ::: {.timetoevent parameter="..."} / the body / ::: | start, mode, end, start_time, end_desc, end_censor |
event | part | []{.event records="..."} on a line of its own | from, date, set, priority, time |
censor | part | []{.censor records="..."} on a line of its own | from, date, censor, set |
waive | block | ::: {.waive rule="..."} / the reason / ::: | block |
document | prose | the body (after the front matter) | - |
itemize | prose | - item | - |
enumerate | prose | 1. item | - |
quote | prose | > text | - |
equation | prose | $$ ... $$ {#eq-1} | - |
equation* | prose | $$ ... $$ | - |
align | prose | $$\begin{aligned} ... \end{aligned}$$ | - |
align* | prose | $$\begin{aligned} ... \end{aligned}$$ | - |
description | prose | Term then : definition | - |