The assistant: proposals, interview, chat

This page is the repository's own document docs/copilot.md, shown as it is, so that it never drifts from the product.

Sapling's assistant works like a colleague who proposes and a statistician who agrees, declines or improves. The logic is in src/ (no editor API there), the surface is the visual editor and the assistant panel. The AI never approves a plan and never decides an open decision (AGENTS.md rule 2): every write is the person's click.

A proposal#

One change of the plan: a block inserted, a block or a paragraph replaced, a block deleted. It has a short title, the AI's reason in a sentence or two, the protocol sentences it cites, and the compile result of the plan with the change (it is compiled in its place before anyone sees it; one that does not compile is retried, and if it still fails it is shown as failed with the compiler's message and cannot be accepted). Status: pending, accepted, declined, superseded.

Kept in .sapling/proposals.json of the study folder (beside the plan when there is none): they survive a reload.
Stale: a proposal is applied only where its target still is (the lines it replaces, or the text around the place it is inserted). When the text changed, it is marked stale and must be made again against the block with the same id.
In the page: a new block with a green edge and the tag "AI proposal"; a change as the block now beside the block proposed, the changed words marked (and the changed attributes named); a deletion struck through. The Markdown is behind a small toggle.
Accept applies it as one edit (one undo) and the block carries  and the "drafted by the AI" tag until you edit or confirm it. Decline asks for an optional reason (kept, and told to the AI later). Improve… revises THIS proposal with your words; the thread of revisions is kept, shown small.
Keys: ⌥↓ ⌥↑ next and previous proposal, ⌘↵ accept, ⌘⌫ decline, ⌘K on a card or a paragraph: "Ask AI to change this".

The conversation and the inbox#

The assistant panel has Chat, Inbox and Cards. In the chat the AI either answers (the protocol sentences are found by the compiler's search, not by the AI) or proposes (cards that jump to their place). Type @ to attach a block of the plan (@T14_3_1), a protocol section (@§4.2), an ADaM variable (@ADSL.AGE), a review finding (@finding:gamble.20) or a guideline item (@guideline:method.mmrm). The turns are in .sapling/chat.json; Clear forgets them.

The tasks make many proposals at once, into the inbox: Answer the review findings, Draft a section from the protocol, Complete an estimand or table, Check consistency. Accept all that compile and cite the protocol lists what it will apply and asks once.

A question is answered briefly; the protocol sentences under the answer are the AI's quotes that the compiler found word for word in the protocol (an invented quote is dropped; a section the AI got wrong is searched in the others). The AI is shown the protocol passages nearest to the question to quote from. A request to remove a block is a deletion proposal (compiled without the block, shown struck through).

A block without an #id (a paragraph, a div) is not given one: the language's ids are the person's names. A stale proposal for it is made again against the block found by its first line, nearest to where it was (relocate() in src/proposals.ts), to the closing fence of a div or the end of the paragraph. An insertion whose place was used by another accepted proposal at the same spot is still placed (the text after it is unchanged).

The decision interview#

Work through the open decisions: one decision at a time, the question in one line, the protocol sentence, what good SAPs do (the knowledge base item, when one bears on it), the AI's recommended answer with its reason and alternatives, and a free-text field. You choose or write; then the blocks that follow from the answer are proposed, compiled and written in the same step (one undo), and the decision is recorded with your name and today's date. Skip and "ask me later" exist. The AI recommends; it never records a decision.

House rules and what the assistant learns#

.sapling/rules.md (and a shared file, the setting sapling.ai.rules) are plain sentences read in every request, after the protocol and before your request. The protocol wins: a conflict is asked as a decision. .sapling/learned.jsonl has one line per accept, decline (with the reason), improve and interview answer, without data values; the relevant lines go into later requests; the same correction twice is offered once as a house rule. Both files are plain text you can open, edit and clear. Nothing leaves the machine except inside the prompts your own AI provider already receives.

Real AI findings (the claude CLI, a sapling new pilot study, 2026-11-14)#

What the first real pass showed, and what was changed because of it:

The AI wrote long decisions (four to six numbered paragraphs inside one decision div) and reasons that talked about the compiler. The request form now asks for a decision of one question of at most 25 words with at most two short sentences, and for a reason in plain words about the study.
The recommended answer of the interview was 60 to 90 words and hedged ("to be confirmed ..."). The form now asks for one sentence of at most 35 words, with the doubts in the reason, and 2 to 3 options that differ in substance.
The learned lines of the person were used as facts of the plan ("an earlier accepted note names an MMRM ..."). The learned lines now say they are preferences, never to be mentioned in the plan or cited.
Two findings asked for the same block (#MD_...): the second proposal would have been a duplicate. The task skips a finding whose fix names a block id already in a pending proposal.
The sentences under a chat answer were chosen by a word search and often did not support the answer. They are now the quotes the AI names, verified by the compiler.
The panel looked stuck on "Thinking" for a long plan: each progress message made the whole page draw again. Progress and chat changes now redraw only the panel; the page redraws once, after a short pause, when the proposals change.

The benchmark#

node tools/copilot_bench.ts compares the forms alone with the proposal workflow on one fixed scenario (the first five open decisions of a plan from the CDISC pilot protocol, each with the blocks that follow), in a keystroke-level model. The AI's answers are REPLAYED from test/fixtures/bench-answers.json, the recording of the real pass above; the AI's waiting time in the totals is the real time of that pass. Read its table, not a slogan: it prints the totals, the ratio with and without the wait, and a row for a person who writes shorter answers.

Not waiting for the AI#

Prepared ahead (src/prepare.ts): the recommendations of the first open decisions are asked in the background, a few at a time (sapling.ai.parallel), cached in .sapling/recommendations.json and keyed by the decision's text and protocol sentence, the knowledge item, the plan's blocks other than decisions, the house rules and the model. A recorded decision does not change the key of the others; a block does. One not ready shows the question and the protocol sentence at once and the recommendation when it comes; you can already write your own answer.
Decide without waiting: the decision is recorded at once. The blocks that follow are drafted in the background (follow() in src/interview.ts, other open decisions named so that a dependent block says what it assumed) and arrive in the inbox as proposals "Follows from decision D7". A counter shows how many are still being written.
Tasks stream: the findings and the sections are drafted in parallel and appear in the inbox one by one. A section's parts are drafted in dependency order (draftWaves in src/tasks.ts): populations, analyses, then estimands and multiplicity, each against the plan with the blocks already drafted; a proposal records needs (the proposals it was made after) and refuses to be accepted before them, saying which.
Never a bare spinner: the step in words and the elapsed seconds. The providers do not stream here, so the answer is not shown as it is written.
sapling.ai.fastModel: a faster model for the recommendations. Measured: see the report of 0.10.2.

Without AI (`src/noai.ts`)#

Every helper has a deterministic form; the AI adds to it. The interview's options are built from the house rules, Sapling's conventions for each kind of decision, and the patterns of recent SAPs with their counts; the blocks that follow come from the omakase generators (generateFor), as proposals marked "from a template". Review findings and compiler diagnostics with a fix become proposals "from the rules"; "What is missing" and "the next block" use the generators. The plan reads no AI at any step.