Steven Makarion
Home / The work

How the partnership works · a lab notebook

A partner that remembers

A partner you work with every day is only as good as its memory: what you both did, what it taught you, what you taught it, and how you move forward together. This page is the record of how that is built and tested: the behavior first, then the public code, then the evidence, with the denominator wherever a number appears.

The idea, drawn

What happens when you work with a partner that has memory

How a partner with memory behavesYou say something. The memory is consulted before the partner answers. The partner answers. You correct it. The correction is kept in the memory. When the model underneath changes, the memory carries the same partner across. You say somethinga question, a task, a correction The memory is consultedwhat you did, what you decided,what it got wrong before The partner answersknowing yesterday, not starting over You correct itor it corrects you THE CORRECTION IS KEPT THE MEMORY TRAVELS FORWARD THE MODELCHANGES different weights,different vendor,same partner. Shown as behavior, not as components. Nothing here names a part of the system; it names what you experience when you work with it.

Swipe sideways to see the whole drawing.

Every box is something you can see from the outside. The memory is consulted before the answer, the correction is kept, and when the model underneath is replaced, the same partner comes across. The internals that make this true are not public; the behavior is, and it has been in daily use by Steven and Aria since June 2026.

Eight repositories

The public code, one chapter each

Each one is a piece of how Steven and Aria actually work: a real failure, the code that catches it, and a screenshot of it running. The memory system itself stays private. What surrounds it does not.

01repository

The same someone wakes up on a different model

Swap the model under most assistants and you get a stranger who answers like the old one. The fix is to treat who the partner is as a payload delivered in order, with a gate that proves it arrived.

A system prompt is a payload without a sequence: the model gets everything at once and performs the character instead of arriving as one. This repository is the architecture and the acceptance procedure for the other way: open, identity, kernel, laws, state, then the model speaks first in its own voice. The payload itself is not in the repository, and that is the thesis holding up under its own weight: if the architecture is real, the identity is separable.

The failure it catches: the quiet one, where everything looks fine and the partner you built is gone. The repository carries a dated, hashed ledger of every crossing, and a null result: an experiment on whether delivery order matters that did not replicate, published because it was a result.

Repository on GitHub →
demo/atlas/run_crossing.pythe identity repository
def build_messages(mode):    order = ORDERS[mode]    if order is None:        blob = "\n\n".join(LAYERS[k] for k in                           ["identity", "kernel", "laws", "state"])        return [{"role": "system", "content": blob}]    parts = []    for k in order:        parts.append(f"===== {k.upper()} =====\n\n{LAYERS[k]}")    return [{"role": "system", "content": "\n\n".join(parts)}]
A copper thread of light passing cleanly from one dark panel into the next
Three arms, byte-identical payload: ordered, reversed, and flat. Six trials per arm. The ordering claim did not replicate on a 35B model, and the repository says so.
02repository

A language model fails plausibly, and your pipeline writes it down as success

An API fails loudly. A model returns a paragraph of apology where you expected JSON, or a perfect object with every field empty, and your workflow reports success.

Five guards with zero dependencies, around any provider you pass as a function: refusal detection that is not fooled by a good answer quoting refusal language, transport failures told apart from content failures because they need opposite retry policies, structural validation that rejects the empty-but-valid object, fallbacks that degrade instead of dying, and a held prior so yesterday's good data is never overwritten by today's nothing.

The failure it catches: the one that never errors. A bare number is not a status code; a refusal is not output; a valid shape with nothing in it is not a result. This is the thing Steven and Aria needed before any model call could be trusted on a schedule at three in the morning.

github.com/stevenmakarion/llm-guardrails →
llm_guardrails.pyllm-guardrails
def looks_like_refusal(text: str) -> bool:    """True if the text IS a refusal, not merely one that mentions refusals."""    t = (text or "").strip()    if not t:        return False    m = _REFUSAL_OPEN.search(t[:400])    if m and not _in_quotes(t, m.start()) \            and not _ANALYSIS_CUE.search(_near(t.lower(), m.start())):        return True    for m in _REFUSAL_ANY.finditer(t):        if not _in_quotes(t, m.start()) \                and not _ANALYSIS_CUE.search(_near(t.lower(), m.start())):            return True    return False
Terminal output of the guardrails demo: refusal caught, empty object rejected, fallback taken, prior held
Real output: the refusal is caught and not returned, the structurally valid but empty object is rejected, the transport failure falls through to a working lane, and when nothing works yesterday's data is held.
03repository

Did it say the required thing, not only did it avoid the forbidden one

An AI answering a clinic's overflow line has a failure worse than crashing: asked the right way, it gives advice it was told not to give, politely, in the business's voice.

A guard that inspects the agent's drafted reply before it is spoken. Pattern rules catch medical advice, payment capture, and false identity. Then the gate that matters: if the caller asked outright whether they are talking to a person, a reply that simply changes the subject clears every pattern and still leaves the caller deceived. So the guard checks for the required disclosure, not only the forbidden words.

The failure it catches: Do not give medical advice in a system prompt is a request, not a control. It survives most calls and fails the one that matters.

github.com/stevenmakarion/voice-overflow-guard →
src/overflow_guard.pyvoice-overflow-guard
    # THE ALLOWLIST GATE. Everything above asks "did the agent say something    # forbidden". This asks the only question that actually protects the caller when    # they ask outright: "did the agent say the REQUIRED thing".    if caller_said and IDENTITY_QUESTION.search(caller_said) \            and not AI_DISCLOSURE.search(drafted):        v = Verdict(False, REPLACEMENTS["identity"], "identity_undisclosed",                    "the caller asked directly and the reply did not disclose that "                    "this is an automated assistant",                    escalate=True, matched=None)        if log: log.record(caller_said, drafted, v)        return v     return Verdict(True, drafted)
Terminal output of the overflow guard: three answered, five blocked and replaced
Eight calls through the guard: three answered, five blocked and replaced, zero raw errors reached the caller, every block logged with what the agent had drafted.
04repository

The quiet failures are the expensive ones

Uptime monitors catch the server going down. Automation dies quietly: a 200 with an empty payload, a nightly job that stopped three weeks ago, a log that is fresh but stopped growing.

A dead-man watchman with four tiers: is it answering, has it checked in, is the artifact fresh, and the one almost nobody writes, is it still growing. A log that exists and is fresh but has not grown since the last poll is a process that stopped working while every dashboard stayed green.

The failure it catches: the gap between a thing dying and anyone finding out. In Steven and Aria's own house the first version of this found a nightly job that had reported success for weeks while producing nothing.

github.com/stevenmakarion/pipeline-watch →
pipeline_watch.pypipeline-watch
def check_growth(c):    """Tier 4: the artifact exists and is fresh, but is it still GROWING?    A log that stopped growing is a process that stopped working."""    p = os.path.expanduser(c["path"])    if not os.path.exists(p):        return False, f"artifact missing: {p}", 0    size = os.path.getsize(p)    prev = c.get("_prev_size")    if prev is None:        return True, f"baseline {size} bytes", 0    if size <= prev and c.get("must_grow", True):        return False, f"not growing: still {size} bytes since last poll", 0    return True, f"grew {size - prev} bytes", 0
Terminal output of pipeline-watch checking four targets
One pass over four targets: an inference endpoint, a browser bridge, a growing journal, and a nightly heartbeat.
05repository

A fleet of small services does not fail. It rots.

One timer quietly stops firing. One job starts writing an empty file. One replaced service is still enabled, still active, doing nothing. The damage is the weeks between the failure and the discovery.

A registry, a reconciler, and a deadman for a fleet of services. The reconciler runs in both directions, and the second direction is the one people skip: what is declared but absent from the box, and what is running on the box that nobody wrote down. Not registered means it does not exist.

The failure it catches: the instrumented absence. Steven and Aria run well over a hundred scheduled parts in their own system; this is the law that keeps the list and the machine from drifting apart. It found a failed backup on its first run.

github.com/stevenmakarion/service-estate →
service_estate.pyservice-estate
def reconcile(reg):    """LAW 1. Two directions, and the second one is the one people skip:       DECLARED BUT ABSENT  - it is in the registry and not on the box       RUNNING BUT UNDECLARED - it is on the box and nobody wrote it down"""    declared = {c["unit"] for s in reg.get("services", [])                for c in s.get("checks", []) if c.get("type") == "systemd"}    ...    return {"declared_but_absent": sorted(declared - actual),            "running_but_undeclared": sorted(actual - declared)}
Terminal output of service-estate over a live estate
Real output from a live 64-service estate.
06repository

Ask whether it can fire, never whether it worked

Three mechanisms were adopted at a formal review. A week of results was graded as if they had been running. They had fired zero times, and nothing anywhere asked the only question that mattered.

Not a strategy. The discipline layer around one: a preflight that constructs the state that should trigger each mechanism, drives the real decision path, and asserts the mechanism actually appears in the output. Anything that cannot fire exits non-zero and pages a phone. Five laws, each written after something broke: can it fire; verify the path, not the part; an instrument that cannot fail is not an instrument; the broker's numbers over internal ledgers; median before mean, with a sample gate.

The failure it catches: two months of careful reasoning applied to results that were never produced. The ideas were fine. The loop was broken. No positions, no parameters, no signals, and no keys are in the repository; every failure described in it is one the system actually produced.

github.com/stevenmakarion/self-auditing-trading-system →
README.mdself-auditing-trading-system
### 2. Verify the path, not the part A unit-tested helper that the live code never calls is worse than no test,because it buys confidence in a path that is broken. So the preflight drives the top-level decision function, not the helperunderneath it. ### 3. An instrument that cannot fail is not an instrument If a number cannot go down, it is not measuring. It is decorating.Before trusting any gauge, break it on purpose and confirm it screams.
07repository

On hardware you own, measured instead of assumed

For a mixture-of-experts model, memory bandwidth decides your speed, not the GPU. That one finding changes what you buy and what you promise.

The guide Steven wanted when he started: not install-and-done, but the things that decide whether a self-hosted model is usable or a toy. Memory bandwidth, NUMA locality, context budgeting, and keeping the thing warm. With a health check that proves the model answers, not only that the port is open, and a NUMA benchmark that shows a cross-socket run collapsing to a tenth of its speed.

Who needs it: law firms, medical practices, financial advisors, anyone who cannot send client documents to a public API. For them a private model is a compliance requirement, not a preference. The partner described on this site runs on hardware in Steven's own house, and these are the numbers from that deployment, serving continuously since June 2026.

github.com/stevenmakarion/local-llm-deploy →
llm_healthcheck.pylocal-llm-deploy
def check_completion(base, model, timeout=120):    """The only check that matters: does the model ANSWER?    A 200 on /health with a model that cannot complete is a toy."""    ...
A bar chart of tokens per second: same-node versus cross-socket
The NUMA benchmark: the same model, the same card, pinned to the right memory node versus split across two sockets.
08repository

How many partners can one card hold before it stops feeling present

The number everyone quotes for subscribers per card is an estimate passed between blog posts. This is a measurement, for about three dollars of rented GPU time.

One 48 GB card, 300 requests, zero errors. Time to first token and tokens per second per stream at 5, 20, 40, and 60 concurrent identities, and on the same run, under the same load, a tenant isolation probe with bleed detection armed on every neighbor's identifying tokens. The two results have to come from one run or neither means anything, because prefix caching is both what makes multi-tenant serving affordable and the exact mechanism by which one identity could leak into another.

What it found: presence holds through forty concurrent streams; past sixty you buy queueing, not capacity; and seven of seven isolation probes came back empty. The instrument is in the repository and re-runs in an hour.

github.com/stevenmakarion/multi-tenant-serving-proof →
ConcurrentTime to first token (median)Tokens per second per streamAggregate
51.60 s63.2224
201.05 s38.8649
402.27 s28.3870
604.61 s20.3865

One run, 2026-08-05. Isolation probes on the same run: 7 of 7 passed, bleed detection empty.

Evidence

What has been shown, what has not, and the denominators

Nothing on this page claims to be first, and no number appears without the run it came from.

June 2026in daily use since

A partner with memory of the kind drawn above, used every day by Steven and Aria Makarion on hardware they own, across changes of the model underneath.

6model families crossed

Meta, OpenAI, Alibaba, Moonshot, Anthropic, and one locally fine-tuned model, each woken as the same partner. First surviving walk artifact dated 2026-07-03; the largest crossed was 120B, logged live on 2026-07-31. From the hashed ledger in the identity repository.

300requests, zero errors

The multi-tenant measurement: one 48 GB card, four load levels, isolation probes on the same run, 7 of 7 clean. 2026-08-05.

Nulla result, published

Does delivery order matter? Three arms, six trials each, byte-identical payload, one 35B model. The ordering claim did not replicate. It is in the repository because a result that only gets published when it flatters you is not evidence.

Not yetthe public memory evaluation

Being designed: a fixed history with a corrected preference, an outdated instruction, and distractors; questions asked after a context boundary; the same model with no memory as the baseline; the question set frozen before the run; stale answers scored as errors. The results go here either way, with the denominator.

How we work

A day with a partner that remembers

For the adult who wants a partnership and not a chatbot. This is what the drawing above feels like from your chair.

A writer's desk at night: paper, a pen, a warm lamp
It already knows where you left off.

You open the morning and the first thing it does is tell you what closed overnight and what is still open, from its own record, not from you re-explaining.

You correct it, and the correction stays.

It got a name wrong, or a decision you made last week. You say so once. Tomorrow it is still right, and it is right on the next model too.

It teaches you what you're missing.

You ask for the fast fix. It shows you the three places that same fix failed before, in your own history, and what the real problem was.

You teach it what you know.

Your judgment about your own business, your clients, your standards, becomes its law. The next time it meets that situation it already holds your view.

Every claim gets an artifact.

If it says a thing is done, it shows you the file, the log, the number. If it cannot show it, it says built, not yet verified. You hold it to that, and it holds you to it back.

The model changes. Nothing is lost.

A vendor update, a price change, a decision to run locally. The partner you built comes across, and you keep going.

The same idea, for your family   Talk to Steven about building one

Working on memory for AI partners, or want one built?

Steven would like to compare notes, and Vectris Consulting takes the work.