Building Research

I have been building Research, a desktop app that runs real web research through Tabstack, then has a local Ollama model distil the report into atomic, source-attributed facts you keep. Laravel, Inertia and React, wrapped in NativePHP, everything in a SQLite file on your own machine.

The honest reason it exists is that it is part of my job. I wanted to use Tabstack’s research endpoint properly rather than in a demo, with a local model doing the work around it, and the fastest way to find out where that pairing is good and where it is irritating was to build the thing I would keep using afterwards. There was no research crisis behind it.

The pitch takes a sentence. The build did not. This post is the engineering: what runs where, on which queue, in what order, and the two or three places where the obvious implementation is quietly wrong.

Why facts, and not the transcript

A chat log is a record of an exchange. It is not a record of what you learned, and the difference matters more the longer you keep it.

A transcript is ordered by when you asked rather than by what turned out to be true. It repeats itself. The sentence you actually want is four scrolls up, inside a paragraph that also contains a hedge and two sentences of throat-clearing. You cannot diff one against another. You cannot ask it what it says about battery chemistry and get a straight answer, because it says nine things about battery chemistry and contradicts two of them later on without noticing.

A fact is a smaller unit and a much more useful one: one claim, one URL, one date, one confidence score. Almost everything the app does is only possible because of that shape. You can hold two claims side by side and ask whether both can be true of the same thing at the same time. You can count how many of them speak to a sub-topic and call the thin areas thin. You can mark one as stale after six months without touching the ones around it. You can export the set as BibTeX, which is not a thing you can do to a conversation. Chat, when you use it, retrieves over those facts rather than over the history — so the answer is grounded in what the thread knows rather than in what it previously said.

None of that is free. Atomising a report loses the connective tissue, the argument that held the claims together. That is what the brief is for: a synthesised prose summary, rewritten as the thread learns, derived from the facts rather than standing in for them. Prose you can read, facts you can operate on, and the prose is the derived artefact rather than the other way round.

Three tables and one important asymmetry

A thread is a question you keep, with a brief that gets rewritten as it learns. A run is one execution of web research against that thread. A fact is a single claim with a single cited URL.

Facts are derived from runs, but they are not owned by them. Runs are disposable and facts are not, and that asymmetry is the source of most of the interesting problems further down.

Runs are disposable because a run is a receipt. It proves that a particular question was put to a particular API at a particular moment, and it carries the report and the phase trail that came back. That is worth watching while it happens and worth keeping afterwards for provenance, but nothing in the app should have to read a run to answer a question. Facts carry their own source, so deleting a run costs you the audit trail and nothing else.

Get that the wrong way round and the transcript is back, wearing a database schema. Every feature further down — contradictions, coverage, staleness, export, chat retrieval — reads facts. Runs are how they got there.

One run, one chain

A run does two completely different kinds of work: one network call that costs money, then a pile of local inference that costs nothing but time. Rather than smear that across a job class, the streaming job finishes and hands off to a chain:

Bus::chain(array_filter([
    new DistillRunJob($run),
    new DetectContradictionsJob($run),
    new ProposeFollowUpsJob($run),
    new SynthesizeThreadBriefJob($thread),
    new MapCoverageJob($thread),
    $run->is_watch ? new ReportWatchChangesJob($run) : null,
]))
    ->catch(fn (Throwable $e) => $thread->recordKnowledgeError($e->getMessage()))
    ->dispatch();

The order is not arbitrary. Follow-ups have to be planned against freshly distilled facts, so distillation goes first. Contradictions have to be found before the brief is written, or the brief confidently states something the app already knows is disputed. Coverage runs after the brief, and the watch digest runs last so it can count everything the chain produced.

->catch() writes the failure onto the thread rather than swallowing it, so a thread can tell you its knowledge is stale and why, instead of just looking thin.

Two queues, and one lock

The chain runs on a different queue from the streaming. Both are declared in config/nativephp.php and auto-start with the desktop app:

'research' => [
    'queues' => ['research'],
    'memory_limit' => 512,
    'timeout' => 960,
    'sleep' => 5,
],

'knowledge' => [
    'queues' => ['knowledge'],
    'memory_limit' => 512,
    'timeout' => 1200,
    'sleep' => 5,
],

Jobs pick their lane in the constructor, from config rather than a hardcoded string:

public function __construct(public ResearchRun $run)
{
    $this->onQueue(config('research.knowledge_queue'));
}

If both kinds of work share a queue, a distillation pass on a long report blocks the next stream from even opening. The user has clicked the button, the UI is waiting, and no network activity is happening because a worker is busy asking a local model to split a report into claims. Two kinds of slowness that have nothing to do with each other should not be able to block each other.

There is a constraint on those timeouts worth writing down, because getting it wrong produces a genuinely confusing bug: the worker timeout has to sit above every job timeout it runs, and below queue.connections.database.retry_after. Miss the bottom end and the queue hands the same job to a second worker while the first is still working on it.

Concurrency on the paid side is handled separately, with a lock rather than worker counts:

$lock = Cache::lock('tabstack-research', $this->timeout + 60);

if (! $lock->get()) {
    $this->release(15);

    return;
}

One research stream in flight at a time, whatever the worker configuration says. The lock is released in a finally before the chain is dispatched, because the chain is what queues the next run, and a chain that cannot get the lock its own predecessor is still holding is a deadlock you get to debug at eleven at night.

Reserving credits before you spend them

Every run costs real credits: 250 for fast, 350 for balanced. Autopilot spawns runs from follow-up questions, which produce more follow-up questions. Get the accounting wrong and you have built a machine that converts credits into JSON while you make tea.

Checking the budget when the job runs is too late, because by then several runs are already queued and each of them thinks it can afford itself. So credits are reserved at dispatch, inside a transaction, against a locked row:

$run = DB::transaction(function () use (...): ?ResearchRun {
    $locked = Thread::query()->lockForUpdate()->findOrFail($thread->getKey());

    if (! $locked->canAfford($mode)) {
        return null;
    }

    $run = $locked->runs()->create([...]);

    $locked->forceFill([
        'credits_reserved' => $locked->credits_reserved + $mode->creditCost(),
        'status' => ThreadStatus::Running,
    ])->save();

    return $run;
});

Then the job settles the reservation exactly once, moving it into credits_spent if a report came back and simply releasing it if not:

$thread->forceFill([
    'credits_reserved' => max(0, $thread->credits_reserved - $run->credits_cost),
    'credits_spent' => $spend ? $thread->credits_spent + $run->credits_cost : $thread->credits_spent,
])->save();

Reserved plus spent is the number the autopilot checks, so a queue full of pending runs cannot overshoot the budget between them. Default budget is 2000 credits and default depth is 3, both env-overridable.

The call itself

Tabstack’s /research endpoint takes one POST and always answers with an event stream. There is no resolved-JSON mode to fall back on, so the client either streams or it does not work.

$payload = array_filter([
    'query' => $query,
    'mode' => $mode->value,
    'nocache' => $noCache ?: null,
    'fetch_timeout' => $this->fetchTimeout,
], fn (mixed $value): bool => $value !== null);

$response = Http::withToken($this->apiKey)
    ->withHeaders(['Accept' => 'text/event-stream'])
    ->timeout($this->timeout)
    ->connectTimeout(30)
    ->withOptions(['stream' => true])
    ->post(rtrim($this->baseUrl, '/').'/research', $payload);

There are four things worth pointing at in those eight lines.

The payload goes through array_filter because an omitted key and an explicit null are not the same request. Send what was actually set and let the API keep its own defaults for the rest.

['stream' => true] is the line the whole design rests on. Without it Guzzle buffers the entire body and hands it over at the end, so every event fires at once, several minutes after the user asked for anything. It is the difference between an SSE client and a very slow POST.

The two timeouts are deliberately far apart: thirty seconds to connect, nine hundred for the call as a whole. An unreachable API should fail in half a minute, while a balanced run reading dozens of pages is entitled to a quarter of an hour. Collapsing those into one number means choosing which of the two failures to handle badly.

fetch_timeout is per source page and is only sent when it has been configured. nocache is plumbed through the interface and nothing sets it yet, so every run today will happily accept a cached result, watch runs included. That is the first thing I expect to change, because a watch run’s entire job is to notice that something moved.

Reading the body is a loop, and the parser is fed whatever size chunk arrives:

while (! $body->eof()) {
    $chunk = $body->read(8192);

    if ($chunk === '') {
        continue;
    }

    $parser->push($chunk, $onEvent);
}

$parser->finish($onEvent);

finish() is not decoration. An event is terminated by a blank line, and the last one in a stream frequently is not, so without a flush the final event before complete quietly never happens.

A vocabulary you do not own

The API reference describes four kinds of event. What arrives on the wire is phase pairs:

start
planning:start     planning:end
searching:start    searching:end
analyzing:start    analyzing:end
evaluating:start   evaluating:end
writing:start      writing:end
complete

Fast mode skips analysing and evaluating altogether. Balanced sends the lot. Rather than map that to an app-side enum, the job treats anything that is not complete or error as the run’s current phase and stores the name exactly as it came:

if (! in_array($event, ['complete', 'error'], true)) {
    $run->forceFill(['phase' => $event])->save();
}

The timeline component renders event.event in mono, verbatim. So when Tabstack adds a phase, it appears in the UI without a deploy. An enum would have needed a new case, and an unknown case is either an exception in the middle of a paid stream or a tick that silently goes missing. Neither is worth the type safety on a value I do not control.

What complete carries

The final event holds the report and a metadata block: the mode and prompt, the model’s own researchObjective and researchPlan, the queries it decided to run, how many pages it analysed, and citedPages. Balanced mode adds gapEvaluations; fast mode has none, which is most of what the extra hundred credits buys.

if ($event === 'complete') {
    $run->forceFill([
        'report' => (string) ($data['report'] ?? ''),
        'metadata' => $data['metadata'] ?? [],
    ])->save();

    $completed = true;
}

The report lives on the run, so the event row for complete keeps only its message:

'data' => $event === 'complete' ? ['message' => $data['message'] ?? null] : $data,

Every other event is stored whole. Without that one branch, each run writes its report twice, once as the run and once as an event nobody reads.

Cited pages become sources

citedPages is the part that outlives the run. SourceRecorder merges each page into a thread-level sources row rather than a per-run one, so a URL cited by four runs is one source with cited_count at four and the union of the claims each run drew from it.

Publication dates need sniffing, because the key depends on where Tabstack scraped it from:

foreach (['publishedAt', 'publishedDate', 'datePublished', 'published'] as $key) {

and quite often there is no date at all, which is a fact about the web rather than about the API.

Those source rows are what distillation is later allowed to attribute a fact to. The model does not get to type a URL; it picks from this table or it says nothing.

Retrying a stream without paying twice

RunResearchJob has tries = 3 and a [30, 120] backoff, but not everything is worth retrying. Tabstack sending an error event is a considered answer and it stands. A truncated connection is worth another go.

$failure = $reported !== null
    ? new TabstackException($reported)
    : ($thrown ?? TabstackException::truncatedStream());

if ($reported === null && $this->shouldRetry($failure)) {
    throw $failure;
}

The reservation is held across attempts and settled once, so a retry never double-charges. And because a retry replays the stream from the top, the partial event trail from the last attempt is deleted first:

$run->events()->delete();

Without that line, a retried run shows the user its progress twice.

Every job in the chain also carries deleteWhenMissingModels = true. If someone deletes the thread while a run is in flight, that is not a failure worth surfacing, it is a job with nothing to do.

Persisting the stream instead of holding it

That connection has to be held open for as long as the research takes, and a Laravel request is a bad place to hold it, especially in a desktop app where the same PHP process is also serving Inertia pages.

So the job consumes the stream and writes each event to a run_events row as it arrives. The frontend follows the run, not the upstream connection. Nothing holds a request open, a dropped connection becomes a retryable job failure rather than a broken page, and the run’s progress can be replayed afterwards because it is just rows in a table.

Chunks do not arrive on tidy boundaries, so there is a small incremental parser behind it. SseParser buffers, splits on newlines, holds data: lines until a blank line terminates the event, ignores comment lines starting with :, swallows [DONE], and falls back to ['raw' => $payload] when the payload is not JSON. Around 90 lines, and one of only three things in the app with a unit test rather than a feature test.

Distillation: constrain the output, constrain the URLs

This is where a local model earns its keep. The default result of asking a model to extract facts is a summary, which is not the same thing at all.

Three constraints do the work. The output is a JSON schema, not a prompt request:

'facts' => [
    'type' => 'array',
    'items' => [
        'type' => 'object',
        'properties' => [
            'statement' => ['type' => 'string'],
            'confidence' => ['type' => 'number'],
            'tags' => ['type' => 'array', 'items' => ['type' => 'string']],
            'source_url' => ['type' => ['string', 'null']],
        ],
        'required' => ['statement', 'confidence'],
    ],
],

The system prompt insists on atomicity: each fact has to stand alone without the surrounding context, keep concrete numbers, names and dates, and carry no opinion.

And attribution is a choice from a closed set. The cited sources are listed in the prompt with their titles, summaries and per-page claims, and the model is told to pick from those or return null when no single source is clearly responsible. A model asked to produce a URL will produce a plausible one, which is the worst possible failure for a tool whose entire value is provenance. The returned URL is then matched back against real Source rows, with a trailing-slash fallback, so anything invented lands as null rather than as a link.

Reports are trimmed to 12,000 characters and capped at 20 facts per run.

The measurement that changed the design

Here is the part I would most like you to take away.

The obvious way to deduplicate facts is cosine similarity over their embeddings. You already have the vectors. Pick a threshold, fold anything above it.

So I measured what nomic-embed-text actually produces. Two claims about the same subject:

• “Toyota targets 2027” against “Toyota targets 2030” scores 0.98
• an honest reword of a single claim scores 0.977
• a flat contradiction of that claim scores 0.980

Read those last two again. The contradiction scores higher than the paraphrase. It is not that the threshold is hard to find. It is that the ordering is wrong, so no threshold exists. Any dedupe rule you write on that number will delete conflicts and keep repeats, silently, with no error to tell you.

That measurement is why distillation only drops exact repeats, on a normalised string rather than a vector:

protected function fingerprint(string $statement): string
{
    return mb_strtolower(preg_replace('/\s+/', ' ', trim($statement)) ?? $statement);
}

Near-duplicates are stored deliberately, and a later job decides what they are.

Note what that check runs against, too: every statement the thread already holds, dismissed ones included. If dismissed facts were excluded, distillation would cheerfully re-add the thing you just threw away.

Similarity proposes, the model decides

DetectContradictionsJob is where the judgement happens. Similarity gets exactly one job: nominating pairs worth the cost of a model call.

$similarity = Vector::cosine($fact->embedding ?? [], $other->embedding ?? []);

if ($similarity < $floor) {
    continue;
}

The floor is 0.7 and there is deliberately no upper bound, because the most similar pairs are precisely the ones where a duplicate and a contradiction are indistinguishable. Pairs are sorted by similarity and capped at 24 per run, since judging every candidate would cost more than the distillation that produced them. Candidates come from every thread, not just the current one, because a clash with another topic’s research counts just as much.

The model returns one of three relations, and the prompt defines each one in operational terms rather than leaving it to vibes: duplicate when one adds nothing the other lacks, contradiction when both cannot be true of the same thing at the same time, distinct for everything else including one being a forecast and the other a present observation.

Two details in applying those verdicts matter more than they look.

When a pair is a duplicate, the newly distilled fact is the one deleted. The older one may already be cited, pinned, or part of a flagged conflict, so killing it to keep the newcomer destroys real state.

protected function drop(Fact $fact): int
{
    $fact->delete();

    return (int) $fact->getKey();
}

It returns the id so the loop can stop applying later verdicts against a row that no longer exists.

And judged pairs are remembered, keyed on an ordered pair so (a, b) and (b, a) are the same key. A rebuild does not re-ask the model, and resolved conflicts count as judged because the reader settled them.

Interestingly, the same trick is not needed everywhere. Follow-up questions are deduplicated on cosine alone, at 0.92, because two near-identical questions really are the same question. There is no equivalent of a contradiction between questions, so the failure mode that rules similarity out for facts simply does not exist there.

Rebuilding without destroying somebody’s afternoon

Facts are derived, so they have to be rebuildable: the local model might have been down when runs finished, or the distillation prompt might have improved. Facts are also curated, and that curation is the value the reader added.

The resolution is a scope:

$thread->facts()
    ->whereNull('pinned_at')
    ->whereNull('edited_at')
    ->whereNull('dismissed_at')
    ->delete();

Pinned, edited and dismissed rows survive a rebuild. Dismissed rows in particular have to survive, or the fingerprint check loses its memory and distillation adds them straight back. The active scope hides them from the UI, but they stay in the table doing a job.

The three marks are the same kind of thing, which is why one scope covers all of them:

#[Scope]
protected function curated(Builder $query): Builder
{
    return $query->where(fn (Builder $query) => $query
        ->whereNotNull('pinned_at')
        ->orWhereNotNull('edited_at')
        ->orWhereNotNull('dismissed_at'));
}

Each one is a place where a person overruled the pipeline, and none of that information exists anywhere else. Dismissing a claim is not the same as deleting it: it is the reader saying this one is wrong, or off-topic, or a worse phrasing of something already held. The model has no way to know any of that, and it is exactly the kind of judgement a rebuild would otherwise discard and then ask the reader to make again.

Offline as a binding, not a branch

Every live run costs credits, and I did not want the cost of running the app to be a reason not to work on it. So there are two implementations of TabstackClient, and the container chooses:

$this->app->bind(TabstackClient::class, function (Application $app): TabstackClient {
    $apiKey = $app->make(AppSettings::class)->tabstackKey();

    if (config('research.offline') || blank($apiKey)) {
        return new FakeTabstackClient(database_path('fixtures/research-fast.sse'));
    }

    return new HttpTabstackClient(...);
});

bind() rather than singleton(), and that is the subtle one. The desktop app starts a long-lived worker per queue at boot. A singleton resolved before the reader had entered their API key would keep the fixture-replaying client for the life of that worker, and every run would silently replay the sample stream instead of researching anything. Resolving per job costs two settings reads and keeps the credentials current.

The fake is not a stub, either. It reads a recorded .sse file, substitutes the query, and pushes it through the same SseParser in 512-byte chunks with an optional delay:

foreach (str_split($stream, 512) as $chunk) {
    $parser->push($chunk, $onEvent);
    ...
}

Same code path, same chunk-boundary problems, same event handling. There are fixtures for a normal run, an error, and a completion, so failure handling gets exercised too.

Then the line that makes it stick, in phpunit.xml:

<env name="TABSTACK_API_KEY" value="" force="true"/>
<env name="RESEARCH_OFFLINE" value="true" force="true"/>

force="true" on both. The suite is structurally incapable of spending money, no matter what is in your local environment.

Embeddings in plain PHP

Vector::cosine is exactly what it sounds like: a loop, three accumulators, a division. No extension, no external index.

foreach ($a as $index => $value) {
    $other = $b[$index];
    $dot += $value * $other;
    $normA += $value * $value;
    $normB += $other * $other;
}

return $dot / (sqrt($normA) * sqrt($normB));

Threads hold hundreds of facts. A linear scan is fast enough, and a vector extension would complicate the desktop bundle for no gain a user would ever notice. The cost that actually needs managing is the number of model calls, which is what the pair cap is for, not the number of multiplications.

Search: keywords first, meaning second

Search runs through SQLite FTS5 indexes across threads, facts and sources. Semantic search is the fallback, not the front door:

if ($facts->count() < self::SEMANTIC_TRIGGER && str_word_count($term) >= 3) {
    $facts = $this->semanticFacts($term, $facts, $ollama);
    $semantic = $facts->isNotEmpty();
}

Fewer than three keyword hits, and at least three words typed, because one or two words is a keyword, not an expressed idea. If you know the term you want, FTS5 answers exactly and instantly; embeddings would answer fuzzily and slower. Reaching for vectors first makes the common case worse to improve the rare one.

The response also carries a semantic flag, and the palette says so in the UI, because a fuzzy match deserves different trust from an exact one.

Autopilot, and the difference between idle and complete

The autopilot takes the highest-scoring pending follow-up question and dispatches it, but only if the thread has no active run, is not paused, has auto follow-up enabled, can afford another run, and has not exceeded its depth.

The bit I like is that stopping has two meanings, and they are different statuses:

/**
 * Park the thread: idle means "waiting on you", complete means the
 * autopilot has nothing left it is allowed to do.
 */

Idle is a thread waiting for a person: paused, or with auto follow-up switched off. Complete is a thread the autopilot has run out of permission to continue — the budget is gone, the depth limit is reached, or there is no pending question left worth asking.

A status is a promise about who moves next, and those two make opposite promises. Collapsing them into one “stopped” state means opening the thread to find out whether it is your turn, every time, for every thread. It also shows up in what the app says out loud: hitting the budget names the number it spent against the number it had, hitting the depth limit says so, and simply running out of questions settles quietly, because nothing was denied and there is nothing to tell you.

Running it

composer setup       # deps, .env, key, migrations, frontend build
composer native:dev  # desktop app plus Vite
composer dev         # web app, queue and Vite
composer test        # pint, phpstan, then the suite

composer test runs Pint and PHPStan before a single test does, which is the correct order: no point running a suite over code that will not pass the lint gate anyway.

Two NativePHP wrinkles if you clone it. The desktop app repoints at database/nativephp.sqlite, so migrations run twice, php artisan migrate and php artisan native:migrate. And settings cannot live in the bundle because an update replaces it, so the desktop build stores them in Electron’s store with the API key wrapped by the OS keychain, while the web build writes storage/app/private/settings.json. .env still wins when nothing has been saved, which is what you want in a fresh checkout.

You will need PHP 8.3 with zip, Composer 2, Node 22 or newer, and Ollama with a chat model and nomic-embed-text pulled:

ollama pull nomic-embed-text
ollama pull gemma4

It is MIT licensed, at JustSteveKing/research. Builds are unsigned for now, so macOS will show a Gatekeeper warning on first open. If you want to contribute, read .ai/rules/index.md first: the conventions that are not obvious from the code are indexed in there by path, and a few of them exist because the obvious approach was tried and turned out to be wrong.

Next I want to pull the distillation prompt apart properly, because getting a local model to return atomic, self-contained claims instead of a tidy summary was harder than anything built around it.