DeepSeek Harness: A Week Building a Lead-Sourcing Prototype

Summary: We built a lead-sourcing prototype on DeepSeek Harness (@deepseek-ai/dsh 0.1.0-rc.7), pinned seven days after the package's first-ever release. The harness runs the agent loop, tool calling, streaming and sessions; our own code makes every scoring decision. We evaluated Cloudflare OS first and moved on in ninety minutes, and we kept our chat agent on Mastra rather than migrating everything to one place.

We wanted one screen. Ten companies worth reaching out to, each with a reason attached, and a contact where we could find one.

That was the whole brief. Not a CRM, not an outreach engine. Just close the gap between "a lead exists somewhere on the internet" and "it's on my screen with a reason attached." Everything after that is a salesperson's job.

We gave ourselves about a week to find out whether it was buildable. What came out of that week is a prototype. Point it at a description of the companies we sell to, and it searches, looks each one up, scores them, and fills a screen we call "This morning". It has done one full sweep, end to end, triggered by hand.

Most of that week went on a question we didn't expect to be the hard part: what runs the agent. That's what this post is about.

What Is an Agent Harness?

An agent harness is the layer around a language model that lets it actually do things. It holds the model's instructions, hands it a set of tools it's allowed to call, and runs the loop (pick a tool, run it, hand the result back, ask again) while keeping the conversation history and deciding when the run is over. That loop is what turns a model into an agent that can observe, decide and act.

If you've used Claude Code, Codex or Cursor, you've used an agent harness. The model isn't the product. The harness around it is.

Why We Didn't Build Our Own Agent Harness

Because we hadn't built one before, and that seemed like a bad reason to start.

We could have designed our own loop and found out six months later which of our assumptions were wrong. Instead we went looking for one built by people who'd already worked through those problems. If your agent's shape is unusual enough that nothing off the shelf fits, writing your own is a fair call. Ours wasn't unusual. Run a defined task against our own pipeline, call some tools, hand back structured results.

Knowing which problems are actually yours to solve is most of the job.

Building the First Version on Mastra

The first thing that worked ran on Mastra, an open-source agent framework, and it worked from close to day one. It was never a script or bare calls to a model. The earliest commits already had Mastra as the harness, wired directly into our NestJS service as a shared Mastra instance imported by our services.

Real agents ran on it. A sourcing agent, a sales agent, tools for research and enrichment, the scoring rubric, a chat copilot. End to end, it did the job.

What sent us looking again wasn't a failure. It was that the harness underneath was something we'd designed ourselves, on our first attempt, with no prior experience of the problem. That's a fine place to start and an uncomfortable place to stay. We wanted one that had been built and tested by a team that had done this before.

Cloudflare OS: Why We Evaluated It and Moved On

Cloudflare OS was the first alternative we tried. It's Apache-2.0 licensed and genuinely well-built, and it turned out to be solving a different problem than ours.

It describes itself as "an operating system for AI workloads, analogous to the sense in which a traditional operating system manages compute workloads." Ask it for a document or a dashboard and the runtime creates a private, sandboxed instance of that app just for you. Cloudflare calls each one a Gadget. Anything with a real-world side effect goes through a Gatekeeper, which applies capability-based security rather than access-control lists: the action gets simulated locally so the agent can keep working, and you approve or reject the queue afterwards, in bulk or one at a time. The security model is worth reading whatever you end up using.

But we weren't trying to hand anyone a sandbox to build apps in. We needed something we could call from our own pipeline: run this agent, with these tools, give me structured results back. Cloudflare OS isn't packaged as an embeddable library like that. It's an application platform. The natural way to get what we wanted would have been to rebuild our sourcing agent inside it rather than wiring up a persona and a set of tools directly in our own service.

We spent about ninety minutes on it. Most of that went into getting local development running, including wiring up a Slack Gatekeeper to see the approval flow, which is one optional integration among several rather than a requirement. We never reached the agent layer at all. In hindsight, the distance between npm install and "my agent ran" told us most of what we needed to know about the fit.

Ninety minutes is a cheap way to find out something isn't for you. The expensive version of that mistake is three weeks in.

DeepSeek Harness: What It Handles Out of the Box

The name, so nobody gets it wrong in a slide: DeepSeek Harness. The CLI and package are dsh, published under the @deepseek-ai scope, MIT licensed, tagline "Everything is a Plugin." We pinned 0.1.0-rc.7, which went out exactly seven days after the package's first-ever release. That is how old the thing was when we started building on it.

What it handled that we'd otherwise have written: a working agent-execution protocol (spawn, handshake, session, turn, over JSON-RPC), plus the tool-calling loop, live token-by-token streaming, session persistence to disk, and clean process isolation between concurrent runs.

The tagline is the real point though. The loop, the streaming, the session handling, the model connection: all DeepSeek's. Anything specific to our business is a plugin, a small set of tools the agent can call to find a lead, enrich it, score it. We stopped maintaining an agent-execution engine and started maintaining a handful of domain-specific tools that plug into one.

Not everything moved inside the plugin system. Because the harness runs as a subprocess per task rather than as an in-process library, we still wrote a small service that spawns it, authenticates each run, and cleans up afterwards.

DeepSeek Harness 0.1.0-rc.7: What Was Rough

Building on a seven-day-old release costs you a real day. None of this stopped us, but it's worth knowing where that day goes.

The most direct path was the wrong one. The CLI's headless mode speaks a completely different protocol than the one we needed. Our first attempt failed outright, and we found the real mechanism buried in an SDK example rather than the documented path working as described.

Personas silently didn't apply. The intended way to define an agent's persona only worked in the browser UI. In the headless mode we needed for production it quietly had no effect. Nothing errored, nothing warned. We only found out because we tested for it directly, which is not a thing you'd normally think to test.

Only the demo wires it together for you. The component that composes a custom agent's persona and tools together was, at our version, only available as a demo bundle. The underlying primitives exist as real product packages, but nothing pre-built composes them the way the demo does, so we hand-assembled that composition ourselves. The docs have moved since, so check the current state rather than taking our word for it.

A packaging bug we had to route around. Several of the harness's own sub-packages had their npm latest tag pointing backwards at broken early releases. @deepseek-ai/dsh-base, dsh-headless, dsh-session, dsh-web-app and dsh-scope were all pinned to the very first release while next had moved several versions ahead. Install one by name and you silently get something old and broken. We isolated our install into its own workspace to get past it. The tags are public on npm, so check before you assume it's fixed. It outlived several releases.

No compatibility promise, and they say so plainly. The README's warning isn't a footnote, it's in capitals: "THERE WILL BE COMPATIBILITY-BREAKING CHANGES." The safety notes describe it as experimental developer-preview software that shouldn't be treated as secure or production-ready. Fair warning, clearly given. It's a reason to size the work honestly, not a reason to stay away.

Running Mastra and DeepSeek Harness Together

We went in expecting to pick a winner. We came out running both, and that turned out to be the better answer.

DeepSeek Harness took the sourcing and hydration agents: short, bounded, scheduled jobs with a clear finish line. The chat copilot stayed on Mastra. In our setup the harness runs as a subprocess per run rather than as an in-process library, so an unbounded number of concurrent chat users would each have cost us a full process. A different integration might carry that differently; that's what it cost us, and we decided not to take it on.

So each agent sits on whichever harness fits its shape of work. Nobody had to be right.

Inside the Agent Loop: How It Decides What to Do Next

The agent has a small, fixed set of things it can do (search the web, look up a company's details, save what it found) and it works through them one at a time.

The loop isn't ours. The harness owns the mechanics, along with retries, conversation history, and running a few tool calls in parallel. What we supply is three things: the brief, in plain English; the tools, along with a strict check on every value the agent tries to save; and a config file naming the model and the tools to load.

Every turn, the model sees its instructions, everything it's already done and found, and the list of tools available. It replies with either "run this next" or nothing further. We don't script the order. A company with a detailed careers page might take three searches. A dead end takes one.

The loop ends when the model stops asking for tools, and we make that land somewhere sensible by design. Each agent has exactly one tool that saves anything, and the instructions are blunt about it:

When you're done researching, call record_hydration_facts exactly once with everything you found. This is the only thing that actually saves your work. Nothing else in this conversation persists anywhere. An empty array is a valid, useful answer.

So "done" means the agent has saved its findings and has no further searches it wants to run. Nothing it says in the chat matters, only that one call. That's our design, not a harness feature. The harness just ends the loop when the model goes quiet; the reason it goes quiet in the right place is that we gave it one job and one way to finish it.

Keeping it from spiralling is mostly a matter of keeping the task small: "research this one company," "run today's search," never "find us customers." The instructions cap it further: at most fifteen companies, save exactly once, only report what the search returned. Each search returns at most ten results. Every value the agent tries to save is format-checked before it's allowed through. The research agents run on a low-cost model, the sweep is meant to run once per weekday, and each run is an isolated process that shuts down when it's finished.

What we don't have yet is a hard cap on steps, a spending cap per run, or a timeout. A stuck run currently leans on those softer limits rather than a firm stop. A step counter, a per-run budget and a timeout are all on the list, and we'd rather say so than imply the fuller version already exists.

Two guardrails are worth showing. The agent can't invent a company to pad its results, because the save tool rejects anything that isn't a real web address:

if (!DOMAIN_RE.test(domain)) {
  throw new Error(
    `"${candidate.domain}" is not a real domain (e.g. example.com). ` +
    'Omit this candidate entirely if you cannot confirm one. Do not invent a placeholder.',
  )
}

And it doesn't choose what it overwrites. Which company's record a run writes to is set by our code before the agent starts. It isn't a value the model gets to pass in:

// Which record this run updates is our decision, set before the AI starts.
const leadId = process.env.DSH_HYDRATION_LEAD_ID
if (!leadId) throw new Error('no target record set')

AI Lead Scoring: Why Ours Runs Without an LLM

Once the agent has saved its findings, there's no AI left in the pipeline. Merging, enrichment, scoring and building the morning list are all plain code.

The scoring rubric has no LLM and no I/O. Fixed weights:

Plus hard disqualifiers for geography, industry and size. Below the threshold, a lead is dropped. A funding round is worth 7 of a possible 15 on the compelling-event axis, and that weight decays with age. A fresh round on its own doesn't get anyone onto the list.

This is the decision we'd defend hardest, and it came from thinking about the person on the other end rather than the architecture. Someone has to spend their morning on this list. If they can't interrogate why a company is on it, they'll stop trusting it by the second week, and a list nobody trusts is worse than no list at all.

So the model does the open-ended work of turning the messy web into structured facts, which is what it's genuinely good at. Every consequential decision after that is arithmetic we can read, test and argue about in a pull request. When a salesperson asks why a company scored what it did, there's an answer, and it's the same answer tomorrow. Changing a weight is a code review, not a prompt tweak.

Sourcing Companies with Exa and Apollo

Every company name starts with Exa, a web-search API whose semantic and find-similar-by-URL search is what we lean on rather than its keyword matching. Apollo comes in later, and only to fill in size and industry once we already have a name. It isn't where names come from.

What we ask for is close to the opposite of what people assume. Not recently funded startups. Established companies stuck on old technology and hiring to deal with it.

So each run fires several differently-worded searches. Job posts naming a specific old technology: Delphi developer, VB6 migration engineer, .NET Framework to .NET 8, AS400 RPG developer, PowerBuilder maintenance. Modernization projects: legacy system modernization, replatform monolith, migrate off mainframe, core system rewrite. A newly hired CTO, CIO or VP Engineering at a mid-size company, which is the strongest single sign that a rewrite budget is coming. Private-equity acquisitions or recapitalizations at companies of the right size, on the working assumption that PE ownership tends to put modernization on a deadline. And repeated hiring for the same legacy role.

Every search is date-limited to roughly the last three weeks, so we're reading current postings rather than archives. We don't filter by country in the search itself. Geography is applied later, as an allow-list in each customer profile. And a React job isn't a target at all. React means a modern stack, which is the wrong signal for us.

What Comes Back That We Can't Use

Plenty. The most common problem by far is no contact email. Exa gives us a company and a role, rarely a named person, almost never an address. Close behind is the wrong domain, because a result usually sits on a job board or a news site rather than the company's own site. Then the same company twice, either across two searches in a single run or one already on the list from an earlier one. Stale news, funding coverage that's months old and republished. Companies that turn out to be nothing: an agency or staffing firm rather than an end customer, a business far too big or too small, a page that mentions a technology in passing with nothing behind it. And padding: left unchecked, a model asked for "up to fifteen companies" will round up with weak guesses.

Some of that gets handled before anything is recorded. Every search carries a date window, so old news mostly never enters, and even when a slightly older event slips through, its weight in scoring fades with age. Before recording anything, the agent pulls the current lead list and is told to skip companies already on it; underneath, companies are keyed by their website, so two records for the same domain collapse into one. That last part was a real bug once. A duplicate got in under a concurrent run, and the fix was to make it a single, all-or-nothing database write.

The rest is handled by being explicit in the brief. Confirm the company's own site, running an extra search if you have to, and drop the candidate rather than guess. Only report what the search returned: a thin morning means fewer companies, never invented ones, and any technology the agent claims has to be backed by a word-for-word quote from the source.

Two things deliberately don't happen. A missing email doesn't bin the lead. A good company with no address is still scored and ranked, and routes to a "find a contact" queue rather than the bin. And size and industry come from Apollo rather than a web guess, with its numbers overriding anything the agent read off a page; if Apollo has nothing, it says so cleanly rather than failing.

Mostly, though, the scoring step is the real filter. Agency, wrong size, wrong country, no genuine evidence of an old stack: all of it gets caught afterwards by fixed rules in plain code. A company that's nothing scores near zero and never reaches anyone's screen.

What We'd Tell Another Team Building on DeepSeek Harness

Three things, having done it.

Using someone else's harness was right for this. The loop, the streaming, the session handling, the process isolation. All problems other people had already solved, and none of them ours to solve again.

Adopting a pre-1.0 tool costs a real day, and not in the code you write. It goes into finding the path that works when the documented one doesn't, and into testing assumptions you'd normally take for granted, like whether your persona is being applied at all. Budget for it and it's fine. Don't, and it looks like the tool is broken when really it's just young.

That's what it costs to build with. Shipping on it is a separate question, and our answer is not yet. Everything above is us getting it working well enough to run internally. DeepSeek's own docs say it isn't secure or production-ready, and having used it, we'd say the same. Experiment on it, learn its shape, keep your integration isolated the way we did. We wouldn't put it in front of a customer, or let revenue depend on it, until there's a stable release.

And you don't have to pick one. Running both was the most useful decision we made.

This is still early, and if something fits better next month we'll move. The part we'd keep either way is the split: the model does the open-ended work at the edges, and plain, readable code makes every decision that matters.

We build agent systems for teams that need them working in production rather than in a demo. If you're weighing up harnesses, wiring agents into a service you already run, or working out which decisions should stay in plain code, that's what our AI agents team does.

And if you're building on DeepSeek Harness yourself, or you think I've got something wrong in here, I'd like to hear about it. You can find me on LinkedIn.

FAQs

What is DeepSeek Harness? An open-source agent harness from DeepSeek, distributed as the dsh CLI and package under the @deepseek-ai npm scope, MIT licensed. Its tagline is "Everything is a Plugin": the harness handles the agent loop, tool calling, streaming, sessions and the model connection, while anything application-specific is written as a plugin.

Is DeepSeek Harness production-ready? Not by its own account. The README states in capitals that there will be compatibility-breaking changes, and the project's safety notes describe it as experimental developer-preview software that shouldn't be treated as secure or production-ready.

What's the difference between an agent harness and an agent framework? In practice the terms overlap. The distinction that mattered to us was scope: we wanted a primitive we could call from our own pipeline (run a defined agent with a set of tools, return structured output) rather than an application platform to build inside. That's what decided Cloudflare OS for us, not its quality.

Should an LLM do lead scoring? In our design, no. The agent extracts facts; a deterministic rubric with fixed weights decides what clears the bar. That's a choice rather than a rule, but it means a scoring change is reviewable, testable and version-controlled, and we can tell a salesperson exactly why a company scored what it did.

Which agent harness should you use? Whichever fits the shape of the work. Short, bounded, scheduled jobs suited DeepSeek Harness for us. A long-lived chat session with unpredictable concurrency didn't, because of what a subprocess per run cost us there, so that stayed on Mastra.