Production AI sales and outreach automation

A SaaS client's outreach ran by hand: search, open the page, check it, judge the fit, write the email, send it, chase it. Every step was slow and every step cost staff time.

I was the lead engineer on the build, working with one other developer. The client is under NDA, so this page describes the system at the architecture level only.

Automate the funnel end to end: find → verify → score → write → send → measure. Keep model costs low at volume. Keep a person in control of what actually goes out under the client's name.

• —Discovery agents on LangGraph. Neural search finds candidate pages, and a cheap model expands each brief into many search queries.
• —Layered verification. Candidates go through several crawling and scraping APIs, with fallbacks, so one flaky provider doesn't stall a run.
• —Six-factor LLM scoring. Each prospect gets a 1–100 fit score across topic, industry, quality, authority, audience and commercial intent, plus a confidence level. Prompt templates are per user.
• —Drafted outreach. A fast model writes a personalised email per prospect and hands it to an email sequencer.
• —Human in the loop. A review step sits between the agents and the send.
• —A management view with open, click and reply rates by campaign, and CSV export.

Cost at volume. Most of the work is query generation, filtering and first-pass scoring. I routed those to nano and flash-class models and kept larger models for search and analysis. Every API call is tracked and every LLM call is traced in Langfuse, so cost per step is visible, not guessed.

Ten-plus external APIs. Each has its own rate limits, failure modes and data shape. The pipeline treats them as replaceable stages behind one interface.

Data you can trust. Prospect records get refetched and re-validated as sources change. A backfill run updated all 752 affected records with zero failures, and a 50,000-row import load test runs against the import path.

Build the eval before the scorer. I'd label a few hundred prospects by hand, freeze that set, and score every prompt change against it. Without it, prompt tuning is taste.

Keep secrets out of agent instruction files from day one. Coding agents read those files constantly, and they end up in git.

• —Orchestration: LangGraph, LangChain, human-in-the-loop review step
• —Models: Gemini Flash (scoring, drafting), GPT-4o / 4o-mini (search, analysis), nano-class models for query expansion
• —Backend: Python, Flask and FastAPI entrypoints, Postgres with row-level security
• —Frontend: React, Vite, shadcn; Chart.js reporting
• —Ops: Docker, Fly.io, Langfuse tracing, API usage tracking