To build an AI workflow, you wire a large language model into a real process: take some input, call an LLM API, and turn the response into an action your app can actually use. That last part — turning free text into something structured and reliable — is where most first attempts fall apart. This guide walks through building your first AI workflow end to end: the API call, the prompt, structured output, and tool calls, plus the patterns and pitfalls that separate a weekend demo from something you can put in front of users.
What is an AI workflow, and when do you need one?
An AI workflow is a sequence of steps where at least one step is a call to an LLM, and the model's output feeds the next step. It is not a chatbot. A chatbot ends with text on a screen; a workflow keeps going — it validates the output, calls a database, sends an email, or triggers another model call.
You need one when the task involves messy, unstructured input that would be painful to handle with regular code. Good candidates include:
- Extraction — pulling fields (name, amount, date) out of an email, PDF, or support ticket.
- Classification and routing — deciding which team, tag, or queue an item belongs to.
- Summarization — condensing long documents into a fixed shape your UI can render.
- Drafting — generating a first pass (reply, description, changelog) that a human approves.
If the input is already structured and the rules are fixed, you probably do not need an LLM — a regex or a switch statement is cheaper and more reliable. Reach for a model when the variability is the whole point. For the business case behind this, see why dev teams are adopting AI workflows.
What you need before you build
You can build your first workflow with surprisingly little. Here is the minimum:
- An API key from a provider (OpenAI, Anthropic, Google, or an open-source model behind an endpoint). Keep it in an environment variable, never in client-side code.
- A server-side runtime — a Node/Next.js API route, a serverless function, or a small backend. LLM calls belong on the server so your key is never exposed to the browser.
- A clear task definition — one sentence describing the input and the exact output you want back.
- A way to validate output, such as Zod or JSON Schema, so a malformed response fails loudly instead of silently corrupting your data.
That is it. No vector database, no framework, no agent orchestration layer for version one. Add those only when a concrete problem demands them.
How to build an AI workflow step by step
The core of any AI workflow is a single API call. Most providers accept a list of messages with roles (system, user) and return the model's completion. Here is a minimal server-side call using fetch:
// app/api/classify/route.js (Next.js server route)
export async function POST(req) {
const { ticket } = await req.json();
const res = await fetch("https://api.provider.com/v1/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.LLM_API_KEY}`,
},
body: JSON.stringify({
model: "your-model-id",
messages: [
{ role: "system", content: "You classify support tickets. Respond with one word: billing, bug, or feature." },
{ role: "user", content: ticket },
],
temperature: 0,
}),
});
const data = await res.json();
const label = data.choices[0].message.content.trim();
return Response.json({ label });
}A few decisions in that small block matter more than they look. Set temperature: 0 for anything where you want the same input to produce the same output — classification, extraction, routing. Put the instructions in the system message and the variable data in the user message; mixing them makes the model harder to steer. And keep the system prompt specific about the shape of the answer, not just the task.
Get reliable structured output from an LLM
Reading a label out of message.content works until the model returns "This looks like a billing issue." Free text is the enemy of a workflow. You want structured output — JSON that conforms to a schema you control — so the next step can trust it.
Modern APIs support this directly through a JSON schema or structured-output mode. You describe the shape you expect, and the model is constrained to return valid JSON matching it:
body: JSON.stringify({
model: "your-model-id",
messages: [
{ role: "system", content: "Extract the fields from the support ticket." },
{ role: "user", content: ticket },
],
response_format: {
type: "json_schema",
json_schema: {
name: "ticket",
schema: {
type: "object",
properties: {
category: { type: "string", enum: ["billing", "bug", "feature"] },
urgency: { type: "string", enum: ["low", "medium", "high"] },
summary: { type: "string" },
},
required: ["category", "urgency", "summary"],
additionalProperties: false,
},
},
},
})Even with schema-constrained output, validate the parsed result on your side with something like Zod. Schema enforcement covers structure, but not business rules — an urgency of "high" on a feature request might still be nonsense you want to catch. The pattern is always the same: constrain the model, parse, then validate, and treat a validation failure as a retry-or-fail branch, not a crash.
Add tool calls so the LLM can take actions
Structured output lets the model return data. Tool calling (also called function calling) lets it decide to do something — look up an order, query stock, send a message. You describe the tools available; the model responds by asking you to run one with specific arguments; you execute it in your own code and feed the result back.
The loop looks like this:
- Send the user message plus a list of tool definitions (name, description, argument schema).
- The model replies either with a final answer or a tool call — a function name and JSON arguments.
- Your code runs the actual function (that is your code, not the model's — the model never touches your database directly).
- You append the tool's result to the message history and call the model again.
- Repeat until the model returns a final answer instead of another tool call.
Two rules keep this safe. First, the model only requests actions — you own the execution, so you validate every argument before running anything. Second, cap the number of loops. A model that keeps calling tools without converging should hit a limit and bail, not spin forever. Tool calling is the foundation of agents, and it builds on the same request-response mechanics as retrieval and RAG systems — the model reasons, you fetch, it reasons again.
Patterns and pitfalls for production AI workflows
Once the happy path works, these patterns keep it working under real traffic:
- Chain small, focused calls. One prompt that classifies, extracts, and drafts a reply is fragile. Split it into steps you can test and debug independently.
- Retry with backoff. LLM APIs return rate limits and transient errors. Wrap calls in a retry, and on a validation failure, retry once with the error fed back to the model before giving up.
- Cache aggressively. Identical inputs at temperature 0 produce identical outputs — cache them and skip the API cost. Many providers also offer prompt caching for long, repeated system prompts.
- Log the prompt and the raw response. When output is wrong, you cannot debug what you did not record. Store inputs, outputs, model version, and latency.
- Keep a human in the loop for any irreversible action until you trust the accuracy on real data.
The pitfalls are just as predictable. Do not trust output you have not validated. Do not paste secrets or customer PII into a prompt without checking your provider's data policy. Do not hard-code a model ID everywhere — pin it in one config value so upgrades are a one-line change. And watch your token counts: long inputs cost money and can silently truncate context. For picking the right tools around your workflow, the 2026 developer AI tools roundup is a useful starting point, and the broader AI workflows category collects more of these patterns.
Frequently Asked Questions
What is the difference between an AI workflow and an AI agent?
An AI workflow follows a path you define — the steps and their order are fixed, and the LLM fills in the hard parts. An AI agent decides its own path at runtime using tool calls, looping until it reaches a goal. Start with a workflow: it is predictable, testable, and covers most real use cases. Reach for an agent only when the task genuinely needs open-ended decision-making.
Do I need a vector database or RAG to build an AI workflow?
No. Retrieval-augmented generation matters when the model needs knowledge it was not trained on — your docs, your product data. For extraction, classification, or drafting from input you already have, a plain API call is enough. Add a vector database only when you hit a concrete need to search over private content, not by default.
How do I get an LLM to always return valid JSON?
Use your provider's structured-output or JSON-schema mode to constrain the response, set temperature to 0, and validate the parsed result with a library like Zod. If validation fails, retry once with the error message fed back to the model. Combining schema enforcement with your own validation catches both malformed structure and outputs that are valid JSON but wrong.
Which language should I use to build an AI workflow?
Any language with an HTTP client works, since LLM APIs are just REST endpoints. JavaScript and TypeScript are a natural fit for frontend developers because you can run the workflow in a Next.js server route right next to your app. Python is popular for data-heavy pipelines. Pick the language your team already ships in — the LLM call is the easy part.
Conclusion: start small, validate everything
You can build your first AI workflow in an afternoon: one server-side API call, a clear prompt, structured output, and validation on the way back. That foundation — constrain, parse, validate, act — scales from a single classification step all the way up to tool-calling agents. Resist the urge to add frameworks and vector databases before you have a working loop. Ship the smallest useful version, log everything, and let real inputs tell you what to build next.