Skip to content
← All posts
AIStartupProduct

AI Integration for Startups — What's Actually Worth Building

Every pitch deck in 2026 has “AI-powered” somewhere on slide one. Investors expect it. Users are curious about it. But here’s what most founders won’t admit: half the AI features out there don’t actually need to exist.

Before you bolt AI onto your product, let’s figure out what’s worth building and what’s just noise. This post is the framework I use when a founder asks whether to add AI, what to buy versus build, and how to find out cheaply whether it works.

Start with the problem, not the model

The most reliable way to waste a quarter is to start from “we should use AI” and look for a place to put it. Start from a user problem and ask what the cheapest thing is that solves it.

AI is a good fit when the input is unstructured (free text, documents, images, audio), when the correct answer is fuzzy rather than exact, or when the volume makes handling it by hand impractical. It is a poor fit when the answer is a lookup, a calculation, or a rule. A deterministic system is cheaper, faster, testable, and never confidently wrong. If a rule handles ninety percent of cases, put the rule first and send only the remainder to a model.

What tends to be worth building

High return: support and internal automation

This is the easiest win. AI-assisted support (ticket routing, drafted replies, answers grounded in your own docs) has a clear, measurable return: fewer tickets handled by hand, faster responses. The same applies to internal work: summarising calls, extracting fields from documents, triaging inbound requests.

If you’re handling more than 50 support tickets a week, this should be on your roadmap. Tools like Intercom and Zendesk already ship AI layers, and for many teams that is the right answer. A custom build starts to make sense when your documentation, product, or workflow is specific enough that a generic layer keeps getting it wrong.

Good if you have data: recommendations and personalization

Recommendation engines work when there is enough behavior to learn from. Netflix, Spotify, and Amazon all run on this, and they all have enormous amounts of data.

If you’re a pre-launch startup with 200 users, a recommendation engine is premature. Focus on getting more users first. Once you have real usage data, personalization can meaningfully improve retention and engagement, and you will know which signals matter.

Expensive but sometimes necessary: custom models

Training your own model costs real compute, months of effort, and a team that knows what it is doing. It only makes sense if AI is the core of your product, not a feature of it. Medical diagnosis tools, fraud detection, and proprietary data analysis are the usual examples. Otherwise, use existing APIs and models.

Fast to build: LLM wrappers

Building on top of OpenAI, Anthropic, or other LLM providers is the fastest path to shipping an AI feature. You can go from idea to working prototype in days.

The catch is that a low barrier to entry means high competition. If your entire product is a thin wrapper around a hosted model, you are one provider update away from obsolescence. Use LLMs to test market fit quickly, then build defensible value on top: proprietary data, unique workflows, domain expertise.

Buy, wrap, or build

Most startups face the same three-way choice. The honest comparison:

OptionUse it whenThe catch
Buy a product with AI built inThe problem is common and your needs are standardYou inherit its limits, pricing, and roadmap
Wrap a hosted model in your own productYou need it specific to your users and want it live in weeksEasy to copy; quality depends on your prompts, data, and evaluation
Build a custom modelAI is the product, or a hard requirement rules out hosted modelsSlow, expensive, and needs data and expertise you may not have yet

The usual path is to buy or wrap first, learn what users actually do with it, and only move down the table when a specific, measured limitation forces you to.

Prove it before you build it

A prototype that impresses in a demo is not evidence that a feature works. Four things separate the two.

Write the evaluation set first. Collect a few dozen real inputs and write down the output you would consider correct for each. Run them automatically whenever the prompt or model changes. This is the single most useful habit in AI work: without it every change is an opinion, and every model upgrade is a gamble. It also gives you a concrete answer to “is it good enough to ship?”

Decide what a wrong answer costs. A wrong product suggestion is a shrug. A wrong summary in a legal or medical workflow is a liability. The cost of failure determines how much checking, human review, and fallback logic the feature needs, and often whether it should exist at all.

Measure cost per request from day one. Hosted models bill per use, and a feature that is cheap at a hundred requests can dominate your margin at a hundred thousand. Log tokens and cost per request now. Route simple requests to smaller models, reserve the larger ones for hard cases, cache repeated work, and cap output length.

Plan for the model changing underneath you. Providers update and retire models. Keep the model name in configuration, keep your evaluation set runnable, and avoid prompts that only work because of one version’s quirks.

Data: the decision you cannot undo later

Where your data goes is an architecture choice, and it is much cheaper to make before the first request than after. Know what the provider retains and whether it trains on your inputs, keep sensitive fields out of prompts where you can, and log what was sent so you can answer a customer’s question about it.

If you handle regulated data, such as health information, the bar is higher: you generally need an agreement with the vendor and a deliberate boundary around what leaves your environment. I wrote up how to think about that in sending PHI to LLM APIs.

The decision framework

Before adding AI to your product, ask four questions:

  1. Does this solve a real user problem? Not “would this be cool”, but does it solve a pain point your users actually have?
  2. Can you measure the impact? If you can’t define what success looks like, you can’t justify the investment, and you can’t tell when it breaks.
  3. What’s the simplest version? Start with an API call and a basic integration. Prove it works before building anything custom.
  4. What happens when it is wrong? If the answer is “something bad”, design the review step and the fallback before the feature.

Don’t build AI for the sake of AI

The best AI features are invisible. Users don’t care that it’s “AI-powered.” They care that it works, it’s fast, and it saves them time. If a simple rule-based system solves the problem just as well, use that instead.

Thinking about adding AI to your product? Request a technical triage: I’ll help you figure out what’s worth building and what’s not. Learn more about my AI & intelligent automation services, or see what the first ninety days look like on the fractional CTO for AI startups page.

Frequently asked questions

How do I decide whether my startup needs AI at all? +

Start from the user problem, not the technology. If a rule-based system or a better form solves it, use that: it is cheaper, deterministic, and easier to debug. AI earns its place where the input is unstructured (text, images, audio), the right answer is fuzzy, or the volume makes manual handling impractical. If you cannot describe what a good output looks like, you are not ready to build it.

Should a startup train its own AI model? +

Almost never at the start. Training or fine-tuning is justified when AI is the core of the product and a hosted model cannot meet a hard requirement such as accuracy on proprietary data, latency, cost at volume, or data that may not leave your environment. Prove the feature works with a hosted model first; that tells you exactly what a custom model would have to beat.

How do I keep AI feature costs under control? +

Measure cost per request from the first day, route simple requests to smaller models and reserve large models for the hard ones, cache repeated work, and cap output length. Cost problems are usually found at scale by accident; instrument it before you have the scale.

How do I know if an AI feature is working? +

Write an evaluation set before you ship: a few dozen real inputs with the outputs you consider correct, run automatically on every change to the prompt or model. Without it, every improvement is an opinion and every model upgrade is a gamble.

Can I send customer or patient data to an AI API? +

Only after you have checked what the provider does with it and what your obligations are. Regulated data such as health information needs an agreement with the vendor and a deliberate boundary around what is sent. That is an architecture decision to make before the first request, not after.

Deciding what is worth building is the whole job

Evals before models, routing before fine-tuning, and cost observability before scale. The AI startups page lays out what the first ninety days of that actually look like.

Fractional CTO for AI startups

New writing on regulated and research software

Occasional notes on building software where being wrong is expensive — HIPAA architecture, grant-funded builds, and the decisions that are hard to reverse. No sequence, no pitch. Reply to any of it and it reaches me directly.

Need a senior engineer?

First 30 minutes complimentary.

Book a call