Which AI features are actually worth building into your software

"Should we add an AI feature?" is a question that comes up in almost any software conversation now, and it's the wrong question — it treats AI as a feature category instead of a tool, which is how you end up with a chatbot bolted onto a settings page that nobody asked for. The right question is narrower: is there a specific, recurring task in this product that an LLM genuinely does better than the alternative? Most of the time, once you ask it that specifically, the answer is no. Sometimes it's a clear yes. Here's how I tell the difference.

Good fit: unstructured input, structured output

Reading a messy customer support email and extracting a category, priority, and summary is a task LLMs are genuinely good at — it's pattern recognition over language, which is exactly what they're built for. Same with summarizing a long thread, drafting a first-pass reply, or classifying free-text form submissions into your existing categories. These work because the model's job is narrow and its output feeds into a system that still has real validation around it.

Good fit: a first draft, not a final answer

LLM output is genuinely useful when a human reviews it before it matters — a first-pass product description from a spec sheet, a draft email a support agent edits before sending, suggested tags for a new blog post. The moment you remove the human review step and let the model's output go straight to a customer or straight into a database unchecked, you've changed the risk profile entirely, and that needs to be a deliberate decision, not an accident of "the demo worked."

Bad fit: anything that needs to be reliably correct

If a wrong answer causes real damage — miscalculating a price, giving legally binding advice, answering a support question incorrectly with confidence — an LLM by itself is the wrong tool, full stop. Hallucination isn't a bug you patch out; it's a property of how these models work. The fix isn't "prompt it better," it's "don't let the model be the last line of defense on anything that has to be right."

Bad fit: things that are already solved

Search that already works with a good index doesn't need to become "AI search." A contact form doesn't need a chatbot in front of it. If the honest answer to "why AI" is "because it's expected" rather than "because it measurably improves the outcome," that's a sign to skip it — clients pay for LLM API calls per token, and a feature that exists for the pitch deck is a recurring cost with no return.

What this looks like in a project

An AI feature worth shipping is usually small, specific, and sits behind a clear fallback if the API is slow or wrong — because a feature that occasionally fails silently is worse than not having it. The interesting engineering isn't the API call. It's deciding, precisely, where a probabilistic tool belongs in a system that mostly needs to be deterministic.

Wondering whether a specific idea of yours is a good fit for an LLM feature, or just AI-shaped noise? Tell me the use case — I'll give you a straight answer either way.