Five Questions to Ask Before an AI System Goes Live

Your team wants to put an AI system into production. The demo worked, the business case looks good, and everyone's excited. Before you sign off, these five questions will tell you more about the risk than any slide deck will.

You don't need to be technical to ask them. You do need to hear clear answers.

1. What data does it learn from, and who can change that data?

AI models learn from data, and many keep adapting as new data comes in. That's what makes them useful; it's also what makes them different from traditional software. If someone can tamper with the training or reference data, they can shape how the model behaves. That's called data poisoning. In healthcare it could mean a bad call about a patient; in finance or logistics it could quietly skew decisions that look perfectly routine.

Ask where the data comes from, who can write to it, and how its quality is checked.

2. Who can reach the model, and what can they get out of it?

A model is an asset. Attackers try to copy it (model theft), probe it, or feed it inputs designed to make it fail (adversarial attacks). If the system uses a large language model (LLM), add prompt injection to that list: instructions hidden in a document or web page that the model follows as if they came from you.

Ask who has access to the model, its outputs, and anything it can act on. If the answer is "the team that built it," keep asking.

3. How would we know if it started behaving badly?

Traditional systems usually fail loudly. AI systems can fail quietly; the outputs still look plausible, they're just wrong. Monitoring has to watch for drift, odd outputs, and unusual usage, not only uptime.

Ask who is watching, what they're watching for, and how fast they'd notice a problem before a customer does.

4. Was security involved in the design, or only in the review?

Fixing a security gap after launch costs more than designing around it. It's the same lesson we learned with software decades ago, and it applies here too. The OWASP AI Exchange makes this point throughout: bring security into the AI development lifecycle from the start, not as a final gate.

Ask when security first saw this project, and what changed because of it.

5. Who owns the risk once it's live?

This is the one that usually gets the longest pause. Somebody has to "own the risk" after go-live; a named person, not a committee. That owner decides what gets monitored, when the model gets retrained, and when it gets switched off.

Ask for a name. If there isn't one, you've found your first action item.

The short version

If your team can answer all five clearly, you're in good shape. If two or three get a shrug, that's not a reason to kill the project; it's a reason to slow down for a couple of weeks and get the answers. Have the plan in place, then work the plan.

If you'd like a second set of eyes before your next AI approval, I offer a complimentary 30-minute discovery call for qualified organizations. Book a time here.

Dan Sorensen is a cybersecurity leader, Forbes Technology Council contributor, OWASP AI Exchange core team member, and fractional CISO.