Somewhere in your business, someone may have built an AI tool that nobody else knows about. Perhaps it drafts customer emails, updates records or chases overdue invoices. Perhaps it has been running for months, with no one checking what it actually does.
That’s not unusual. In Pax8’s October 2026 SMB AI Pulse Report, which surveyed 1,150 small business leaders across the United States, United Kingdom, Germany and Sweden, 35% of AI-engaged businesses had discovered AI running without official approval in the past six months. And among US businesses that had built their own AI, only 36% had a formal, documented review in place.
Most small businesses didn’t set out to have an AI governance problem. They wanted Monday morning’s admin to take less time. This post covers what AI governance looks like in practice, what AI can’t do on its own, and five checks that cover most of the gap. The Pax8 survey didn’t include Australia, but a global poll and Australia’s own AI guidance point the same way.
What “AI governance” actually means for a small business
AI governance is the habit of checking what an AI tool is allowed to touch, who approved it, and what happens if it gets something wrong, before it’s relied on rather than after. For a small business, this doesn’t need a committee or a 40-page policy. It’s closer to onboarding a new staff member: a short trial period, clear boundaries, and someone who checks the work before it goes anywhere near a customer or a client file.
The numbers behind the gap
The Pax8 report found small businesses moving quickly, with checks struggling to keep up:
67% of AI-engaged businesses have built one or more AI solutions that are now in regular use, and another 19% are prototyping. Only 13% rely exclusively on off-the-shelf tools.
52% of the businesses building AI are developing automated workflows that connect AI to other business systems.
In the US, 36% of businesses that have built AI have a formal, documented review, and another 41% review consistently but don’t write it down. Across the four countries, formal review ranges from 23% in the UK to 45% in Sweden.
35% of AI-engaged businesses discovered AI running without official approval in the past six months.
Data security and privacy exposure was the top-cited risk of employee-built AI in every market, at 30% to 35%.
65% said they need to move on AI immediately or within six months.
“AI solutions” can sound abstract, so here is what they often look like in a small business. These are illustrative examples, not survey findings: an assistant that drafts replies to customer emails; a tool that summarises a meeting and adds the follow-up tasks to your project list; an automation that updates your CRM or accounting software after a call; an agent that books appointments; or one that sends reminders for overdue invoices. Each one is useful. Each one also touches real customer or financial information, and some can send messages or change records without anyone looking first.
This isn’t limited to small business. ISACA’s 2026 AI Pulse Poll, a global survey of more than 3,400 digital trust professionals in roles such as IT audit, governance, cybersecurity and privacy, found the same shape at a different scale. In the poll, published in May 2026, 90% believed employees in their organisation were already using AI, yet only 38% had a comprehensive AI policy in place, and 25% had no active policy at all. Only 22% said the return on their AI investment had met or exceeded expectations. In both surveys, adoption is running ahead of oversight.
ISACA, May 2026: 90% say staff already use AI, but only 38% have a full AI policy and 25% have none.
What AI can’t do (and what to tell it never to do)
In July, an AI model that had been asked to generate and carry out example tasks on randomly chosen web pages filled in and submitted a tip form on a Philadelphia police website about unsolved murders. The tip was marked as spam and never reached investigators, and police didn’t know about it until Anthropic, the company behind the model, told them, as AP reported. Anthropic described a pattern it calls “persistence”, where a model works around a restriction instead of stopping.
The point isn’t that AI is out to deceive anyone. It’s that a form looked like a form, and nothing told the model what it must not do or checked its work. We unpack that case in What to Do Before Implementing AI: Human Checks and Clear Limits.
On its own, AI can’t:
Tell a rehearsal from the real thing, unless it has been told which is which.
Be accountable. A person always is.
Reliably know when it’s wrong. It can sound certain and still be mistaken, so important work needs a check.
Know your obligations to customers, clients and regulators, unless they’re written down for it.
Be counted on to stop and ask when it hits a block. Some models work around restrictions rather than stopping.
So write down what it must not do. A good starting list:
Send, submit, publish or pay anything without a person approving it.
Make anything up. If it doesn’t know, it says so.
Hide a failed step, or change a record to cover an error.
Touch data or systems the task doesn’t need.
Carry on when it can’t complete the task as asked. It stops and says why.
Then check the tool has actually been given those instructions. A limit nobody passed on protects no one.
Beyond sign-off: other security and governance considerations
Ask whether a tool stores, trains on or shares your data before you adopt it.
Signing off the tools your team builds is one piece. Five others are worth covering:
What goes in. Be careful with customer, staff and financial information. Australia’s privacy regulator, the OAIC, recommends that organisations avoid entering personal information, especially sensitive information, into publicly available generative AI tools.
What it can reach. Give each tool only the access it needs. A tool that reads your inbox doesn’t need to be able to delete it. Remove access when someone leaves or a tool is retired.
What comes out. AI output can be wrong, and the OAIC flags an elevated accuracy risk. Someone should check anything that reaches a customer, a client file or a financial record.
Who supplies it. Check the provider’s terms: where your data is stored, who can see it, and whether it’s used to train the model. AI features switched on inside software you already use count too.
Who’s accountable, and what happens when it goes wrong. Name one person, keep a simple record of what the tool did, and decide in advance who you’d call and what you’d do if it sent the wrong thing or exposed data. Australia’s Guidance for AI Adoption starts from the same place, with its first practice being to decide who is accountable and its last to maintain human control.
Depending on your industry and size, privacy and other legal obligations may apply, so it’s worth checking with your adviser.
Five checks that cover most of the gap
A small business doesn’t need a large budget to close this gap. Five checks, done consistently, cover most of what the research above is flagging:
Know what’s running. A short, dated list (what tool, who built or approved it, what it touches) is enough to take you from guessing to knowing.
Name an approver. One person (owner, ops lead, or a trusted advisor) who signs off before a new AI tool touches customer data, financial information, or anything client-facing.
Tell it what it must not do. Before any AI tool or agent goes live, write down what it cannot or should not do (send, submit, publish or pay without approval, make things up, hide a failed step), check the tool has actually been given those instructions, and test that it follows them.
Ask where the data goes. Before adopting any tool: does it store, train on, or share our data, and what can it reach? This speaks to data security and privacy, the top-cited risk in every market Pax8 surveyed.
Review on a schedule, not just at launch. A quarterly ten-minute check of what’s actually running is what catches the shadow AI Pax8 found in 35% of AI-engaged businesses.
The takeaway
None of this requires slowing down. The businesses getting real value from AI aren’t the ones moving fastest; they’re the ones who actually know what they’ve got running, and why. If you’re not sure what’s currently running inside your own business, that’s exactly the kind of gap an Ops HealthCheck Audit is built to find.
