Keith Norbie

Since July, our team has had more than 100 calls about AI implementation with CEOs, CFOs, CTOs and PE operating partners. These 12 questions came up most often: 1. “Where do we start?” Where your customer would feel it first. Pick one workflow, run it for a month, then decide what comes next. Start with a long list of ideas and you’ll end up shipping none of them. 2. “Can’t we just build this ourselves with Claude?” For a lot of it, yes. Reports, research, content: get your people using agents every day. You don’t need a firm for that. You need one when the work touches a system of record, patient data or payment data, or when it has to run every day without somebody babysitting it. 3. “How do we measure ROI?” Name the number before you build: days to close, denial rate, hours per onboarding. If nobody can name it, you’re buying a demo. 4. “How do we know the AI is right?” You test it. Build a golden set of answers your business agrees are correct, then score every release against it. Anything below the bar goes to a person. 5. “Who owns it, and who keeps it running?” You do. Your cloud, your repo, your accounts. Before kickoff, name the person on your side who will run it. Make your first AI hire somebody who can operate it and keep it healthy. 6. “Is our data safe?” It should stay in your environment. Ask where it goes, who at the vendor can see it and what the model provider keeps. Expect a plain answer to all three. 7. “Do we have to fix our data first?” Only the data your first workflow touches. Then build in order: a simple pipeline that runs every day, a layer that defines what your numbers mean, and then the agent. One prompt can’t do all three. 8. “What happens to our people?” Start with them. You can’t expect agents to work on top of a team that doesn’t use them. 9. “Which model should we use?” It matters less than you think, and you’ll change it. Pick a setup that lets you switch. Your workflows, evals and data are what you keep. 10. “What will it cost to run?” Get it in writing before the build: tokens, hosting and licences, with costs broken down per month and per transaction. 11. “Do we pay for time or for outcomes?” Either works as long as you can walk away. Work month-to-month, with no lock-in and a checkpoint after the first month. 12. “How do we tell a real AI partner from one that started last year?” Before you sign, ask to see something they built that’s running in production for another customer today. Then call that customer. After you sign, give them one team for a month. If nothing changed, throw them out. The question I’d add to the list: what happens if it works? Who runs it, who improves it, and what does your team do with the time it gives back? Ask it before you start.