How to Hire an AI Developer Safely

A registration team checking every application by hand has a choice: keep reviewing every record, approve people on a simple name match, or hire an AI developer to automate the checks the team already trusts. When I think about how to hire an AI developer, I start with that choice, not a model demo. Automating a checklist is different from asking software to make up its own rules.

We faced that decision on a platform where coordinators reviewed every scientist's registration. We asked the coordinator to list the checks they already ran, automated those checks against ORCID records, and sent only failed registrations to a person, tagged with the reason. We chose that over keeping every review manual or approving anyone whose name matched. The difference mattered because Ukrainian and English transliterations can make one person’s name look different on paper. You can see the scientist registration system we built.

In the case brief, a backtest found that 541 of 599 approved profiles passed without a person, while a name check across about 366,000 pairs of different people wrongly matched three pairs, all real namesakes (project details). Those are results for that workflow, not a prediction for yours. The point I’d carry into a hiring decision is narrower: ask for your existing checklist, then test whether the developer can reproduce it and route exceptions without hiding how a decision was made.

Start with the work, not the model

An AI developer should first understand what staff do now, what information they use, and what happens when a case is unclear. A model demo proves that a system can produce an answer in a prepared example. It doesn’t prove that the answer follows your process on incomplete, contradictory, or unusual records.

I’d ask a reviewer to walk through a recent example and explain each check in order. Which fields do they compare? What counts as a mismatch? Which cases do they send to a manager? If nobody can answer those questions consistently, automating the decision is premature.

Our registration workflow needed more than a plain name comparison. A simple match wrongly flagged real scientists whose names were transliterated differently between Ukrainian and English. We had to account for transliteration, while keeping the other checks anchored to what the coordinator already trusted (project details). That’s a useful hiring test: ask a candidate to show how they’d map your staff’s checklist, including the awkward cases, before they propose a black-box approval system.

Compare the hiring routes

A ready-made tool, an AI freelancer, an agency, and an in-house hire solve different problems. The right comparison is not just the initial price: ask who maps the workflow, handles integrations, tests failures, and owns the system when the first developer is no longer involved.

Route	What it can suit	What to check before committing
Off-the-shelf tool	A common task that fits the product’s built-in process	Whether it supports your actual checks, records exceptions, and gives you control over data
AI freelancer	A tightly scoped build with a clear owner for requirements and ongoing support	Who handles testing, integration, handover, and problems after delivery
AI development agency	A project that needs several skills or a defined delivery process	Who does the work, how you see progress, and who is responsible for the finished system
In-house AI engineer	Ongoing development where your business can support a permanent technical role	Whether you have enough work and technical oversight to support the role

A pilot can help you compare approaches, but it should be a paid, bounded task with a result you can inspect. I’d pick one workflow, provide representative examples, and ask for a short written plan: inputs, checks, exceptions, tests, and operating costs. A prototype that only handles the happy path is not proof of production readiness.

NIST’s July 2024 Generative AI Profile says risk management should reflect an organization’s goals, requirements, risk tolerance, and resources (NIST AI 600-1). For hiring, that means a developer should connect the test plan to the consequences of your workflow, not just show that a model can answer a prompt.

Check the test plan before the portfolio

A portfolio can show that a developer has built something. It doesn’t tell you whether their system behaves safely on your data or whether anyone checks the results. Ask for a production example they’re allowed to discuss, then focus on the parts that matter to your work: what the system does, how it handles an uncertain result, and how the team tested it.

I’d ask for an evaluation set, meaning a group of examples used to check system performance, with acceptance criteria you can understand. For application screening, that might mean comparing the system’s decisions with your documented review checklist and recording which cases it sends to a person. A candidate should also explain how tests will cover missing fields, conflicting records, misleading inputs, and requests outside the system’s job.

NIST’s July 2024 guidance recommends evaluations proportionate to risk, fact-checking generated information, and regular testing for vulnerabilities and misuse (NIST AI 600-1). It also recommends defining human roles and responsibilities in human-AI workflows. So ask who approves an output before it affects a customer, applicant, client, or official record.

That human boundary isn’t a formality. In a June 22, 2023 order, a federal court described lawyers submitting fabricated opinions and citations generated by ChatGPT; the order emphasized the attorneys’ responsibility to check their filings (Mata v. Avianca order). I’d want the contract and test plan to name the person who verifies consequential outputs against primary records.

Make the cost legible

A developer’s labor is only one part of the cost. Model usage, data retrieval, tools, hosting, monitoring, and future changes can sit on separate bills, so ask the developer to show which costs are included and which are not.

OpenAI’s API pricing page, checked October 25, 2026, lists gpt-6.1-sol at $2 per million input tokens and $10 per million output tokens, while gpt-6-luna is listed at $0.10 and $0.50 per million respectively (OpenAI API pricing). Those are model usage charges, not the total cost of building or running an AI system. The same page lists web search at $10 per 1,000 calls, plus model-rate charges for search content tokens, and file-search storage at $0.10 per GB per day, with 1 GB free (OpenAI API pricing).

I’d ask for a cost estimate using your expected workflow: model name, approximate input and output volume, tool calls, storage, and any regional-processing requirement. The October 25, 2026 pricing page lists a 10% uplift for eligible regional-processing endpoints for models released on or after March 5, 2026 (OpenAI API pricing). Ask for cost per representative task and what assumptions would change it. “AI costs” without those details is not a useful estimate.

For smaller, clearly defined jobs, fixed-price packages can be easier to evaluate than an open-ended project. WebDaddy lists AI agents from $500 for a single routine task and document assistants from $800 for a web chat on up to 200 documents; its services page describes those scopes and delivery times. That’s one option, not a price standard for the market. A different workflow should be scoped against its actual integrations, checks, and risks.

Put ownership, privacy, and handover in writing

Before sharing business records, ask where data goes, who can access it, and whether the proposed model runs through a cloud API or locally. If secrecy is a requirement, the developer should explain the data path and the trade-offs rather than treating it as a checkbox.

For accounting work, the IRS’s Publication 4557, revision May 2024, says professional tax return preparers must create and enact security plans under the FTC Safeguards Rule (IRS Publication 4557). The publication recommends limiting access to people who need taxpayer data, using multi-factor authentication, encrypting sensitive files, keeping secure backups, and maintaining audit logs that record who did what and when (IRS Publication 4557). Your lawyer or compliance officer should confirm which requirements apply to your business.

I’d also put the practical exit terms in the agreement: ownership of code, accounts, and data; who pays model-provider fees; how access is removed; where backups sit; who receives incident notices; and what handover includes. Those details matter just as much when a system works as when a vendor relationship ends. The IRS guidance also addresses system failures, data-loss reporting, and recovery, so ask the developer to explain the incident and handover process in plain language (IRS Publication 4557).

A hiring process you can run

A useful process starts with a reviewer and a defined workflow, not a long technical specification. I’d use these steps to compare candidates on the same evidence.

1. Choose one workflow. Pick a repeatable task where you can identify the inputs, the checks, and the cost of a mistake.
2. Ask the current reviewer to write the checklist. Record what they check, what counts as a failure, and which cases need judgment.
3. Request a workflow map. Ask each candidate to show how the system will apply the checks, preserve the reason for an exception, and involve a human.
4. Set the test before development. Agree on representative examples, acceptance criteria, failure cases, and who approves consequential results.
5. Get a separate operating-cost estimate. Require the proposed model, expected usage, tool charges, hosting, and any regional-processing premium.
6. Agree on privacy and handover. Put data access, code and account ownership, incident notification, backups, and access removal in writing.
7. Run the pilot with human review. Compare results against the current process before deciding which cases, if any, can proceed without a person.

The steps follow the same principle as the registration workflow: document the checks, automate those checks, and send exceptions to the person who understands them. NIST’s July 2024 profile supports defining responsibilities and evaluating systems in proportion to their risks (NIST AI 600-1).

Warning signs, and when not to automate

A developer who jumps straight to a model choice before understanding the workflow hasn’t answered the first question. I’d also be cautious if a demo is the only evidence offered, if nobody can explain how errors are found, or if a quote combines development and API costs into one unexplained total. Ask for a working test and a clear owner for each step instead.

There’s a real case where my advice to automate only exceptions does not apply. If reviewers cannot explain what they check, or the rules change every week, keep a person on every case until the checks can be written down. Otherwise, you’re asking software to reproduce a process that hasn’t been defined.

The practical test is simple: ask the person who reviews the work by hand today to list their checks. If they can explain the checks and agree on what counts as an exception, ask the developer to automate those checks and route only the exceptions to them. If they can’t, keep human review in place until they can.