Course 02 · Lesson 03 · 15 min

Running UAT: cycles, defects, severity and sign-off

Good scripts are half of UAT. The other half is running it with discipline: the right testers, protected time, a single defect log and an honest sign-off.

Who tests

UAT is done by the people who'll do the work, not by the consultant and not by the IT team. The AP accountant tests vendor bills. The store keeper tests receipts, in the warehouse, on the scanner they'll actually use. The consultant's job during UAT is to watch, explain and fix, not to click.

That only works if testers' time is protected. Three hours a day for two to four weeks is typical for key users on a mid-sized project. If their normal workload hasn't been reduced to make room, UAT becomes something they squeeze in after 6 PM, and it gets skipped. Get this agreed in writing with their managers before UAT starts.

Prepare the environment

  • A separate UAT database, a copy of the configuration, never the production database.
  • Migrated data from a trial load, not demo data. Testing with "Test Customer 1" hides problems that real customer data will expose: duplicate names, missing BINs, odd units of measure.
  • Real devices: the label printer, the barcode scanner, the 6.3 printer in the accounts office.
  • User accounts with the real access rights. If testers use admin accounts, access-rights problems stay hidden until go-live day.

Cycles

Plan for at least two cycles, three on larger projects:

CyclePurposeTypical length
Cycle 1Run every case once. Expect failures, which is the point.1–2 weeks
Fix windowPartner fixes configuration and defects; you re-run trial load #2.3–5 days
Cycle 2Re-run every failed case plus all Must cases, and run the end-to-end chains on the new data load.1 week
Cycle 3 (if needed)Only for remaining failures and regression checks.2–3 days

Why re-run Must cases that already passed? Because fixing one thing often breaks another. A change to tax configuration for exports can quietly change domestic invoices.

One defect log, four severities

Every failure goes into one shared log: not WhatsApp, not email threads, not the consultant's notebook. Each defect has an ID, the test case it came from, steps to reproduce, a severity, an owner and a status. The UAT template includes this log with a summary that counts open defects automatically.

SeverityMeaningExample
S1 — CriticalA core process can't be done, and there's no workaroundVendor bills can't be posted; Mushak 6.3 can't be generated
S2 — HighWrong result, or only a painful workaroundTDS not calculated for one vendor category
S3 — MediumMinor issue with an easy workaroundReport missing a column; print alignment off
S4 — LowCosmetic or a request for something newChange a label; reorder columns

Watch for S4s in disguise. "The system doesn't do it the way Tally did" is usually a change request, not a defect. Log it separately, because change requests go through change control (and possibly cost), not through the defect fix list.

Daily triage

During UAT, hold a 20-minute stand-up every morning with the client PM, the partner PM and the key users. Go through the new defects: confirm the severity, assign an owner, and agree when the fix will be ready to retest. Resist the urge to debate solutions in this meeting. Its only job is to keep the log accurate and moving.

Common pitfall

Severity drift. Under deadline pressure, S2s get relabelled as S3s so the exit criteria look met. Agree the severity definitions before UAT starts, and let the process owner, not the vendor, have the final word on severity.

What sign-off means

A process owner's UAT sign-off means: "I tested my process with my team, with real data, and I'm willing to run the business on it." It doesn't mean "the consultant told me it's fine". Sign-off is per process area (finance, supply chain, production, HR), each by its owner, and the exit criteria are:

  • All Must cases have passed in the latest cycle.
  • There are no open S1 or S2 defects.
  • Open S3/S4 defects are listed with agreed workarounds and fix dates.
  • The end-to-end chains have reconciled in stock, cost and tax.

Exercise (15 minutes). List your UAT testers by name, with the hours per day each will spend and who covers their normal work. If you can't name who covers their work, that's your first risk to escalate.

Key takeaways

  • The people who'll do the work test it, in their real environment, with protected time.
  • Use trial-load data and real access rights, never demo data and admin accounts.
  • Plan two or three cycles; re-run Must cases every cycle.
  • Keep one defect log with agreed S1–S4 definitions, and triage it daily.
  • Sign-off is per process owner, against written exit criteria.