Introduction
Technology OpsAQE by the ticketAutonomous quality engineering
Technology Ops
AQE
AUTONOMOUS QUALITY ENGINEERING

Test design, measured by the ticket.

AQE builds an execution-ready test pack for each ticket that reaches QA, from the ticket and the code that actually shipped. It runs every day, with nobody having to start it, and is proven in production trials on tickets.

About67
test-design jobs every working day, across all 16 projects
About5.5min
from ticket to finished test pack, on average
Under$1
AQE cost per ticket on average, under 5¢ per test case
Sources

Ticketing system, all 16 projects in the daily run, last six complete months: 8,792 QA and staging entries over 131 working days. Production trials on tickets, current pipeline on company infrastructure and the enterprise AI cloud: averages per ticket and per test case.

85+ code repositories, kept currentOne ticket at a time
THE WORKLOAD

About 67 test-design jobs. Every working day.

We measured six complete months across all 16 projects in the daily run. Each entry into QA or staging is one test-design job.

887
tickets reach QA in an average month
578
tickets reach staging in an average month
About1,465
test-design jobs in an average month
About67
test-design jobs every working day

SourceTicketing system, all 16 projects in the daily run, last six complete months (April to September 2026). Monthly QA entries 782, 853, 891, 939, 905, 954. Monthly staging entries 387, 617, 695, 652, 578, 539.

QA Staging One tile per entry
THE PROBLEM

Done by hand. Cost per ticket unknown.

Before the first test is written, someone reads the ticket, the discussion, the code changes and the system around them. Nobody can say what that costs per ticket.

  • Code reaches QA faster than test design can keep up.
  • Depth and format depend on who picks the ticket up, and what else needs retesting is often a judgement call.
  • Senior QA time goes into reading and writing, which takes it away from the judgement-heavy testing only people can do.
  • It happens about 67 times every working day, and none of it carries a recorded cost.
About67
test-design jobs every working day, none with a recorded cost per ticket

SourceTicketing system, all 16 projects in the daily run, last six complete months: 8,792 entries over 131 working days.

Next: the same flow with AQE
HOW IT THINKS

AQE reads the change before it writes a test.

The ticket states the intent. The code carries the risk. AQE thinks like a senior QA engineer and works through both in five steps, on every ticket, in the same order.

85+
code repositories across 10 code groups, kept current

SourceAQE code awareness, production pipeline, Oct 2026.

Output:One execution-ready test pack in a standard format: test case ID, title, objective, type, how it can be run, persona, dependencies, steps, expected result and test data. Three real ones follow on the next slide.

QUALITY · SAMPLE TEST CASES

Precise enough for an agent to run.

Three test cases AQE wrote during the production trials on tickets, one for each way a test can run, with product details replaced. Each says who runs it, the exact data to use, every step and a result anyone can check.

Persona

Who runs it, and the access they need.

Test data

The exact data to use, taken from the ticket or named up front.

Steps

Actions in order, including any wait for background work.

Expected result

An outcome a person or a machine can check, down to the status code or the placeholder character.

How it can run

Auto, Assisted or Manual: by an agent alone, by an agent once a person supplies one thing, or by a person. AQE names the gap.

A next targetWith test cases at this level of detail, an AI agent can pick up an Auto case and execute it on its own, and an Assisted one once a person supplies what AQE flagged. Autonomous test execution is one of AQE's next targets.

From real test packs · product details replaced
TC-14HighDirect impact

A non-quarterly period is rejected with a clear error

Verify that the new rolling-totals endpoint accepts quarterly periods only, and rejects full-year and half-year formats with HTTP 400 instead of returning a wrong or partial total.

Type
API
How it can run
Auto
runs alone
Persona
Analyst user
signs in as
Depends on
None

Test data

Any valid QA account. The periods 2025FY and 2026H1 to reject, and 2026Q3 as the control.

inputs

Steps

  1. Get an access token for the analyst user.
  2. Send GET /api/v1/rolling-totals?account=<id>&metric=revenue&period=2025FY.
  3. Record the HTTP status and the response body.
  4. Send the same request with period=2026H1 and record the result.
  5. Send the control request with period=2026Q3.
actions

Expected result

Both 2025FY and 2026H1 return HTTP 400 with a message that only quarterly periods are accepted. The 2026Q3 control returns HTTP 200, and no request returns a 500 error.

checks

Gap check by AQENone. The persona, the data and the checks are all known, so an agent can run it on its own.

Under 5¢the average AQE cost to design one test case like these, in the production trials on tickets

AT SCALE · WHAT TO EXPECT

Minutes and cents a ticket, at scale.

Every run records its time, test count and cost. Averaged across the production trials, those records show what running AQE on every ticket looks like, from one ticket to a month of them.

Lower cost

Recorded for every ticket: under $1 a ticket and under 5¢ a test case.

Faster turnaround

About five and a half minutes a ticket, and nobody has to start it.

One standard

One method and one format, whoever picks the ticket up.

Senior focus

QA starts from a prioritised pack. Design intent, not yet measured.

SourcesProduction trials on tickets, current pipeline on company infrastructure and the enterprise AI cloud: averages per ticket and per test case. Ticketing system, all 16 projects in the daily run, last six complete months: about 67 test-design jobs every working day, about 1,465 in an average month. Day and month figures are estimates at trial averages.

One cell = $1
About 1,465jobs in an average month
About 28,000test cases
Under $1,500AQE cost, estimate

Estimate at trial averagesAbout 1,465 test-design jobs in an average month, at under $1 each, puts AI-driven test design across all 16 projects at under $1,500 a month. No manual baseline appears here, because none has been measured.

Per ticket, on average · production trials
About5.5min
from ticket to finished test pack, on average
About19
test cases per ticket, on average, prioritised by impact
About$0.85
AQE cost per ticket, on average, recorded for every run
Under5¢
AQE cost per test case, on average
TRUST AND GOVERNANCE

Narrow access. Full audit trail. People in control.

AQE runs on our own infrastructure and our own enterprise AI cloud account. The AI sees only what it needs, every call can be traced, and people decide what reaches a ticket.

Least privilege

The AI can read only the prepared material and the code. It cannot run commands, has no internet access and holds no credentials.

Access per ticket

Issued fresh for each ticket and limited to calling the AI model.

Audit trail

Every AI call is traceable to the ticket it served.

People decide

Packs go to QA for review with a feedback request, and a go-live switch governs every write to tickets.

Fully measured

The cost, time and outcome of every run are recorded.

Audit ledger · every AI call
    WHERE IT FITS

    Same step. No new tool. Now autonomous.

    Test design has always sat between code reaching a test environment and testing beginning. AQE does not add a step: it makes that one autonomous, built entirely from systems already in our stack, so nothing new had to be bought, installed or learned, and results arrive on the ticket the team already works in.

    Ticketing system

    Where tickets are collected and where test packs are delivered.

    Code history

    What actually shipped, and the code used to map impact.

    Build system

    AQE runs on our existing build machines.

    Enterprise AI cloud

    The company's own account carries out the analysis.

    Analytics dashboards

    The cost, time and outcome of every run.

    5 of 5
    building blocks were already in our stack
    0
    new tools to buy, install or learn

    Easy to adopt elsewhereThese are standard building blocks in most engineering organisations. Any team with a ticketing system, source control, build machines and an enterprise AI cloud account can run the same approach on what it already has.

    Step 04, test design, over time
    1. By handPeople read every ticket and write every test.
    2. AI skillsSkills do the work; people still start and drive them.
    3. AutonomousAQE runs it every day, with nobody starting it.
    01Plan
    02Build
    03Deployed to a test environment
    04Test design
    AQEAutonomous test design
    05Testing begins
    06Release

    Nothing new for engineers to adopt. Standard building blocks most teams already run, and results arrive on the ticket.

    THE RESULT · TICKETS DELIVERED

    Less time designing. More tickets delivered.

    With AQE writing the test design, QA's time goes into testing. We measured what that did to delivery on the two trial projects that contributed the most tickets: tickets delivered a month before AQE, against every full month since.

    +54%
    more tickets delivered a month, the two projects together
    About176
    tickets delivered a month with AQE, up from about 115
    Project A

    From 47 to 82 tickets delivered a month, and 91% more than in the same months a year earlier.

    Project B

    From 67 to 94 tickets delivered a month, and 29% more than in the same months a year earlier.

    The other trial projectsAQE ran on five more trial projects. A reorganisation means one of them did not exist before the trial and two were discontinued after July; the other two had only short trial runs. None has a fair before-and-after, so they are left out.

    Sources

    Ticketing system, data to 26 Oct 2026, for the two trial projects that contributed the most tickets. "Through QA" means the ticket was in Ready for QA, In QA or In Staging at some point; Declined and similar closures don't count either. Before: the six full months before the month AQE started on the project. With AQE: the full months after that, to September 2026; October is partial and left out of the averages. Same months a year earlier: the with-AQE months of 2025. Together: the two projects' own averages added up. Averages show what changed once AQE arrived; release timing and the mix of work also play a part.

    Tickets delivered each month · Nov 2025 to Oct 2026
    Project AAverage 47 → 82 a month
    Project BAverage 67 → 94 a month

    DeliveredA story, bug or task that went through QA and was then closed as done, counted in the month it closed. Tickets closed as Won't Do or Duplicate, epics and sub-tasks don't count.

    WHAT COMES NEXT

    Live by a switch. Wider by configuration.

    Proven in production trials on 7 projects, then switched on for all 16. The design keeps every moving part flexible: posting to tickets, the projects covered, how many tickets run at once and when the daily run starts are all settings, switched on or off without a code change.

    01

    Now

    Autonomous test design, live every day on all 16 projects. Going from the 7 trial projects to all 16 was a configuration change.

    03

    Later

    A closed loop, in which execution results and reviewer feedback improve future test design.

    About4 in5
    tagged trial test cases can run by machine, fully or with help (978 fully, 3,104 with help, of 5,082)
    16 of 16
    projects in the daily run today, up from 7 in the trials, by a configuration change
    16 projects, one tile each

    Today: all 16 projects, full coverage. Going from 7 to 16 was a configuration change.

    From 7 projects to 16 was one setting. So is every next step.

    SourcesProduction trials on tickets, 7 projects: auto 978, assisted 3,104, manual 1,000. Coverage: the projects in the daily run are a configuration setting, which took AQE from 7 in the trials to all 16 today.

    THE DAILY RUN

    How each ticket becomes a test pack.

    Every day, unattended, on our own build machines, with every run on record.

    1. Ticketing systemEach day: every ticket that reached QA or staging in the last 24 hours.
    2. Find what shippedThe exact code changes deployed to each ticket's test environment, from source control and deployment history.
    3. Code library85+ repositories across 10 code groups, kept current.
    4. Gather contextThe ticket, its discussion, linked work and the deployed changes, in one bundle.
    5. AI analysisFive senior-QA steps in our own enterprise AI cloud, with access issued per ticket.
    6. Test packAbout 19 test cases on average, prioritised critical, high, medium or low.
    7. On the ticketFor QA review, with a context note and a feedback request; a go-live switch governs every write.
    8. Run recordStatus, test count, time and cost for every ticket.
    9. Analytics dashboardsThe cost, time and outcome of every run, over time.

    SourcesFigures in the flow are averages across production trials on tickets: about 5.5 min, about 19 test cases and about $0.85 a ticket, under 5¢ a test case. The flow itself is illustrative. Ticketing system, all 16 projects in the daily run, last six complete months: about 67 test-design jobs every working day.

    Move