HookThe ambulance system that failed on its first morning
At 07:00 on 26 October 1992 the London Ambulance Service switched on its new Computer Aided Dispatch system across the whole of London at once. Within hours it was in chaos: calls vanished from screens, the same incident generated two or three ambulances while others got none, crews arrived at addresses that were already covered, and the call queue grew faster than anyone could clear it. The service reverted to paper within days. The official inquiry that followed did not blame a single line of code — it blamed the method. Requirements had never been agreed with the ambulance crews who would use it, the software was loaded onto hardware that had never been tested under real London call volumes, and there was no fallback for when it went wrong. A better program would not have saved it; a better process would have.
That is the entire subject of this section. Systematic problem solving is the discipline of moving through five stages in order — analysis, design, implementation, testing and evaluation — so that each stage hands the next one something solid to build on. The stages are not bureaucracy; they are the accumulated scar tissue of projects like the London Ambulance Service, the Mars Climate Orbiter and TSB's bank migration, every one of which failed by skipping or rushing exactly one of these steps. The exam rewards you for naming the right stage, knowing what it produces, and — above all — for treating testing and evaluation as things you design in advance, not things you do at the end if there is time.
ModelAnalysis: understand the problem before you touch the code
Analysis is the stage where you work out what the system actually has to do, before deciding how. It starts by investigating the current situation and the people who will use the system, then captures their needs as requirements and asks whether the project is even worth attempting through a feasibility study — is it possible technically, affordable economically, legal, and deliverable in the time available? The deliverables of analysis are a documented set of requirements, the scope of what is in and out, and a set of success criteria that are specific and measurable, because those exact criteria are what evaluation will later be judged against. Analysts model the problem by decomposing it into its inputs, processes and outputs, often drawing data flow diagrams so nothing is missed. Get this stage wrong and every later stage inherits the mistake: the most expensive defects in software are requirements that were never captured, not typos in the code.
Requirements split into two kinds and the exam expects both. A functional requirement says what the system must do — the system shall dispatch the nearest available ambulance to a logged incident. A non-functional requirement constrains how well it must do it — the system shall handle 300 simultaneous calls with a response under two seconds. The London Ambulance failure lived almost entirely in the second kind: the functional behaviour was roughly sketched, but the load the system had to survive was never pinned down and never tested against, so it buckled under real demand. Writing the system should be fast earns nothing at analysis; writing retrieve any patient record in under one second with 50 concurrent users is measurable, testable at evaluation, and therefore where the marks are.
ModelDesign: turning requirements into a blueprint
Design converts the agreed requirements into a plan detailed enough to build from without further guesswork. It is where decomposition earns its keep: the problem is broken into modules or subroutines, each with a clearly defined interface — what it takes in, what it gives back — so that they can be written and tested separately and reassembled later. Design also fixes the algorithms (as pseudocode or flowcharts), the data structures that will hold the data, the file or database organisation, and the human-computer interface the user will see. A recurring exam theme is that these choices have consequences you are expected to justify: picking a data structure is really picking a performance profile, and the design document should say why. Good design makes implementation almost mechanical; weak design pushes those unmade decisions downstream, where they are far more expensive to fix.
Suppose analysis fixed the success criterion look up any of 20,000 members by ID in under a tenth of a second. Design is where you honour it. Decompose the system into modules — addMember, findMember, removeMember, renewMembership — each with a defined interface, then choose the data structure by its cost rather than by habit. Holding members in an unsorted array forces findMember to scan on average 10,000 records: linear search, O(n). Holding them in a hash table keyed on the member ID makes the same lookup O(1) — effectively constant however large the membership grows. The design records that decision and its justification, so the header FUNCTION findMember(id) RETURNS record plus the chosen structure and its complexity is exactly the level of detail an AQA design answer is marked on.
MechanismImplementation: writing code the design already decided
Implementation is translating the design into working code in a suitable high-level language, and its quality is judged less by cleverness than by discipline. Meaningful identifiers, consistent indentation, comments that explain intent, small single-purpose subroutines and version control all exist so that a human — often the person maintaining it years later — can read and change the code safely. Implementation is normally iterative: you build a module, test it, then build the next, rather than writing the whole system and hoping. The stage also covers the discipline of getting code into a live environment safely. On 1 August 2012 the trading firm Knight Capital deployed new software to eight servers but missed one, and on that server an old, dormant piece of code was accidentally reactivated by a reused control flag. In 45 minutes it fired millions of unintended orders and lost roughly $440 million — a pure implementation-and-deployment failure, not a design one. The lesson the exam wants is that clean, well-managed, well-documented code is a requirement in its own right, because untested reuse and sloppy deployment are how correct-looking systems still detonate.
MechanismTesting: proving it works, and finding where it doesn't
Testing is designed, not improvised. You choose test data in advance and decide the expected result before running anything, then compare it against the actual result. The three classes of test data are the backbone of the topic: normal (typical valid data the system should accept), boundary (values right on the edge of what is allowed, where off-by-one errors hide), and erroneous (invalid data the system must reject gracefully). Testing also comes in layers — unit testing of individual modules, integration testing of how they behave when joined, and system testing of the whole — plus alpha testing in-house and beta testing with real users before release. Black-box testing checks behaviour against the specification without looking at the code; white-box testing uses knowledge of the code to exercise every path. Two real disasters were integration failures: NASA's Mars Climate Orbiter was lost on 23 September 1999 because one team's software worked in pound-force and another's in newtons and the interface between them was never tested together, and Ariane 5 Flight 501 exploded 37 seconds after launch on 4 June 1996 when reused Ariane 4 code overflowed converting a 64-bit float into a 16-bit integer — code that was never re-tested in the faster rocket's flight envelope.
Take a subroutine validateMark(m) that should accept only whole-number exam percentages from 0 to 100. Good testing does not throw random numbers at it; it takes one representative from each class and then attacks the boundaries. A black-box test table records the value, the class it targets, the expected result and the actual result:
Input 57 — normal/typical — expected ACCEPT Input 0 — lower boundary, valid — expected ACCEPT Input 100 — upper boundary, valid — expected ACCEPT Input -1 — lower boundary, invalid — expected REJECT Input 101 — upper boundary, invalid — expected REJECT Input 'seven' — erroneous, wrong data type — expected REJECT Input (blank) — erroneous, missing value — expected REJECT
The two most valuable rows are 100 and 101. An off-by-one slip in the code — writing m < 100 where the design says m <= 100 — passes every typical value and only ever reveals itself at the boundary. That single pair of tests catches more real faults than a hundred random values between 20 and 80, because exhaustive testing is impossible and boundaries are where working-looking code actually breaks.
DataEvaluation: judging the finished system against its own promises
Evaluation is the disciplined judgement of the finished system against the measurable success criteria that analysis wrote down — not a vague verdict on whether the project felt successful. You assess whether each requirement was met and back the claim with evidence from testing and from real users. Evaluation weighs several qualities the exam names explicitly: robustness and reliability (does it cope with erroneous input and heavy load without crashing?), usability (can the intended users actually operate it?), maintainability (can it be understood and changed later?), and efficiency (does it meet its performance targets?). A strong evaluation is honest about limitations and proposes specific improvements, which is what turns the lifecycle from a straight line into a loop feeding the next version. When TSB migrated 1.9 million customers from Lloyds' systems onto a new platform in April 2018, the migration was declared complete and switched on, yet customers were locked out for weeks and the eventual cost ran past £300 million — an evaluation that measured against the plan rather than against what real users could actually do on the morning it went live.
CaseWhy the order — and the loop — is the whole point
The five stages are usually drawn as a sequence, but in practice they form an iterative cycle: evaluation exposes gaps that send you back to analysis for the next version, and testing routinely uncovers a design flaw that must be fixed before implementation continues. The reason the order matters so much is the cost of change. Barry Boehm's long-running research into software projects found that a defect introduced in the requirements stage but not caught until after release can cost on the order of a hundred times more to fix than if it had been caught during analysis, because by then it is baked into design, code, tests and user habits. That single curve explains every case in this section. The London Ambulance Service, the Mars Climate Orbiter, Ariane 5, Knight Capital and TSB were not undone by programmers who could not code — they were undone by a requirement never agreed, an interface never tested, reused code never re-checked, a deployment never verified, or an evaluation that measured the plan instead of the user. When an exam hands you a scenario, run the same scan: which of the five stages was skipped or rushed, what was its proper deliverable, and what would catching it early have saved?
VocabularyKey terms the mark scheme pays for
TrapsMisconceptions that cost marks
ExamWhat examiners want
This section underpins the non-exam assessment and is examined in the written papers through short structured and scenario questions, so it is marked heavily on AO1 (knowing the stages and their deliverables) and AO2 (applying them to an unfamiliar scenario). Name the stage precisely and pair it with what it produces — analysis yields requirements, a feasibility study and success criteria; design yields modules, algorithms and data-structure choices; evaluation judges against those success criteria. Interchanging the stages, or calling design 'planning' and evaluation 'testing', loses marks a mark scheme will not forgive.
On test-data questions, always give a concrete value, state which class it belongs to (normal, boundary or erroneous) and the expected result — 'a suitable item of test data' means an actual number or string, not the word 'invalid'. The commonest dropped mark is confusing boundary with erroneous: for a valid range of 0 to 100, the boundary tests are 0, 100, -1 and 101, while a letter or a blank is erroneous. When a question hands you a project scenario, apply the cost-of-change reasoning explicitly: identify which stage was skipped, name its proper deliverable, and explain that a fault caught in analysis is far cheaper than the same fault caught after release — that applied judgement is where the higher-tariff marks sit.