Scoping a software audit: which decision should the evidence support?

Define the decision first

A useful starting question might be whether a customer portal can support a second organisation or whether a new team can operate an existing application. The investigator can then select relevant access, data boundaries and release processes. Asking whether the software is good remains too broad to define a comparable assignment.

Describe the possible outcomes as well. The investigation may show that a small change is sufficient, further investigation is needed or the proposed expansion is currently unwise. Expecting confirmation of a solution already chosen reduces the value of the exercise. Agree who decides and which uncertainties that person needs to understand.

Identify the system, environment and version

Record which applications, integrations, code versions and environments belong to the investigation. A repository review says little about production configuration when that configuration was not inspected. A test environment may use different permissions or data. The report should describe these differences so its conclusions are not silently extended to a wider environment.

Agree how changes during the investigation are handled. If a major release occurs meanwhile, each finding still needs a clear relationship to the state examined. A bounded retest can revisit an earlier finding without repeating the entire audit. This requires a traceable version and a defined test rather than an ambiguous claim that the report is current.

Separate observation, consequence and hypothesis

An observation states what the investigator actually saw: an error, a code path, an absent control or a completed test. The consequence explains why it matters to a business journey. A hypothesis describes what might fail but has not been demonstrated. A useful report keeps these layers distinct.

Suppose an export appears to start without a visible permission check. That is not automatically proof of unauthorised access; an earlier step may protect the function. Ask which part of the route was investigated and which test could confirm the hypothesis. This avoids unnecessary alarm while still treating a potentially important signal seriously.

Agree how findings will connect evidence, consequences and a follow-up decision: From audit finding to improvement: evidence, priority and retesting.

Choose samples that explain something

A sample should match the question. To investigate organisation boundaries, choose users with different roles and records from two organisations. For maintainability, examine a recent change and a path with many dependencies. A random selection of files may help exploration but says less about the actual business risks.

Record selection and limitations. One correctly rejected request does not prove every route uses the same control. One slow search does not prove all users experience it. Ask which follow-up sample would be useful if a signal appears. The organisation can then choose deliberately between a narrow answer and additional coverage.

Security investigation needs its own agreement

Decide which checks are authorised and where they may run. A code discussion, configuration review and active security test are different activities. Include limits for system load, personal data and data modification. An investigator should not improvise a potentially harmful test merely because a general audit was commissioned.

OWASP ASVS can help select and name particular security requirements. Referencing the framework does not itself constitute complete verification or certification. Ask which requirements were examined, how they were assessed and what limitations applied. Connect access or data-boundary findings to the affected user journey so the technical observation has an understandable meaning.

OWASP Application Security Verification Standard ↗

Make performance questions reproducible

Translate slow into a described operation with data volume, environment and observation time. A screen showing one hundred records has different demands from a historical export. Investigate whether delay occurs while fetching, calculating, transferring or displaying information. A general speed assessment without this context makes subsequent proposals difficult to verify.

Use an agreed measurement setup and describe differences from production. A small test may identify a bottleneck without supporting predictions about every future workload. Also decide whether the assignment investigates the cause or implements an improvement. Diagnosis and optimisation are different deliverables even when the same developer performs both.

Google SRE: Testing for Reliability ↗

Request a report that supports choices

For each significant finding, state the investigation, available evidence, possible business consequence and suitable next step. Include uncertainty and dependencies. A severity label without explanation does not show whether an issue needs correction today or further investigation first.

A practical priority list distinguishes immediate measures, additional investigation and work that can be planned later. Name an owner who can assess each recommendation. Identify related findings too: one cause may explain several symptoms. This helps prevent the organisation treating every bullet as a separate development assignment and buying duplicate work.

Example: expanding a portal

Illustrative example: a company wants to offer its existing portal to a second customer organisation. The initial investigation focuses on how users, files and exports are associated with an organisation. A general assessment of all programming style sits outside that first scope.

The investigator follows two user journeys using specially created test records. One export path applies organisation selection differently from the normal screens. The report records the difference, the performed test and routes not yet inspected. Follow-up proposes a targeted correction and retest. That is an evidenced step without claiming the entire portal has been exhaustively examined afterwards.

Agree how findings are closed

A code change does not automatically close a finding. Agree which behaviour must be demonstrated afterwards, in which environment and on which version. Include checks for unwanted side effects where relevant. Preserve the original observation and the outcome of the retest.

For some findings, informed acceptance is a business decision, perhaps because a limited feature will soon be retired. Record who makes that decision, which limitation remains and when it will be reconsidered. The audit then becomes an updateable decision record rather than a one-off list that loses its meaning after delivery.

For example, closing an access finding might require one authorised user to complete the intended operation and a different user to be refused on the same record. The agreement should identify the operation and evidence rather than merely request another screenshot. If access is blocked for everyone, the original exposure may disappear while the business function also stops working. A meaningful retest checks both the boundary and the legitimate path.

Turn the explanation into a project decision

Record your assumptions, unknowns and acceptance checks in the matching tool. Use the service guides to discuss delivery and scope.

• Legacy software modernisation ↗
• Software audit ↗

Questions about this topic

Continue with a related question