Audit Process

Context

P1: Audit Process

When teams need an audit, there should be a clear process with owners for all required steps: defining requirements and invariants, getting internal approvals, working with program management, talking to auditors, determining how many audits to get, what kinds of audits, negotiating audit prices, scheduling the audit, determining if a fix review is needed, and what to do with the results of an audit.

This document describes the use of software audits at Op Labs. It includes:

The resulting process integrates with the SDLC and enlists PgM and EVM Safety to help the Tech Lead execute the steps that are common to all audits, so that effort and uncertainty are minimized.

For further context on this process, you can read this companion document and the references.

Summary

Note: If you identify that a new audit vendor will be required, you should start the onboarding process as soon as possible. The Zip request for new auditors will need to include legal agreements as well as the SOW. Approval can take several weeks and is a requirement for the audit starting.

Audit Procurement

We use Spearbit as our preferred auditing services provider and have established a retainer with them to streamline approval. However, the feature team can choose a different provider from this list, from past engagements, or from any other source if they have a strong reason to go outside of Spearbit.

Auditors must agree to review the fixes to the vulnerabilities reported. Auditors not wishing to agree to this step should not be selected.

The Security Readiness Document is one of the deliverables from the design review and the primary artifact needed to schedule an audit. This document will be updated as necessary during the delivery lifecycle. It contains:

  • A summary of the project (or a link to a suitable summary if it already exists).
  • All relevant links to the project documentation, including specs and FMAs.
  • The scope for the audit.
  • The desired start and end date for the audit.

Once the Security Readiness Document has been submitted, an SOW will be obtained from the vendor for approval on Zip by:

  • Choosing "Request a Purchase/Vendor Onboarding/Purchase Renewal".
  • Under "What are you looking to purchase?" select "Other".

Audit Execution

A devnet deployment is a requirement for the audit execution. As the date for the alphanet deployment is known with certainty, a date for the audit can be agreed so that the audit can be executed in parallel with the alphanet and betanet deployments and acceptance testing, and concluded before the testnet deployment.

We prefer to communicate with auditors over Slack during the audit. Questions from auditors should be answered promptly and carefully. These questions reveal gaps in the specifications or the scope, which should be amended accordingly.

Each vulnerability disclosed will be considered separately, fixed in an individual commit, and reviewed again by the auditors in the repo.

For all audit findings that we will fix as part of a later feature, create an issue for each finding in the monorepo. The issue title should be the finding title, the description should link to the audit report, and the issue should be labeled "TBD".

After Each Audit

Once all the fixes are applied and reviewed, the project lead should upload the final audit report to our repo. Please make sure that the keccak256 digest of every audited file is included in the report.

If a valid high severity vulnerability was found, and this is the last expected audit for the project, a post-mortem must be conducted and another audit of the same type must be scheduled. These new audits follow the same process as any other audit.

Emergency Process

The audit process is tied to the SDLC process. A fast-track audit process would only be needed if we find out that we need audits later in the SDLC process, most likely as a result of updates to the risk modeling or excessive vulnerabilities in the last scheduled audit. The process described above is still applicable in these cases.

If the audit process is started in later stages of the SDLC, the documentation will be ready and can be assembled into the Security Readiness Document by including a summary of the project, if that does not yet exist.

We already know that we need an audit, and we can safely assume that an external audit by Spearbit will fulfill the requirements.

The audit request still needs to be approved via the Zip process above.

Updating This Process

This process will be reviewed if SEV0 or SEV1 incidents are revealed during production, reported through a bug bounty, or caught in the last audit before production. The post-mortem might recommend updating this process.

Conversely, this process can also be reviewed with the goal of relaxing its requirements if no SEV1 or SEV0 bugs or incidents have happened in production, via the bug bounty, or in any last audit for at least six months.

References