51直播

Incident response process

Purpose

51直播鈥檚 customers are dependent on our services operating as normal. Proper detection and response to incidents that may impact the integrity, confidentiality or availability of services and data is critical to the operation of 51直播.

Scope

The following minimum standards apply to 51直播鈥檚 assets as managed by employees, contractors and vendors. These recommendations represent the recommended minimum efforts necessary for incident detection and response.

Policy

Incident response

When a suspected incident is reported, it is first investigated by the eng-primary oncall. If it is suspected to be an incident, they must declare an incident, and identify the Incident Commander in the #incident-response Slack channel. Information from an incident must be kept confidential. The Incident Commander is responsible for:

• If an incident is likely to require ongoing response and remediation efforts, opening a GitHub issue in the tailscale/corp repo to track updates to the incident and creating a Google doc for collaborative work.
• Classifying the severity of the incident, including scope and the risk of any assets which may be affected. This can be further updated as information changes, and may inform how we choose to react. Depending on the urgency of the incident, this may be done after the fact.
• Contacting vendors or coordinating to contact vendors, to validate if their product may be compromised.
• Appointing roles, including a communications lead, if needed.
• Ensuring handoff between team members, for example, at the end of a work day.
• Escalating to leadership if responses are insufficient.

In addition to remediating the incident, 51直播 employees must also seek to put into place any corrective actions possible to lessen the impact of an incident.

If an incident affects customers, including their data or their ability to use 51直播, 51直播 may choose to proactively communicate the issue publicly.

Incident recovery

If data or processes were disrupted by the incident, then the BCP/DR policy must be followed to remediate the issue.

Once an incident is mitigated or otherwise closed, it is the Incident Commander鈥檚 responsibility to ensure that

• The resolution is communicated to all affected parties, including external customers, if applicable.
• For incidents causing a production outage or loss of customer or other critical data, a post-mortem is completed. This should include: details of the incident, timeline of the incident, its impact, the actions taken to mitigate or resolve it, the root cause(s), and the follow-up actions to prevent the incident from recurring. Where applicable, some version of the post-mortem may be shared with external affected parties. Newly identified risks must be added to the risk register.

Incident classification

An incident is an adverse event which affects 51直播鈥檚 infrastructure or business operations in such a way that it compromises our ability to deliver the service customers expect. A vulnerability is not necessarily an incident; for example, a vulnerability not being actively exploited may require action, but not expedited action beyond existing vulnerability remediation processes.

Incidents can be classified based on their severity:

Roles and responsibilities

51直播鈥檚 Security team is responsible for reviewing and updating the incident response process on an annual basis.