Github Webhooks Documentation Review

This post is a part of our "Documentation Review" series, where we collaborate with different companies to review, analyze and improve their docs. You can find the rest of the posts in the series here.

As the webhook as a service provider to companies like Brex, Benchling, and Lob, we obsess over our documentation and have read a lot of webhook documentation.

GitHub is a web-based platform for version control and collaboration, allowing developers to host, review, and contribute to software projects.

The best documentation we've seen generally has these five elements: (1) signature verification, (2) event catalog, (3) retry policy, (4) troubleshooting and failure recovery, and (5) testing tools.

TL;DR:
Signature Verification: A+
Event Catalog: A
Retry Policy: B
Troubleshooting and Failure Recovery: A+
Testing Tools: A-

Signature Verification

Signature Verification is a critical security feature of webhooks. It lets users verify that a webhook was actually sent from the expected source. The signature verification section of webhook documentation should explain the signature scheme (how the signature is created) and give instructions on how to verify the signature with code samples.

GitHub uses HMACSHA256 to sign their webhooks and they provide a plethora of sample code demonstrating how to verify the signature. We also want to applaud their use of a time invariant comparison function to determine if the signatures match as this is a detail most people miss.

They also provide some common reasons for why signature verification fails and how to troubleshoot those issues.

Grade: A+

Event Catalog

The Event Catalog documents the event types users can subscribe to. They denote the type of message being sent and are the primary way for webhook consumers to configure what events they are interested in receiving. The best event catalogs list every available event type with sample payload schemas and description of what actions trigger the event.

GitHub has a very extensive event catalog. Every event type listed, full descriptions of each event, full descriptions of each parameter. The only thing we would suggest is to provide sample JSON payloads for each event.

Grade: A

Retry Policy

Retries help make webhook systems more reliable. Endpoints can fail intermittently so retrying failed messages automatically avoids users losing important notifications.

GitHub does not automatically retry failed webhook deliveries but does offer an API endpoint that lets you request a retry for Repository, Organization, and GitHub App webhooks. There is also a set of instructions as well as sample code for how to check for failed deliveries on a schedule.

Grade: B

Troubleshooting and Failure Recovery

Describing a few of the most common failure modes can save users time and effort when troubleshooting a failing endpoint. You also want to provide instructions on how to re-enable an endpoint once the issue is resolved.

Cloudinary highlights payload corruption, an invalid signature, a disabled access key, an invalid URL, and connection timeouts as common causes of webhook failures. They also offer some advice on how to avoid these issues and have a note that they currently do not disable failing endpoints.

Grade: A+

Testing Tools

GitHub again, dedicates an entire article to testing. It covers how to test webhook deliveries by triggering an event as well as how to test your webhook endpoint locally using smee.io. It's also possible to test repository and organization webhooks locally using the GitHub CLI.

There is a suggestion to open an issue in your repository to trigger an event for testing. We'd like to see the ability to send test events instead of having to trigger an actual event.

Grade: A-

Summary

We were really impressed with GitHub's webhook docs. With an entire section dedicated just to webhooks, they have articles for describing events and payloads, verifying webhook signatures, handling failed deliveries, webhook best practices, and how to troubleshoot and test webhooks (they actually have many more articles which are all very well written and informative).

It's also extremely easy to find exactly what you need despite there being so much content because there are many links throughout all of the articles to help you navigate the documentation.

We can't wait for them to implement automatic retries both as webhook documentation reviewers but also as GitHub users. Retries not being automatic and needing to write a script to poll for failures feels a bit against the spirit of webhooks. It was a bit of a challenge to give a fair grade to the documentation for the current state of retries because while retries are currently not automatic, they are documented very thoroughly and written well.

Overall, we love GitHub's webhook docs as they're extensive and give a lot of guidance to ensure an excellent developer experience.

---

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.