Consuming Webhooks with OpenTelemetry

Svix heavily leverages OpenTelemetry internally and has many features available to make OpenTelemetry data consumable by our customers and yours. In this article I'd like to discuss some of the specific ways your customers can use OpenTelemetry to consume your events.

Svix offers the OpenTelemetry Collector Advanced Destination Type to give your customers better ways to monitor the events important to them. Our documentation covers the basics of endpoint configuration, which you can easily enable for most major Observability platforms with minimal effort. We also have Connectors for many of the most popular platforms, including Datadog, New Relic, and Grafana Cloud, in order to make endpoint setup even easier for your customers.

Let's say you run a SaaS platform that manages email campaigns on behalf of your customers and offer the following events as webhooks to your customers to let them know the status of their campaigns.

• email.sent
• email.clicked
• email.unsubscribed
• email.bounced
• email.delivery_failed

Using an Observability Platform like Datadog or Grafana, your customers can easily build visualizations and alerts based on these incoming events. Setting it up for your customers is easy. You just need to enable the OpenTelemetry Advanced Destination type in the Dashboard to make it available to your customers.

If you're planning to use a Connector customized for one of the supported Observability platforms, you can create it in the Dashboard Connectors page.

Using the Datadog Tracing connector, for example, your customers will be prompted to enter the appropriate values to enable exports directly to Datadog.

Searching and visualizing data

The default Javascript transformation that's provided by the Advanced Destination works out of the box, turning the fields in your event payloads into span attributes that can be searched and aggregated. With an example payload like this, your customers can easily search APM data based on data in the webhook payload.

{
  "address": "email@example.com",
  "campaign_id": "campaign-123"
}

Searching for specific fields in payloads is simple right out of the box:

And building rich dashboards based on incoming traces takes little effort, and visualizations can easily break down data by event type or attribute:

Building metrics and alerts

While all of the event types you send may provide valuable information to your customers, some event types in particular, e.g., email.delivery_failed, represent error conditions and may mean your customers need to notify their on-call support teams so they can take action.

Your customer has two options here: Their dev team could manually build a service to track and consume these webhooks and emit internal notifications for critical events. However, this is both time-consuming and error-prone. Chances are good that your customer already uses an Observability platform like Grafana or Datadog for monitoring and alerting. In this case, they can build metrics and alerts to notify the relevant teams directly in their existing tools, with no internal development required.

Metrics are also easy to build from alerts in tools like Datadog and Grafana. Converting the raw tracing data that Svix exports into simple metrics allows your customers to capture only the specific data needed for dashboards and alerting while eliminating most of the storage costs of full tracing data.

Wrapping up

Svix's OpenTelemetry Advanced Destination turns your events into first-class observability data that your customers can use with little to no development effort. They can quickly search events, build dashboards, and configure alerts using tools they already use. Got a use case you'd like to see covered, or ideas for how we can extend our OpenTelemetry support further? Hit us up!

For the latest updates, make sure to follow us on X, Github, RSS, the Svix newsletter, and join the discussion on our community Slack.