Omniqueue 0.2.0 is out

We're excited to announce version 0.2.0 of Omniqueue!

Initially published in June 2023, Omniqueue is a Rust library that provides a common interface over queueing services like Amazon SQS, RabbitMQ and so on.

It allows users to send messages to the queue backend or receive messages from it using a uniform API that's independent of the backend, including optional dynamic dispatch.

New Backend: Google Cloud Pub/Sub

Our first release included one in-memory backend for testing as well as three different backends for production (Amazon SQS, RabbitMQ, Redis Streams). This release adds Google Cloud's Pub/Sub as another backend. It is enabled by default, but if you have default features disabled for better compile times, add gcp_pubsub to the Omniqueue features in your dependency specification to enable it.

You can initialize a producer + consumer pair as follows:

use omniqueue::backends::{GcpPubSubBackend, GcpPubSubConfig};

let config = GcpPubSubConfig {
    topic_id: "...".to_owned(),
    subscription_id: "...".to_owned(),
    // optional file path of a credentials file
    // if not specified, credentials will be read from the environment
    credentials_file: None,
};
let (producer, consumer) = GcpPubSubBackend::builder(config).build_pair().await?;

Refined public API

When we first announced this crate, it was still in early development. It was clear that there could still be substantial changes to the overall shape of the API.

The API of the new release should still look familiar if you have previously used Omniqueue, but you'll notice that some symbols are now available at different paths or no longer need to be imported for some code that previously required them.

For instance, consider a module that contains a function that takes a DynProducer (a producer where the backend has been type-erased) and sends a message through it. Previously, you would have had to

// QueueProducer trait needs to be imported for the sending method,
// i.e. `producer.send_raw(...)` or `producer.send_serde_json(...)`.
use omniqueue::queue::producer::{DynProducer, QueueProducer};

With omniqueue 0.2.0, this is reduced to

// All producer types now have inherent `send_raw` and `serde_serde_json` methods.
// Commonly-used modules are now available higher up in the module hierarchy.
use omniqueue::DynProducer;

Likewise, we wrote more documentation comments to make the crate docs more accessible (but we still have some more documenting to do!).

Removed feature: Custom encoders / decoders

In the process of integrating Omniqueue into the Svix Webhooks Server, when revisiting the API, one thing stood out that looked slightly weird and was also relatively complex internally: custom encoders and decoders.

After considering potential alternative ways to encode and decode payloads outside of the built-in serde_json support (which is much simpler), we decided that this is a concern that should be handled at the application level and thus removed this feature from Omniqueue.

If you were using this API, we hope that you will find it easy enough to by wrap the Omniqueue API in a higher-level application-specific API that takes care of encoding and decoding.

We are open to feedback on this removal, and any other parts of Omniqueue, through our public issue tracker on GitHub.

Upgrading

For a more raw, technical summary of the changes to check when upgrading, see the changelog.

About v0.3 and beyond

With this release out, we have removed the "early development" note from the readme and don't expect to do further large-scale API changes short-term. However, since

• some of our dependencies frequently have breaking changes and are part of the public API
• sometimes very small API changes are also breaking
• we are now using Omniqueue internally as well, so have a stronger incentive to keep it updated

it will likely not take very long until there's a v0.3.0 and v0.4.0 after it. We will try to minimize the number of changes that would usually require manual changes, and make the upgrading process as smooth as we can!

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.