Sharing `Channel` over async green threads

[raui100]

I have built a web-app using axum&tokio that creates a new green thread per request. I'm using lapin for RPC with RabbitMQ over a single channel which seems to work.
I've stumbled across the RabbitMQ documentation which discourages sharing Channels. Does this apply to lapin::Channel too and does this apply in an async context?

Is it recomment to share the connection and create a new lapin::Channel per incoming request?

Also it says in the documentation that closing a connection closes the underlying channels. Dropping a lapin::Connection doesn't close the underlying Connection which surprised me. Is it necessary to implement clean up code that closes all lapin::Connection and lapin::Channels at the end?

[Keruspe]

Hello,
I'm so sorry, I was sure I answered you but I obviously haven't.

Basically, my point here will be: i'll always encourage you to follow the official documentation. It's there for a reason.

That being saidn I'm also well aware that this kind of specificities can be hard and/or counter intuitive. Especially when dealing with green threads and not OS threads.

Everything possible has been put into place in lapin to make sharing a Channel across threads (even OS threads) safe. Well, as safe as possible anyways.
In high throughput context, with finely crafted interactions, I managed to actually make rabbitmq-server bug in several ways. It's actually possible to reproduce in a monothreaded env, but was easier with several threads. The issue got reported upstream but then I stopped being paid to work on it so I never got time to actually complete the debugging with upstream. A workaround got implemented in lapin to avoid hitting the faulty codepath in the server, forcing some ordering of frames even if not strictly necessary in theory according to the protocol specs.

WRT the connection and channel closing, you can see lapin::Connection and lapin::Channel like some kind of smart pointers, backed by std::sync::Arc. Basically, we'll automatically close them when the last ref is dropped (unless you already clsoed it before). Each Channel holds a ref to the Connection, so the Connection will get closed automatically only after all the Channels got closed themselves.

[raui100]

Hi, thank you for your thorough answer :) To clarify:

1. lapin::Channel and lapin::Connection can be shared in multiple threads, because they implement their own syncing mechanism.
2. It is not necessary to clean up after them, because they implement their own RAII and clean up mechanisms.

This means the lapin::Channel and lapin::Connection have implementation details that makes them different from the Channel and Connection that are described in the RabbitMQ documentation.

I have another question regarding best practices. I have been using the connection pool from deadpool in combination with lapin.
Whenever I need to send a new message I fetch a lapin::Connection from the pool, create a new lapin::Channel and use it to send a message. This seems to work really well. I had used long living lapin::Channels before and they sometimes timeout and loose their connection.

1. Is it reasonable/idiomatic to create a lapin::Channel for each message I want to send? lapin::Connection::create_channel(..) seemed cheap to me. Is there a better way?
2. Is it correct that lapin::Consumer have an heartbeat that keeps them alive forever even if they don't get any messages?
3. Why do the examples of this crate often use multiplelapin::Channels created from the same lapin::Connection? Is this recommended? Is it bad to use a single lapin::Channel to create multiple lapin::Consumer and use it to send messages?

[academiaresf]

I'm also interested in this questions.

[snichme]

Regarding creating a channel for every message, it depends on how you workload look. But generally you shouldn't create a new channel for each message. Yes they are cheap to create but constantly create and drop channels still impacts the broker. Each connection also have a channel max so if you create one channel per message and after a while start sending more messages you might not be able to open new channels.

Error handling in AMQP is on a channel level, so if you do multiple actions on the same channel, like consume and produce, one might affect the other.
The best setup is that you have one connection per producer and then open one connection for your consumers (where each consumer uses it's own channel on that connection).
This allows the broker to manage flow control for the producers without impacting your consumers and each consumer has it's own isolated channel to handle errors. If one consumer crashes the others will continue and you can just open a new channel for the consumer that crashed.

Hope that helps.

[Keruspe]

This is definitely a complex topic (which is why I haven't yet provided an actual answer here).
I'll try to provide a proper documentation for this directly in the repository and/or rustdoc