Enable use of PacketStream with Offline capture

[fcelda]

Attempting to use stream() on an offline capture (e.g. returned by pcap::capture::from_file) results in NonNonBlock error. The offline capture however cannot be switched to non-blocking mode (it's not supported by the API of the pcap crate nor by pcap_setnonblock).

I think it would be nice if the stream also work for the offline stream. I imagine the async future would always return as ready until the whole file is read.

I have a use case in unit testing packet processing that normally happens with live capture.

I would also be happy to contribute but I'll likely need some pointers in what needs to done. I'm relatively new to Rust.

[Stargateur]

you can try to remove the if

pcap/src/stream/mod.rs

Line 25 in 8b7bcfe

if !self.is_nonblock() {

and see what happen.

[fcelda]

Removing the check for non-blocking mode doesn't work. The offline capture just cannot be polled by libpcap.

I think that in order to support async rust with offline capture the implementation in rust-pcap has to be different from the online capture.

I tried to provide an alternative implementation of

pcap/src/stream/unix.rs

Line 43 in 8b7bcfe

impl<T: Activated + ?Sized, C: PacketCodec> futures::Stream for PacketStream<T, C> {

for Capture<Offline> but couldn't get it working. This is where my Rust knowledge may be lacking. I think "specialization" is not possible under Rust type system so it either needs dynamic dispatch for poll_next or an incompatible change in the API. I welcome any guidance on that.

Btw, I don't need the streaming functionality per se. I just want to be able to read the packets from the capture with async. I noticed there is #322 that is related.

[Stargateur]

if pcap can't handle it we have no other choice to implement a rust side solution to read pcap savefile, https://www.tcpdump.org/manpages/pcap-savefile.5.html. It's look very simple.

Thus this look like more suitable for separate crate.

I just want to be able to read the packets from the capture with async.

since it's almost instant, why not just do a wrapper that is async on your side and just call the "sync" version of pcap for file ? ugly but should work.

[fcelda]

since it's almost instant, why not just do a wrapper that is async on your side and just call the "sync" version of pcap for file ? ugly but should work.

I want a single implementation for any kind of active capture (i.e. online or offline). I have a service that does online capturing that I cannot unit test with a capture loaded from a file. That is essentially my ask.

[Stargateur]

I still wait for a lending stream crate that is worth trying https://docs.rs/lending-stream/latest/lending_stream/index.html is meh. Also, the main problem is pcap can't do a trait that is 100% correct to your use case. It's very hard mixing all there sync and async of libpcap, and the GaTs problem. To do it properly we must use async of Rust to check if the file is readable like I did with the socket.

Removing the check for non-blocking mode doesn't work. The offline capture just cannot be polled by libpcap.

why ? I'm pretty sure it would work honestly. At least on unix. MMMM after reading I think we just need to set the nonblock at the hand since pcap don't want to.

Thus I think even our current async implementation is far from good cause pcap API is limited.

[fcelda]

why ? I'm pretty sure it would work honestly. At least on unix. MMMM after reading I think we just need to set the nonblock at the hand since pcap don't want to.

I originally removed the check you indicated in your first comment and the Rust code paniced. I'm don't remember exactly where but I believe it was related to polling.

The offline capture cannot be put into non-blocking mode for sure, see pcap_setnonblock. The pcap_get_selectable_fd man page is confusing and I'm not sure if it returns anything meaningful for offline captures.

I also found an older issue the-tcpdump-group/libpcap#870 where somebody tries to poll on the savefile and I think the answer from the maintainers is that it's not possible and it would require non-trivial changes.

[Stargateur]

it's probably because windows doesn't handle select for file, legacy limitation. I think the best would be to handle savefile without pcap as long as they don't handle it properly. 1. do the parsing 2. open file with async of rust