Performance optimization

[lordbink]

How does nbd manage processes and threads? I am looking into this to find out what optimizations may be required and what settings to watch for as I start to use nbd-server more. Is there anything I need to watch out for when it comes to performance?

Is there a limit to the number of processes that are started? Is there a way to manage the number of processes?

The man page describes that the nbd-server will read requests in a main thread and handle those requests in separate threads. It appears, in my test environment, that a new process is always started when needed, as a client connects.

What is the correct way to get the number of threads?

When I set max_threads to 30, threads always stays at 1. I am checking this with cat /prod/<nbd process id>/status | grep Thread. Does the fact that I am making read-only requests to the server have something to do with this?

My environment

(for context)

• PXE bootable
• ~1000-2000 PXE booted read-only connections (to nbd server)
• raw file image created from a Cloud-init install, Ubuntu 22.04
• overlayfs to create a read only operating system (changes are not save after a reboot)
• nbd-server
• 1 nbd conf.d export (soon to be 5+)

Thank you in advance for your help.

L

[yoe]

nbd-server uses glib's GThreadPool to implement async handling of requests. That has a function to set the maximum number of threads, which is what we use when you set max_threads.

At this point, nbd-server still uses fork-per-child semantics, and the threadpool handling is only started after the fork. While I tried improving that situation a few times in the past, unfortunately that resulted in security issues every time, so I'm not going to do that again without a major refactoring of nbd-server first. I still want to do that at some point in the future, but unfortunately I don't have much time to do this right now.

[lordbink]

We are no longer using NBD and switched back to NFS for our use case.

[yoe]

However, the issue still exists, and I do still want to keep it on my radar, so reopening.