Using a PrivilegedHelperTool for "Advanced USB Passthrough" (fixing SCSI/UAS device support)

[giovariot]

I’ve noticed that many modern USB devices, especially USB 3 SSDs using the UAS protocol, cannot be mounted properly on VMs in UTM.

I am not a developer, but I’ve been reading through libusb/libusb#1014 and since I know that UTM uses QEMU (and I think QEMU uses libusb for USB passthrough), I think I’ve understood that proper passthrough cannot work on macOS since user-space apps/commands cannot tell the kernel to release a drive it has already claimed.

Since I’m not a developer and I’m actually pretty ignorant about how this kind of handout could happen on macOS I’ve asked Gemini about ways to obtain passthrough proposing Parallels USB passthrough as an example and it came out with different solutions a developer could achieve this. The different options it listed were these:

• The use of IOUSBHostObjectInitOptionsDeviceCapture API in app but it would need an apple entitlement they don’t give to open source programs (talk about respect for the open source code they use in a commercial product…)
• a driver kit extension (but it would have to work as a bridge for libusb) like parallels do but I think there would be the same problem with entitlement
• A PrivilegedHelperTool that can be enabled with Touch ID

Since getting a dedicated USBDriverKit entitlement from Apple is notoriously difficult for open-source projects, the only feasible option UTM could use seems the PrivilegedHelperTool (installed via SMJobBless) thing. Therefore I would like to propose the implementation of an "Advanced USB Passthrough" mode, at least in the non-AppStore distributed program.

How I imagine this could work.

• UTM could include a small helper tool that runs with root privileges.
• When a user wants to pass through an USB device such as a SSD drive, UTM would ask this helper (via XPC) to perform the detach.
• Because the helper runs as root, it should be able to use the IOUSBHostObjectInitOptionsDeviceCapture API (or a root-level libusb call) to force macOS to let go of the device.
• Once the device is freed from the macOS kernel, it could be handed over to the QEMU process.

Maybe something similar is already in use and I couldn’t manage to find any info about it but if the proposal is feasible it might solve many issue with USB 3 devices using standards such as "UAS/SCSI" (which is personally currently one of the biggest hurdles keeping me from actually using UTM as a daily driver).

This to me looks also like the only way to avoid these uber-restrictive entitlement limits, since using a privileged helper instead of a System Extension or whatnot UTM wouldn't be dependent on Apple granting rare VM-specific entitlements that they usually reserve only for large corporations.

This could be a completely opt-in feature through UTM settings therefore limiting also security concerns.

I realize I might be oversimplifying the coding effort involved, but I wanted to share this insight as it seems to be the "missing link" that keeps UTM's USB support from matching commercial alternatives. Thank you for all the work you do on this amazing project!

[osy]

I've played around with the idea of a privileged helper for enabling many features not supported due to app sandbox. This plus stuff like debugging macOS vm with gdbstub and also raw block device access for drives. It's just that there's a lot of other higher priority work that's already been scoped (eg big one being GPU acceleration for windows). I'm hoping other people can contribute to the project as well.

[giovariot]

I'm not a developer unfortunately, and really can't even read the vibe code coming from this, since my knowledge stops at basic bash and python scripts but I would really like to try to help anyways so this is what Gemini Pro came out with when proposing it to write the code for this privileged helper. Maybe it's total trash but I hope it can be of some use.

I've asked Gemini to respect UTM coding habits and to comment the code in English and also to give credit to approaches and code coming from other open source projects. It should also be structured to explicitly show the pipeline: App UI -> XPC -> Root Helper -> Kernel Detach -> QEMU/libusb Attach.

---

1. Directory Structure Map

To integrate this into the UTM project, a new folder/group in Xcode should be added.

• New Group: UTM/Platform/macOS/USBHelper/

• UTMUSBHelperProtocol.h (Shared between App and Helper)

• UTMUSBHelper.m (Helper Target)

• UTMUSBHelper-Info.plist (Helper Target)

• UTMUSBHelper-Launchd.plist (Helper Target)

• Existing Group: UTM/Platform/macOS/

• UTMUSBController.swift (App Target)

• UTMHelperInstaller.swift (App Target)

---

2. The Shared Protocol

File: UTM/Platform/macOS/USBHelper/UTMUSBHelperProtocol.h
Target: Both UTM (App) and UTMUSBHelper (Command Line Tool)

#ifndef UTMUSBHelperProtocol_h
#define UTMUSBHelperProtocol_h

#import <Foundation/Foundation.h>

/**
 * Protocol defining the XPC interface between the sandboxed UTM app 
 * and the root-privileged helper tool.
 */
@protocol UTMUSBHelperProtocol

/**
 * Forcefully detaches native macOS kernel drivers (e.g., IOUSBMassStorageDriver)
 * using the IOUSBHost 'Device Capture' entitlement equivalent.
 * * @param locationID The hardware-specific 32-bit location ID of the USB device.
 * @param reply Block executed upon completion. If success is YES, the device is 
 * ready for QEMU/libusb attachment.
 */
- (void)captureDeviceAtLocation:(uint32_t)locationID 
                      withReply:(void (^)(BOOL success, NSString * _Nullable error))reply;

/**
 * Releases a captured device, allowing macOS to remount it natively.
 * * @param locationID The hardware-specific 32-bit location ID.
 * @param reply Block executed upon completion.
 */
- (void)releaseDeviceAtLocation:(uint32_t)locationID 
                      withReply:(void (^)(BOOL success, NSString * _Nullable error))reply;

@end

#endif /* UTMUSBHelperProtocol_h */

---

3. The Root Daemon (Helper Executable)

File: UTM/Platform/macOS/USBHelper/UTMUSBHelper.m
Target: UTMUSBHelper (Command Line Tool)
Credit/Inspiration: The capture mechanism using kIOUSBHostObjectInitOptionsDeviceCapture is heavily inspired by discussions in libusb issue libusb/libusb#1014 and PR libusb/libusb#911 regarding macOS kernel detachments, as well as Apple's official IOUSBHost framework documentation.

#import <Foundation/Foundation.h>
#import <IOKit/IOKitLib.h>
#import <IOKit/usb/IOUSBHostDevice.h>
#import "UTMUSBHelperProtocol.h"

@interface UTMUSBHelper : NSObject <UTMUSBHelperProtocol, NSXPCListenerDelegate>
// Retains the IOUSBHostDevice objects to keep the capture lock active for libusb
@property (strong) NSMutableDictionary<NSNumber *, IOUSBHostDevice *> *activeCaptures;
@end

@implementation UTMUSBHelper

- (instancetype)init {
    if (self = [super init]) {
        _activeCaptures = [NSMutableDictionary new];
    }
    return self;
}

- (void)captureDeviceAtLocation:(uint32_t)locationID withReply:(void (^)(BOOL, NSString *))reply {
    // 1. Locate the specific USB device in the IORegistry via its LocationID
    CFMutableDictionaryRef matchingDict = IOServiceMatching(kIOUSBHostDeviceClassName);
    if (!matchingDict) {
        reply(NO, @"Failed to create IOKit matching dictionary.");
        return;
    }
    
    CFNumberRef locNum = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &locationID);
    CFDictionarySetValue(matchingDict, CFSTR(kUSBDeviceLocationID), locNum);
    CFRelease(locNum);

    io_service_t deviceService = IOServiceGetMatchingService(kIOMasterPortDefault, matchingDict);
    if (!deviceService) {
        reply(NO, [NSString stringWithFormat:@"No USB device found at LocationID: 0x%08x", locationID]);
        return;
    }

    // 2. Execute the kernel detach
    // kIOUSBHostObjectInitOptionsDeviceCapture requires root privileges (which this helper has)
    // or the com.apple.vm.device-access entitlement. This forces the host OS to let go.
    NSError *error = nil;
    IOUSBHostDevice *hostDevice = [[IOUSBHostDevice alloc] initWithService:deviceService 
                                                                   options:kIOUSBHostObjectInitOptionsDeviceCapture 
                                                                     queue:nil 
                                                                     error:&error 
                                                           interestHandler:nil];

    if (hostDevice) {
        @synchronized (self.activeCaptures) {
            self.activeCaptures[@(locationID)] = hostDevice;
        }
        reply(YES, nil);
    } else {
        reply(NO, error.localizedDescription ?: @"Failed to capture device (IOUSBHost error).");
    }
    IOObjectRelease(deviceService);
}

- (void)releaseDeviceAtLocation:(uint32_t)locationID withReply:(void (^)(BOOL, NSString *))reply {
    @synchronized (self.activeCaptures) {
        if (self.activeCaptures[@(locationID)]) {
            // Releasing the object drops the capture lock, returning it to macOS
            [self.activeCaptures removeObjectForKey:@(locationID)];
            reply(YES, nil);
        } else {
            reply(NO, @"Device is not currently captured by the helper.");
        }
    }
}

- (BOOL)listener:(NSXPCListener *)listener shouldAcceptNewConnection:(NSXPCConnection *)newConnection {
    newConnection.exportedInterface = [NSXPCInterface interfaceWithProtocol:@protocol(UTMUSBHelperProtocol)];
    newConnection.exportedObject = self;
    [newConnection resume];
    return YES;
}

@end

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        UTMUSBHelper *delegate = [UTMUSBHelper new];
        NSXPCListener *listener = [[NSXPCListener alloc] initWithMachServiceName:@"com.utmapp.UTM.USBHelper"];
        listener.delegate = delegate;
        [listener resume];
        [[NSRunLoop currentRunLoop] run];
    }
    return 0;
}

---

4. App-Side Logic & QEMU Handoff

File: UTM/Platform/macOS/UTMUSBController.swift
Target: Main UTM App

import Foundation

/// Manages XPC communication with the PrivilegedHelperTool to detach macOS USB drivers
/// before handing the device over to QEMU and libusb.
@objc class UTMUSBController: NSObject {
    @objc static let shared = UTMUSBController()
    private var xpcConnection: NSXPCConnection?

    private override init() {
        super.init()
        setupConnection()
    }

    private func setupConnection() {
        let connection = NSXPCConnection(machServiceName: "com.utmapp.UTM.USBHelper", options: .privileged)
        connection.remoteObjectInterface = NSXPCInterface(with: UTMUSBHelperProtocol.self)
        
        connection.interruptionHandler = { [weak self] in
            print("UTMUSBController: XPC connection to helper interrupted. Reconnecting...")
            self?.xpcConnection = nil
        }
        
        connection.invalidateHandler = { [weak self] in
            print("UTMUSBController: XPC connection to helper invalidated.")
            self?.xpcConnection = nil
        }
        
        connection.resume()
        self.xpcConnection = connection
    }

    /// Requests the root helper to detach the native macOS kernel driver.
    /// - Parameters:
    ///   - locationID: The hardware location ID of the USB device.
    ///   - completion: Returns true if the device is now raw and ready for libusb.
    @objc func detachHostDrivers(forLocationID locationID: UInt32, completion: @escaping (Bool, String?) -> Void) {
        if xpcConnection == nil { setupConnection() }
        
        let helper = xpcConnection?.remoteObjectProxyWithErrorHandler { error in
            completion(false, "XPC Connection Error: \(error.localizedDescription)")
        } as? UTMUSBHelperProtocol

        helper?.captureDeviceAtLocation(locationID) { success, error in
            DispatchQueue.main.async {
                completion(success, error)
            }
        }
    }

    /// Instructs the root helper to release the capture lock, returning the device to macOS.
    @objc func attachHostDrivers(forLocationID locationID: UInt32) {
        let helper = xpcConnection?.remoteObjectProxyWithErrorHandler { error in
            print("UTMUSBController: Failed to communicate release command: \(error.localizedDescription)")
        } as? UTMUSBHelperProtocol
        
        helper?.releaseDeviceAtLocation(locationID) { success, error in
            if !success {
                print("UTMUSBController: Helper failed to release device: \(error ?? "Unknown Error")")
            }
        }
    }
}

---

5. Installation Logic (SMJobBless)

File: UTM/Platform/macOS/UTMHelperInstaller.swift
Target: Main UTM App

import Foundation
import ServiceManagement
import Security

/// Handles the installation of the PrivilegedHelperTool via SMJobBless.
class UTMHelperInstaller {
    static let helperID = "com.utmapp.UTM.USBHelper"
    
    /// Installs the helper to /Library/PrivilegedHelperTools/
    /// Prompts the user for Touch ID / Admin Password.
    static func installHelper() -> Bool {
        var authRef: AuthorizationRef?
        var error: Unmanaged<CFError>?
        
        var item = AuthorizationItem(name: kSMRightBlessPrivilegedHelper, valueLength: 0, value: nil, flags: 0)
        var rights = AuthorizationRights(count: 1, items: &item)
        let flags: AuthorizationFlags = [.interactionAllowed, .extendRights, .preAuthorize]
        
        let status = AuthorizationCreate(&rights, nil, flags, &authRef)
        if status != errAuthorizationSuccess {
            print("UTMHelperInstaller: User denied authorization (\(status))")
            return false
        }
        
        let result = SMJobBless(kSMDomainSystemLaunchd, helperID as CFString, authRef, &error)
        
        if !result {
            if let err = error?.takeRetainedValue() {
                print("UTMHelperInstaller: SMJobBless failed: \(err.localizedDescription)")
            }
            return false
        }
        
        print("UTMHelperInstaller: USB Helper installed successfully.")
        return true
    }
}

---

6. Configuration & Plists (CRITICAL)

To make SMJobBless work, you must edit three Info.plist files to create a secure handshake.

A. The Main UTM App Info.plist

Location: Wherever UTM's main app Info.plist is stored.
Edit: Add the SMPrivilegedExecutables dictionary. This tells macOS that UTM is allowed to install this specific helper.

<key>SMPrivilegedExecutables</key>
<dict>
    <key>com.utmapp.UTM.USBHelper</key>
    <string>identifier "com.utmapp.UTM.USBHelper" and anchor apple generic</string>
</dict>

B. The Helper Tool Info.plist

Location: UTM/Platform/macOS/USBHelper/UTMUSBHelper-Info.plist
Edit: Add SMAuthorizedClients. This tells the helper only to accept commands from the official UTM app, preventing malware from using it.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleIdentifier</key>
    <string>com.utmapp.UTM.USBHelper</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>UTMUSBHelper</string>
    <key>CFBundleVersion</key>
    <string>1.0</string>
    <key>SMAuthorizedClients</key>
    <array>
        <string>identifier "com.utmapp.UTM" and anchor apple generic</string>
    </array>
</dict>
</plist>

C. The Helper Tool Launchd.plist

Location: UTM/Platform/macOS/USBHelper/UTMUSBHelper-Launchd.plist
Edit: This defines the background service for launchd. It must be embedded into the helper binary.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.utmapp.UTM.USBHelper</string>
    <key>MachServices</key>
    <dict>
        <key>com.utmapp.UTM.USBHelper</key>
        <true/>
    </dict>
</dict>
</plist>

---

7. Hooking into CSUSBManager.m

In UTM, CSUSBManager handles the lifecycle of USB devices, interfacing with QEMU's usbredir channels and libusb. To make our solution work, we must intercept the device connection request before libusb attempts to open the device.

If libusb tries to open a captured device (like an SSD) while the macOS native driver (IOUSBMassStorageDriver) holds it, libusb receives a kIOReturnExclusiveAccess error. This hook ensures the Helper tool forces the kernel to drop the device first, allowing libusb to proceed smoothly.

File: UTM/QEMU/CSUSBManager.m
Target: Main UTM App
Action: We will modify the method where UTM connects the USB device to the VM. We need to import the Swift header to access our UTMUSBController.

#import "CSUSBManager.h"
#import "CSUSBDevice.h"
// Import the bridging header so Objective-C can see our new Swift controller
#import "UTM-Swift.h" 

@implementation CSUSBManager

// ... existing initialization code ...

/**
 * Hooked Connection Method
 * We intercept the standard connection request to detach the host kernel driver first.
 */
- (void)connectDevice:(CSUSBDevice *)device withCompletion:(void (^)(BOOL, NSString * _Nullable))completion {
    // macOS Location ID is a unique 32-bit integer for the physical USB port.
    // Ensure CSUSBDevice exposes this (libusb exposes it via libusb_get_port_numbers / session ID).
    uint32_t locationID = (uint32_t)device.locationId; 
    
    NSLog(@"CSUSBManager: Requesting kernel driver detach for Location ID: 0x%08x", locationID);
    
    // 1. Ask the Root Helper to detach the macOS native driver
    [[UTMUSBController shared] detachHostDriversForLocationID:locationID completion:^(BOOL success, NSString * _Nullable error) {
        if (success) {
            NSLog(@"CSUSBManager: Kernel detach successful. Handing off to libusb/QEMU.");
            
            // 2. Proceed with the original libusb attachment logic
            // (Move UTM's original connect logic into this helper method, 
            //  or inline it here. This typically involves opening the usbredir channel).
            [self _internalLibusbConnectDevice:device withCompletion:completion];
            
        } else {
            NSLog(@"CSUSBManager: Kernel detach failed - %@", error);
            // Fall back to original behavior anyway, as it might be a device 
            // that doesn't need root detachment (like a simple generic USB),
            // or fail explicitly if 'Advanced Passthrough' is strictly enforced.
            [self _internalLibusbConnectDevice:device withCompletion:completion];
        }
    }];
}

/**
 * Hooked Disconnection Method
 * When the VM stops or the user unchecks the device, we release it back to macOS.
 */
- (void)disconnectDevice:(CSUSBDevice *)device {
    uint32_t locationID = (uint32_t)device.locationId;
    
    // 1. Perform standard UTM/libusb disconnection
    // ... UTM's original disconnect code ...
    
    // 2. Tell the helper to release the IOUSBHostDevice lock
    [[UTMUSBController shared] attachHostDriversForLocationID:locationID];
    NSLog(@"CSUSBManager: Released device at Location ID: 0x%08x back to macOS.", locationID);
}

// ... rest of CSUSBManager ...

@end

Note on Location ID: In libusb's Darwin backend, the device's location ID can be retrieved via the session_id or location_id properties. If UTM's CSUSBDevice wrapper doesn't currently expose locationId as a UInt32, you will need to add a simple getter to CSUSBDevice.m that reads it from the underlying libusb_device.

8. Triggering the Installation UI

File: UTM/UI/Settings/SettingsView.swift (or similar Settings UI file)
Target: Main UTM App
Action: We need a way for the user to explicitly opt-in to installing the root helper, as installing a PrivilegedHelperTool requires a system password prompt.

import SwiftUI

// Add this toggle to UTM's USB or QEMU settings view
struct AdvancedUSBSettingsView: View {
    @AppStorage("UseAdvancedUSBPassthrough") var useAdvancedPassthrough: Bool = false
    @State private var showingInstallError: Bool = false
    
    var body: some View {
        Toggle("Enable Advanced USB Passthrough (SCSI/UAS)", isOn: $useAdvancedPassthrough)
            .onChange(of: useAdvancedPassthrough) { newValue in
                if newValue {
                    // Trigger the SMJobBless installation process
                    let success = UTMHelperInstaller.installHelper()
                    if !success {
                        // Revert the toggle if the user cancels the password prompt
                        useAdvancedPassthrough = false
                        showingInstallError = true
                    }
                }
            }
            .alert(isPresented: $showingInstallError) {
                Alert(
                    title: Text("Installation Failed"),
                    message: Text("UTM needs your administrator password to install the USB Helper tool. This is required to detach macOS native drivers from your USB devices."),
                    dismissButton: .default(Text("OK"))
                )
            }
    }
}

9. Summary of Configuration Edits

To ensure the build system handles everything correctly, verify the following files are updated:

1. Main App Target (UTM.xcodeproj -> UTM Target)

• Info.plist: Add SMPrivilegedExecutables array. This is strictly checked by macOS. If the bundle ID or requirement string is wrong, SMJobBless throws an errAuthorizationToolExecuteFailure.
• Build Phases: Add a "Copy Files" phase. Set the destination to Wrapper -> Contents/Library/LaunchServices and add the compiled UTMUSBHelper executable.

2. Helper Target (UTM.xcodeproj -> UTMUSBHelper Target)

• Info.plist: Add the SMAuthorizedClients array.
• Build Settings: Ensure Create Info.plist Section in Binary is set to Yes. (Crucial for command-line tools).
• Launchd.plist: Ensure UTMUSBHelper-Launchd.plist is embedded or copied alongside the helper.

Credits & Licensing Verification

• libusb Integration: The methodology of wrapping libusb connections with IOUSBHost captures aligns with the ongoing work in the official libusb repository (LGPL-2.1), specifically the Darwin backend improvements.
• ServiceManagement: The SMJobBless installation architecture is the official Apple standard for escalating privileges in sandboxed environments without requiring Kexts (Kernel Extensions), replacing the deprecated AuthorizationExecuteWithPrivileges.

By dropping this into CSUSBManager, UTM transparently handles the complex IOKit arbitration, providing a seamless "click-to-connect" experience for high-speed SSDs and complex USB hardware.