feat: VST3 sprint — GUI editor, MIDI routing, PluginEngine, categories

[ChuxiJ]

Summary

Consolidation PR for the VST3 sprint second wave:

• GUI Editor: GuiManager + CocoaBackend for native VST3 plugin windows via IPlugView
• MIDI Routing: Piano roll → pluginEngine.noteOn/noteOff for VST3 instruments
• PluginEngine Integration: loadPlugin creates VST3PluginAdapter and wires audio graph
• Category Display: Split "Fx|EQ" into category + subcategory
• Protocol fixes: Optional reqId, instanceCreated rename, HMR-safe state sync

Test plan

• 137 Rust tests, 2665 browser tests pass
• 0 type errors
• E2E verified

Refs #888

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Consolidation of the VST3 “second wave” work: native GUI editor window support in the companion app, MIDI routing from piano roll to VST3 instruments, initial PluginEngine integration for VST3 instances, and improved plugin category/subcategory handling across the protocol.

Changes:

• Add companion-side GuiManager with a macOS Cocoa backend to open/close VST3 editors via IPlugView.
• Route piano-roll note playback to VST3 instances via pluginEngine.noteOn/noteOff, and create a VST3PluginAdapter on plugin load.
• Split moduleinfo category strings (e.g. Fx|EQ) into category + subcategory, and propagate the new field through protocol + browser mapping.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/store/vst3Store.ts	Creates a VST3PluginAdapter on load and registers it with pluginEngine; updates openEditor messaging.
src/services/vst3bridge/VST3AudioWorklet.ts	Adds a runtime guard/error when SharedArrayBuffer is unavailable (COOP/COEP requirement).
src/hooks/useVST3Connection.ts	Updates scan result mapping to support subcategory and improved instrument detection heuristics.
src/hooks/useTransport.ts	Routes piano-roll MIDI playback to VST3 via pluginEngine when a VST3 instance is present on the track.
companion/src/ws_server.rs	Wires OpenEditor/CloseEditor to the new GuiManager and stores it in shared app state.
companion/src/protocol.rs	Adds subcategory to PluginInfo and updates tests accordingly.
companion/src/plugin_scanner.rs	Splits moduleinfo.json category strings into category and subcategory (and updates tests).
companion/src/plugin_host.rs	Exposes get_controller() to retrieve IEditController for GUI creation.
companion/src/gui_manager.rs	Implements native editor window management with a macOS Cocoa backend and a stub backend for tests.
companion/Cargo.toml	Adds macOS-only dependencies for Cocoa GUI support.
companion/Cargo.lock	Locks new macOS dependency graph.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The adapter is built with a synthetic instantiateResponse where latencySamples is hard-coded to 0 and presets/tail latency info are dropped. The companion’s instanceCreated message includes latencySamples/tailSamples/presets; use those fields so VST3PluginAdapter.latencySamples and any latency compensation / preset UI stay accurate.

[Copilot]

vst3Instrument is inferred from “any enabled+online VST3 instance on this track”, but instances don’t distinguish instrument vs effect. If an effect VST3 is loaded on a piano-roll track, this will disable the sampler/synth and route noteOn/noteOff to the effect (via pluginEngine), which is incorrect. Suggest looking up the plugin’s category (e.g. via useVST3Store.getState().plugins by pluginId) and only treating category === 'instrument' as a MIDI target.

Suggested change

	// Check for VST3 instrument on this track
	const vst3Instances = Object.values(useVST3Store.getState().instances);
	const vst3Instrument = vst3Instances.find(
	(inst) => inst.trackId === track.id && inst.enabled && inst.online,
	);
	// Check for VST3 *instrument* on this track
	const { instances, plugins } = useVST3Store.getState();
	const vst3Instrument = Object.values(instances).find((inst) => {
	if (!(inst.trackId === track.id && inst.enabled && inst.online)) {
	return false;
	}
	const plugin = plugins[inst.pluginId];
	return plugin?.category === 'instrument';
	});

[Copilot]

midiVelocity is clamped with Math.max(1, …), which turns zero-velocity notes into audible note-ons at velocity 1. If zero velocity is used (e.g. muted notes or imported MIDI), this changes playback. Consider allowing 0 (or skipping note-on when velocity is 0) and relying on the scheduled note-off for silence.

[Copilot]

EditorOpened / EditorClosed responses aren’t correlated to a request (reqId isn’t captured from OpenEditor and isn’t included in the outgoing messages). The browser client’s VST3BridgeClient.openEditor() uses reqId-based request() semantics, so it will time out against this server. Consider adding optional req_id to IncomingMessage::OpenEditor/CloseEditor and echoing it back on OutgoingMessage::EditorOpened/EditorClosed (similar to Instantiate).

[Copilot]

Using state.gui_manager.lock().unwrap() will panic the whole server if the mutex is poisoned (e.g. a previous GUI operation panicked while holding the lock). Prefer handling the LockResult and returning an OutgoingMessage::Error so the server stays up.

[Copilot]

core-graphics is added as a macOS-only dependency but doesn’t appear to be used anywhere in companion/src yet. If it’s not needed for the current implementation, consider removing it for now to reduce compile time and dependency surface (can be re-added when required).

Suggested change

core-graphics = "0.24"

[Copilot]

loadPlugin registers a VST3PluginAdapter with pluginEngine, but PluginEngine's nodes are never spliced into the track audio graph (no pluginEngine.getInputNode/getOutputNode usage anywhere, and TrackNode only supports spliceEffects). As-is, adding the adapter won’t route audio through the plugin (and instruments won’t be audible either). Consider wiring the returned PluginAudioNode into the track node (similar to TrackNode.spliceEffects) or adding a dedicated splice point for plugin chains.

[Copilot]

The empty catch {} around adapter creation silently leaves the store thinking the instance is online/enabled while pluginEngine never gets the plugin, so MIDI/audio routing can fail with no feedback. At minimum, log/report the error (or mark the instance offline / retry once the AudioContext is available) so failures aren’t silent.

Suggested change

	} catch {
	// Audio engine not ready — adapter will be created on next rebuild
	} catch (error) {
	// Audio engine not ready or adapter creation failed — log and mark instance offline
	console.error('Failed to create VST3 audio adapter', {
	pluginId,
	instanceId,
	error,
	});
	const { instances } = get();
	const current = instances[instanceId];
	if (current) {
	set({
	instances: {
	...instances,
	[instanceId]: { ...current, online: false },
	},
	});
	}
	toastError(`Audio engine is not ready. "${pluginInfo.name}" is loaded but currently offline.`);

[Copilot]

Minor: trackId is declared twice (const trackId = track.id; above and again inside the vst3Instrument branch). This shadowing is unnecessary and makes the scheduling block harder to read—reuse the existing trackId constant.

Suggested change

const trackId = track.id;

[Copilot]

This change adds a new MIDI routing branch to pluginEngine.noteOn/noteOff for piano-roll playback, but there doesn’t appear to be test coverage validating that (a) VST3 instances on a track are detected correctly and (b) noteOn/noteOff are scheduled with the right timing/velocity. Consider adding a unit test around the scheduling logic (similar to other hook tests under src/hooks/__tests__).