Sound API: Web Audio errors and unexpected `playSound()` promise behavior with `loop` parameter

[julianharty]

Description

This issue documents two sound-related bugs discovered during sound test integration, along with important behavioural findings about the playSound() API that may need addressing.

Note: the tests that found this issue will be submitted in a PR together with the proposed fix(es).

---

Issue 1: Web Audio linearRampToValueAtTime Error

Problem

When running sound tests, the following error appeared in browser console:

[browser:pageerror] Error [TypeError]: Failed to execute 'linearRampToValueAtTime'
on 'AudioParam': The provided double value is non-finite.
    at Object.playMidiNote (http://localhost:5173/api/sound.js:302:19)

Root Cause

The error was triggered when playNotes() received mismatched array lengths (e.g., 3 notes but only 2 durations):

flock.playNotes('box', {
  notes: [60, 62, 64],     // 3 notes
  durations: [0.5, 0.5]    // Only 2 durations!
});

Bug chain:

1. durations[2] returns undefined
2. Number(undefined) converts to NaN
3. playTime + NaN - gap - NaN = NaN
4. Web Audio API's linearRampToValueAtTime(0, NaN) throws error

Expected Behavior

The API should handle mismatched array lengths gracefully without throwing Web Audio API errors. Missing durations should either:

• Default to a reasonable value (e.g., 0.5 beats)
• Be validated and rejected with a clear error message
• Be documented as requiring matching array lengths

---

Issue 2: Sound Replacement Test Failures and loop Parameter Behavior

Problem

Tests attempting to verify sound replacement on a mesh were failing with:

AssertionError: expected undefined not to be undefined

Investigation Findings

The playSound() API behaves fundamentally differently based on the loop parameter:

With loop: true

• Promise resolves immediately after sound is created and attached to mesh
• Returns the Sound object
• Sound continues playing indefinitely until stopped

With loop: false

• Promise resolves only when the sound finishes playing
• When sound ends, it automatically removes itself from mesh.metadata.currentSound (sound.js:104-106)
• If you await the promise, by the time it resolves, currentSound is already undefined

Sound Replacement Timing

When playSound() is called on a mesh that already has a sound:

1. Synchronous deletion (immediate): Old sound deleted from mesh.metadata.currentSound
2. Async creation (~10-50ms): New sound created via CreateSoundAsync()
3. Async attachment (~10-50ms): New sound attached to mesh.metadata.currentSound

This means there's a brief window where currentSound is undefined between deletion and attachment.

Diagnostic Test Results

Created tests/sound-replacement-diagnostic.test.js to investigate:

Test: Playing second sound (loop=false)
Before second playSound:              currentSound = "test.mp3"
Immediately after playSound call:     currentSound = undefined       ← Old sound deleted
After 50ms (promise not awaited):     currentSound = "test2.mp3"     ← New sound attached
After awaiting promise:               currentSound = undefined       ← Sound ended, auto-removed!

Impact

This behavior makes it difficult to:

1. Test sound replacement - Cannot await the promise with loop: false and check currentSound
2. Chain sound operations - Promise timing differs based on loop parameter
3. Understand API contract - Not documented that promise resolution depends on loop

Questions

1. Is this the intended behavior, or should promises always resolve when the sound is attached?
2. Should there be a separate method/option to wait for sound completion vs. sound attachment?
3. Should the auto-cleanup of currentSound be optional or configurable?

---

Reproduction

Issue 1 - Web Audio Error

npm run test:api sound

Look for console error:

[browser:pageerror] Error [TypeError]: Failed to execute 'linearRampToValueAtTime'
on 'AudioParam': The provided double value is non-finite.

Issue 2 - Sound Replacement

Created diagnostic test suite to investigate the behavior:

• tests/sound-replacement-diagnostic.test.js (3 diagnostic tests)
• Shows timing of deletion, attachment, and auto-cleanup
• Demonstrates promise resolution differences between loop: true and loop: false

---

Files Involved

Source Code

• api/sound.js:233-250 - playNotes() loop that accesses durations[i]
• api/sound.js:267-324 - playMidiNote() with linearRampToValueAtTime call
• api/sound.js:52-117 - playSound() promise resolution logic

Tests

• tests/sound.test.js:212-220 - Test that triggers Issue 1
• tests/sound-integration.test.js - Integration tests affected by Issue 2
• tests/sound-replacement-diagnostic.test.js - Diagnostic tests revealing behavior

---

API Documentation Impact

The investigation revealed important API behavior that should be documented:

1. playSound() promise resolution depends on loop parameter
   • loop: true → resolves when sound is attached to mesh
   • loop: false → resolves when sound finishes playing
2. Sound auto-cleanup - loop: false sounds remove themselves from mesh.metadata.currentSound when they end
3. Replacement timing - synchronous deletion + async creation pattern (brief undefined window)
4. playNotes() array handling - behavior when notes.length > durations.length is undefined

---

Related Issues

None - These were discovered during sound test integration from https://github.com/commercetest/babylonjs-sound-testing

---

Labels

• bug - Actual bugs to fix
• documentation - Needs API documentation updates
• sound - Sound system
• question - Needs decision on intended behavior (Issue 2)