When getting into the groove and typing fast, I can move the controllers fast enough that they miss the key colliders.
One way to fix it would be to have an actual physics object that follows the end of the sticks using forces. If you have continuous physics enabled on that object it should hit the keys no matter how fast the user swings.
Another solution would be to increase the size of the keyboard collider's depth. But that would introduce other errors.