Comment # 19 on bug 1205640 from Antonio Larrosa

That's very interesting and makes me think it might be a case of the kernel
powering the usb device down due to power management. You can test this by
running "sudo powertop" (install powertop if you don't have it in your system).

Inside powertop, press TAB a few times to go to the "Tunables" tab. Search
there for your audio devices (using arrow up and down) . It will probably be in
"Good" state (please, confirm it). "Good" means the device will have enabled
power management (in other words, it will be autosuspended when not in use).
Some devices are not compatible with this and will misbehave, which might be
the case for your audio device. Press space to change it to "Bad" (disable
power management). Press ESC to leave powertop.

Then check if you can reproduce the problem. Note that a system reboot will
restore the original state, so you'll have to repeat the process to turn
autosuspend off again.

Btw, it's safe to turn all devices to Good/Bad if you want to make tests, but
beware that sometimes keyboards/mice stop working if you enable autosuspend on
them.