Styjun

From my tests, and from searching to find out more about the problem, my best guess is that css animations may be using a different physical clock from the one used to stream audio. If so perhaps the answer to this is that it can't be done, but am asking in case I am missing anything.

It is for my online metronome here.

It is able to plays notes reasonably accurately in response to an event listener for the css animationiteration event. The eventlistener is set up using e.g.
x.addEventListener("animationstart",playSoundBall2);

See here.

However if I try to synchronize the bounce with the sample precise timing of the AudioContext method that's when I run into problems.

What I do is to use the first css callback just to record the audio context time for the css elapsed time of 0. Then I play the notes using the likes of:

oscillator.start(desired_start_time);

You can try it out with the option on the page: "Schedule notes in advance for sample-precise timing of the audio" on the page here.

You can check how much it drifts by switching on "Add debug info to extra info" on the same page.

On my development machine it works fine with Firefox. But on Edge and Chrome it drifts away from the bounce. And not in a steady way. Can be fine for several minutes and then the bounce starts to slow down relative to the audio stream.

It is not a problem with browser activity - if I move the browser around and try to interrupt the animation the worst that happens is that it may drop notes and if the browser isn't active it is liable to drop notes. But the ones it plays are exactly in time.

My best guess so far, is that it might be that the browser is using the system time, while the audiocontext play method is scheduling it at a precise point in a continuous audio stream. Those may well be using different hardware clocks, from online searches for the problem.

Firefox may for some reason be using the same hardware clock, maybe just on my development machine.

If this is correct, it rather looks as if there is no way to guarantee to precisely synchronize html audio played using AudioContext with css animations.

If that is so I would also think you probably can't guarantee to synchronize it with any javascript animations as it would depend on which clocks the browser uses for the animations, and how that relates to whatever clock is used for streaming audio.

But can this really be the case? What do animators do who need to synchronize sound with animations for long periods of time? Or do they only ever synchronize them for a few minutes at a time?

I wouldn't have noticed if it weren't that the metronome naturally is used for long periods at a time. It can get so bad that the click is several seconds out from the bounce after just two or three minutes.

At other times - well while writing this I've had the metronome going for ten minutes in my Windows 10 app and it has drifted, but only by 20-30 ms relative to the bounce. So it is very irregular, so you can't hope to solve this by adding in some fixed speed up or slow down to get them in time with each other.

I am writing this just in case there is a way to do this in javascript, anything I'm missing. I'm also interested to know if it makes any difference if I use other methods of animation. I can't see how one could use the audio context clock directly for animation as you can only schedule notes in the audio stream, can't schedule a callback at a particular exact time in the future according to the audio stream.