BEYOND - Important audio delay (not in sync with the audio graph)

[HKoncept]

Hi HK,

Please get the show file and audio files into our hands. If we have the files and can see the problem you're having with our own eyes and on our own computers, I am sure we can solve whatever problem you are having.

By the way, we don't convert anything to anything. 100% of the codecs used in BEYOND were written by us -- for the very reason that we can have absolute control over the quality of the results client see. What we never wanted to deal with is that one client had a problem that wound up being due to the Windows codecs and such.

So if you have particular files or a particular show scenario that isn't working, we're *really* interested in seeing it, understanding it, and then solving it.

Bill

I really appreciate your feedback. I will take time to show you the latency in a video and include a "how to setup" file to reproduce issue...

[The_Doctor]

I'm mixing with VirtualDJ and Serato, video production with SonyVegas (ASIO soundcard) I can get as low as 2ms of latency for ALL my MP3, even VBR. So I can confirm that some professional don't have latency issue.

I'm a software developer too and I understand that converting to .WAV could represents an acceptable workaround, thanks.

I agree that VBR should not be an issue, except when randomly seeking with an attempt at precision for timing or location, where it can fail. So long as you start from the beginning, VBR or constant bit rate methods will work equally well. Given how fast a disk access, frame decode to PCM, filling a buffer, can occur on a computer, then preparing a stereo stream is easy. On a 1GHz machine many complex operations can occur for every sample of output data without a glitch.

But IS there a glitch? You said the audio was getting progressively later, de-syncing by several seconds. I'm certain you know this but as no-one mentioned it I have to ask: Is there some bottleneck of demand on the CPU? Some spike of demand a process monitor might reveal? If the sound is continuous and not playing at the wrong sample rate, then the de-sync is somewhere else, maybe in the waveform rendering as you're syncing with that.

Note that if WAV works, it may not prove that VBR was failing, it might just be reducing the kinds of CPU spiking common to many decompression methods, as they won't be running during WAV playback.

It is also possible that some driver conflict might affect timing of hardware access, so maybe look into that too.

[dnar]

Its a known issue with vbr mp3.

[The_Doctor]

Its a known issue with vbr mp3.

Why? Once you straighten all the variable density in a tangled string, and run that string at constant speed from start to end, it might as well be straight to start with. Like I said, seeking accurately is one thing, but smooth playback from start to finish is another. The only way I can make immediate sense of this is if the MP3 was rendered to PCM for the sync-by-waveform method described in the first post, then later the references are attempting to relate to the undecoded MP3 during playback. Anything else VBR-related should not be a problem so long as playback is sequential from start, and the decoding does not demand so much time that the audio breaks up in glitches.

[The_Doctor]

VBR is mainly a problem when the file is used to create show. If you scroll to a random position within the audio file and begin playing from that point the time that comes from the media player is not necessarily the same time that would be reported if you played the song from the beginning. When people create shows they jump all over the place within the audio files so when they play it back all at once the timing is way off. As far as whether or not there is a timing drift when playing back the song from beginning to end goes, I don't believe so because I did a lot of timing checks when I was troubleshooting this issue. Constant rate MP3 and WMA files don't seem to have this problem, though. Using WAV files might be overkill but it certainly is a good way to avoid the problem.

Always jump randomly? The guy who posted first never mentioned it. His post left the impression that a single stream of events got desynced as he sat there watching it. If there is no alternative to seeking locations on demand during playback it limits the formats a lot, as many compressible formats have trouble with it, (and MPC, Musepack does so even with constant rate, and some others might too). Even MP3 might desync a bit in CBR purely because the frame size is a lot bigger than a few samples!

Look at it another way: So long as it CAN seek reliably enough (unlike MPC which will try, and most times fail and restart from the top), then it really shouldn't matter if the playback software can grab a few frames earlier and decode fast enough to locate the right point in PCM data. Bill says Pangolin make their own decoders so I have no idea whether they do this or not. Maybe the best way is to use a compressible format that allows markers like those in Sound Forge to work. WAVpack can do this, I imagine others can too. Also, as WAVpack has an excellent 'hybrid lossy' mode that puts its lossless data across two files, the smaller one sounds excellent, is CBR (I think the user can choose this or VBR), compresses to similar to 256kbps MP3 or smaller with good sound, and the smaller, playable file holds the Sound Forge markers, then this might be the best format to use. It's also very efficient to decompress too, leaving CPU time for the other stuff.

EDIT:
I'll argue that never having created a show, I still think that method may be wrong. If I were setting a show to music, I wouldn't think like a movie editor, cutting up sound to fit the action. This is choreography, the music is the foundation, and everything else is made to fit that. Is there a reason why a laser show cannot be made this way? After all, if it were done to a live show it would have to fit with that. The commands sent to the galvos and colour intensity modulation are about the most responsive and reactive things in the show, so it seems wrong to me to lay them down like foundations and then try to choreograph the most clumsy dancer in the room.

[White-Light]

lots of audio packages have difficulty with mp3s that have been encoded with VBR (variable bit rate). try converting your mp3 to an aif/wave or other audio format.

+1.

I understand its not just Pangolin products that have this issue it's common place.

WAV is the best way around it.

I'm mixing with VirtualDJ and Serato, video production with SonyVegas (ASIO soundcard) I can get as low as 2ms of latency for ALL my MP3, even VBR. So I can confirm that some professional don't have latency issue.

But when you're mixing you're only taking 10 or so seconds when you're actually doing the mixing no? The time slip occurs over a larger time scale so that might be why you're not seeing it when DJ'ing.

[The_Doctor]

I am not sure if you are getting the picture or not. Here is a better example. Let's say there is some cymbol in the song at time 1:00. If you play the VBR file from time 0:00 each time, the cymbal will hit at 1:00 every time. But, if you start the song at 0:50, the cymbol might occur at 0:59.4 or 0:60.3 or who knows when. It's enough to make the show out of sync. When designing a show people constantly reply sections over and over to make sure there synchronization is correct. But, if the time isn't being reported correctly by the media player then after they are done and play it all, it doesn't line up. Why it is that way I don't know but I know it to be true based on several people reporting it to me.

I get the picture perfectly. Always did (I've known about VBR and frame boundaries in compressed files for many years now). Trying to make me understand what I already understand, by repeating it at me isn't going to resolve anything. The question is: why do people insist on forcing the music to stop and start? If you're setting a show to music, you start the music, then sync everything else to that. Problem solved. So long as the music plays at correct speed without a glitch, all you have to do is trust it. If it sounds ok, it IS ok. So people should trust that, and change the timing of events over which they DO have proper control.

If there is a need to play back a compressed VBR file, render it in FULL for waveform-based timing cues, that way they will be accurate even when playing back from the original VBR file again later. All you have to do is start them at same time. Assuming that a show's visual controls are either clocked as accurately as the audio hardware, or are cued live by a performer with good timing, it will work fine.

[White-Light]

The question is: why do people insist on forcing the music to stop and start?

Because when you're trying to put a show together its very difficult to drop a marker on the exact note you want because of the speed with which the cursor moves, or if zoomed out to slow the cursor down because you simply can't see the peaks in the waveform as a result, so the logical way to do it is to expand the time line so you're zooming in, then set playback markers on the section you're working on and then play back multiple times and move the marker etc a little each time until you have the marker and cue in exactly the right position that corresponds to the beat or highlight in the music that you want to emphasise.

To do that with a straight play through would be:

1: Nigh on impossible with regards to alignment because its simply not possible to drop accurately when zoomed out or react fast enough when zoomed in

2: Very, very time consuming as you'd have to allow the tune to play all the way through for every key / cue (bearing in mind a show might contain hundreds of keys / cues). OK when placing keys in the 1st few seconds, but when the show is several minutes into the tune, the time taken to place each cue / key rises exponentially if you have to wait over 3 minutes of play through to place every marker etc. and bear in mind that whilst 3 mins would be painful some songs run to nearly 10 minutes!

[masterpj]

Sorry for the late response .
I was at the LEM.
Anyhow I have virtual DJ.
And yes it cuts off the silence but that doesn't mean it cuts it so it would be in sync with a show, some shows want that bit of silence for various purposes.

The thing why mp3 delays can't be corrected easly is because it relies on codecs which can vary per computer and that causes slightly different delays.
Cutting off can solve some problems but also make new ones, because like again: cutting off desired silence is also not nice.

Work with wav, it saves you so much trouble.

[The_Doctor]

Because when you're trying to put a show together its very difficult to drop a marker on the exact note you want because of the speed with which the cursor moves, or if zoomed out to slow the cursor down because you simply can't see the peaks in the waveform as a result, so the logical way to do it is to expand the time line so you're zooming in, then set playback markers on the section you're working on and then play back multiple times and move the marker etc a little each time until you have the marker and cue in exactly the right position that corresponds to the beat or highlight in the music that you want to emphasise.

To do that with a straight play through would be:

1: Nigh on impossible with regards to alignment because its simply not possible to drop accurately when zoomed out or react fast enough when zoomed in

2: Very, very time consuming as you'd have to allow the tune to play all the way through for every key / cue (bearing in mind a show might contain hundreds of keys / cues). OK when placing keys in the 1st few seconds, but when the show is several minutes into the tune, the time taken to place each cue / key rises exponentially if you have to wait over 3 minutes of play through to place every marker etc. and bear in mind that whilst 3 mins would be painful some songs run to nearly 10 minutes!

Before I answer you I'll get one thing out of the way: There are times when people want to have no specific foundation, where they want to drop a sound the way they might drop a small group of ILDA frames or some abstract pattern. In this case there is no substitute for PCM data ready in an an audio buffer, or the next best thing, hence fast load of WAV data. I have no argument with that.

The first post did not seem to imply that though, it looked to me like loading some compressed VBR file to render it as PCM wave data on the assumption that playing it back later the timing would be accurate. If it is not, then perhaps like MP3DirectCut which operates on UNcompressed MP3 files, the wave data is limited in accuracy for timing (basically, frame boundaries and not samples). If these cues he missed were merely an accuracy problem for dropping markers, then I wouldn't expect the de-syncing to accumulate over time, it would vary erratically around some small value right from the start.

Now, I mentioned Sound Forge for a specific reason: I use it, so I'm familiar with some of how it works by long habit. When placing a marker the first thing to do to correctly set a point is to find it, no marker gets dropped till the last move. I've tried dropping them on the fly, and it's a terrible method. What you do is zoom to some fairly close view with up/down arrow keys (fastest way to change zoom scale), and start playback live from as far back as you need to, then hit ENTER, not Space, so next time Space starts it playing, it starts where it stopped, not where it played from last time. If you overshot, hit PgUp to go back, alternating Space to start/stop until you're close. Zoom in to fine tune timing of start point if needed. This can be a very fast series of moves with practise. The moment your Space button starts the beat with precision, drop the marker and move on in realtime listening for the next instant to hit Enter. Hit Space the first time or two just to be sure, it's quicker to adapt your own responses than to fumble it and move badly placed markers later.

That might not sound easy, but it IS, if you practise. I not only found every audible glitch in 40 minutes of vinyl recording once, plus several shorter bits, using this method, I even drew out the small spikes by hand because the usual snap/crackle removers were making a pig's ear of trying to automate it! There is no way I could have handled that if the method of finding stuff wasn't as easy as I said it was.

In short, play it by ear and hand, like any other musical instrument. If your audio editor isn't good enough to do this, it isn't good enough. I suspect many people underestimate the power of the keyboard shortcuts in Sound Forge.

EDIT: Where did you get the idea that you'd have to play from the start to find every cue? You don't. If a marker ends up correctly placed relative to start, then a similar marker will be correct relative to the first, as well as the start, and so it goes... All that matters is that the PCM wave view must be decoded data, not an approximation based on frame amplitude like MP3DirectCut uses, or some other inaccurate means. Once you have real accurate timings (for stuff seen by eyes with vision's slow response compared to ears, it doesn't even have to be sample accurate anyway, just good enough to convince your brain in realtime), a show set to music will play in time throughout, so long as sound and light start together and are clocked correctly in realtime. There's no need to run through the decoded audio dropping markers more than once, you just take each step carefully by ear and hand before you drop a marker. If the time isn't worth taking, then you need to ask how much the show is really worth doing.