I need some advice from experienced coder regarding A/V hdd streaming please

[ImmortalA1000]

have some spare time, although writing a new game too so not that much free time for this, but I was discussing playback of movie files on an Atari STE with someone and had a few ideas of a better way to do this on Amiga. Feedback appreciated.

Fix the resolution of the movie for my first attempt at doing this to simplify things. So for example I will take a 320x160 32 colour resolution as a start point.

So my idea now is to create a single file. This file will be composed of uncompressed data so the images for the frames and the associated sound for each grouping of frames are at fixed regular intervals. This means that if I create a file and this file goes as follows.....

| Frame1 | Frame2 | Frame3 | Frame4 | Frame5 | 0.2sec Sound Frames1-5 | Frame6 | ...

So as all the frames are uncompressed the start of each 5 frames of video and it's associated 1/5th of a second of uncompressed 8 bit sampled sound will be at fixed points and can easily be loaded in segments of repeating equal sizes for each 20% of a seconds worth of movie.

So here is the key, the bit that sort of struck me after thinking about removing seek time delays for lots of tiny files and distinct separate audio/video files. What if I do the following....

1. Load the first 180kb (5 x 32kb frames and 20kb of sample data) of the file and begin to transfer it via DMA starting at one of two predefined chip memory address locations called start_addr1.

2. Load the next 180kb of the file into the second predefined address in chip memory called start_addr2.

REPEAT

3. IF audio sample not currently playing on left channel 2 and right channel 2 THEN......
Begin to play the 20kb of audio data stored at start and end address locations
(start_addr1 + 160kb) - (start_addr1 + 180kb) using left channel 2 and right channel 2.
ELSE go back to 3

4 Begin a loop to set beginning of screen memory to 5 values stepped through
(start_addr1) + 0
WAIT VSYNC x2
(start_addr1) + 32
WAIT VSYNC x2
(start_addr1) + 64
WAIT VSYNC x2
(start_addr1) + 96
WAIT VSYNC x2
(start_addr1) + 128

5 Start reading in next 180kb segment of file into chip ram at start_addr1 via DMA

6. IF audio sample finished playing on left channel 1 and right channel 1 THEN......
Begin to play the 20kb of audio data stored at start and end address locations
(start_addr2 + 160kb) - (start_addr2 + 180kb) using left channel 2 and right channel 2.
ELSE go back to 6

7 Begin a loop to set beginning of screen memory to 5 values stepped through
(start_addr2) + 0
WAIT VSYNC x2
(start_addr2) + 32
WAIT VSYNC x2
(start_addr2) + 64
WAIT VSYNC x2
(start_addr2) + 96
WAIT VSYNC x2
(start_addr2) + 128

8 Start reading in next 180kb segment of file into chip ram at start_addr2 via DMA

UNTIL end of file.

Worried audio would crackle due to gaps when no audio is being played or even overlap because I am locked to a VSYNC fiftieths of a second and that only gives the code as much time effectively as two 2/50th of a second for

a. setting the 5th,10th,15th,20th...... frames to display by writing a new screen memory location
b. initiate DMA transfer operation for reading in next segment of 180kb of data.
c. polling the hardware to start playing next 0.2 second segment of audio
d. setting screen memory to display frames 6,11,16,21....etc

It's not accurate enough for synchronizing each audio segment perfectly is it OR will it be OK as long as I can get to part c. within 2/50ths of a second to check (and start if it has just finished playing) the next audio segment before continuing to display the first frame of next five segments.

What do you think?

[Kalms]

Having a datastream with interleaved audio/video frames is a good idea.

Rather than triggering the audio to play every 0.2 seconds, you could have a cyclic audio buffer (size 0.4 seconds or larger). You let the audio hardware play continuously from that buffer, and you copy in a new chunk (0.2 seconds worth of audio data) when you know that that part of the buffer is not currently being played. As long as your calculations are correct, you will not get any crackle/gaps. This is a convenient model if the source audio originally is available in fastmem.
If it so happens that audio naturally lands in chipmem, then you could treat each 0.2 second block as a sample, and use the audio interrupts to trigger block after block.

Steps a,b,c which you describe here will be completed in a couple of microseconds. So synchronization is not dependent on performance of the code, it is purely dependent on if you put the A/V frames in the right order and correctly compensate for the amount of buffering you have in each subsystem.

[ImmortalA1000]

Buffering was a real issue to think about, on the one hand I only have so much chip memory, and the samples and frames need to go into chip memory so they can be loaded and instantly displayed/played instantly with no impact on the bus bandwidth? But the more you buffer the less points of desynchronization are present.

I will probably try it on a 2mb all chipram A600 or something first just to see if it works, and then tweak it to try and get it running well on a 512kb OCS Amiga.

Someone said to me on another forum the way I am storing/reading the data is not how PCs playback even uncompressed AVIs which kinda spooked me because I can't see how else you would playback an uncompressed AVI on a PC today

[Kalms]

By the way -- your suggested scheme has 5 video frames, then 0.2s of audio, then 5 video frames, then 0.2s audio, etc.

Why do you have more than 1 frame of video between each chunk of audio data? You could just as well have 1 frame of video, then 40ms of audio, etc.
Having the audio data in larger blocks will only make a difference if there is a large setup time for the processing of a block (which, in your current example with uncompressed audio, there isn't) or if you don't know exactly how many samples will be played between each frame (which is not the case either in your scenario) or if the context switching between audio processing and video processing is prohibitively large (which it isn't in your example either).

So if you interleave at 25fps then you only need as much buffer as you expect veriance in the HDD read operation.

In the PC case you would set it up like:
1 read ahead thread (delivers from HDD to main memory, either compresed or uncompressed)
1 on-demand video decompression thread (delivers from main memory to textures in video memory, doing decompression if necessary); this job triggers the read-ahead thread
1 on-demand audio decompression thread (delivers from main memory to audio buffers); this job triggers the read-ahead thread
1 output synchronization thread, swaps video frames at the right speed and tracks progress through the audio buffer; this job triggers the decompression threads

... and there you go, a design which is reasonably robust and handles potentially compressed datastreams. Each thread is supposed to have a bit of buffering in their outputs, so temporary variances (HDD performance, CPU availability) doesn't interfere with video+audio playback. Video + audio codecs are reasonably easy to handle via a plugin architecture as well.