Use for PIPE: HippoPlayer and aget

koobo · 29 September 2023, 06:55

Use for PIPE: HippoPlayer and aget

Text by me & @patrik, early 2023.

I thought I'd write something about this since it's the first time I've used the queue-handler provided PIPE: for anything and it was kind of an interesting AmigaOS learning experience. In fact I didn't know what the whole PIPE business was, even though I remember noticing the file "queue-handler" in the directory "l" back in the days.

So, in the shell you can do:

echo "hello" >PIPE:

...and then get it back with:

type PIPE:

Whee! Or even better, you can name your pipe:

echo "hi again" >PIPE:SuperPipeline

type PIPE:SuperPipeline

If the pipe is empty, trying to read it will block until data is available.

Googling about pipe usage results in some disconcerting information, such as: "Up to OS 3.1, the queue-handler has some bugs which don't allow a really foolproof transmission of data.". There are also some alternative implementations which purportedly fix bugs and generally work much better. During this experiment kickstart 3.0 and 3.2 were used for testing, no 3rd party queue-handlers were used.

HippoPlayer uses DOS file handles for reading data (obviously), so replacing the handle with something that doesn't point to the local filesystem would probably work just fine, right? Let's grab the fine UHCTools package and the aget tool, which is able to do HTTP-transfers wonderfully. Then, redirect aget output into a named pipe and on the other end, make Hippo read it. We get an interprocess communication network data streamer thingy without having to implement a HTTP stack or internet connectivity, since the UHCTeam already did all of that. What could possibly go wrong?

After a not insignificant amount of programming, testing and fine tuning, eventually it started to work. There were some gotchas on the way. Here HippoPlayer is the pipe consumer, and aget the pipe producer. Hippo passes a url to an mpeg audio stream to aget and asks aget to write the data into a named pipe. Hippo will then start consuming the data from the pipe and feed it to mpega.library for decoding, and after that to Paula or AHI. Radio streams work fine, as well as MP3 files without needing to download them first. Nice!

When the pipe runs out of data, ie. all data has been transferred, the consumer end will see an end-of-file, similarly as with local files.

If the consumer wants to stop reading the pipe, that is, the user presses the stop button, the process is more complicated. First, CTRL+C needs to be sent to the producer, aget in this case. Then the pipe must be flushed by the consumer. Aget will be able to complete the current write because the pipe is now empty and then notice the incoming break signal, and exit cleanly. Stream playback completed.

It’s possible to configure the pipe buffers. We tried a lot of buffer combinations, different sized pipe buffers and aget buffers. Some radio stations were difficult as they provided data in throttled manner, the bytes came barely fast enough to sustain real time playback. In these cases it also helped to just wait a few seconds before starting to consume the pipe data to allow some data to be buffered. Hippo also had a small rounding error in the MP3 downsampling part which contributed to data being consumed ever so slightly too fast.

Another thing to take into account is that pipe file handles can only be read sequentially, so the seek operation won’t work. We did not notice any problems with the pipe usage in terms of data loss or reliability.

It’s notable that there are differences in how the pipe device works in different kickstart versions, details below.

CTRL-C

Example under 3.2.1 where the CTRL-C behaviour can be observed:

Code:

aget shell - starting aget:
7.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout 
Buffer size: 128kB
Connected to se.aminet.net:80
Sent GET request!
Server response: 200 OK
Receiving file: 0% - [                                ] - 100% (6.80MB)

In second shell I use the uhc typeline command which just read lines, til it finds the requested line number, then closes and exits:

Code:

8.Ram Disk:> uhc:c/typeline PIPE:agetout 1 
| Aminet index, created on 27-Sep-2023
8.Ram Disk:>

…leads to this final aget shell:

Code:

7.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout 
Buffer size: 128kB
Connected to se.aminet.net:80
Sent GET request!
Server response: 200 OK
Receiving file: 0% - [                                ] - 100% (6.80MB)
Ctrl-C signal caught, ending transfer!
160.79kB received in 13.46s - 11.94kB/s
7.Ram Disk:>

Example under 3.1:

Aget shell - starting aget:

Code:

11.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout
Buffer size: 128kB
Connected to se.aminet.net:80
Sent GET request!
Server response: 200 OK
Receiving file: 0% - [                                ] - 100% (6.80MB)

In the second shell I use the uhc typeline command as before, but as this just consumes the first two rows of the pipe (row 0 and 1) before closing it and the 3.1 queue-handler does not send ctrl-c and consume the last write, aget is still hanged and I need to manually do that:

Code:

10.Ram Disk:> uhc:c/typeline PIPE:agetout 1
| Aminet index, created on 27-Sep-2023
10.Ram Disk:> break `status COMMAND=aget`
10.Ram Disk:> type PIPE:agetout >NIL:
10.Ram Disk:>

Which leads to the wanted and final aget shell result where it has exited:

Code:

11.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout
Buffer size: 128kB
Connected to se.aminet.net:80
Sent GET request!
Server response: 200 OK
Receiving file: 0% - [                                ] - 100% (6.80MB)
Ctrl-C signal caught, ending transfer!
160.79kB received in 294.07s - 559B/s
11.Ram Disk:>

Thomas Richter · 29 September 2023, 07:14

The Ctrl-C behaivor was added in 3.2 to address the case of the reading end of the pipe terminating, thus leaving the writing end with a "broken pipe" left. Unixoid systems have their own signal for this case, terminating the writer. AmigaOs does not, so it instead sends a ^C to the writer and ignores all further incoming write requests.

This is particularly useful if you use pipes along with "more" as a pager:

list all | more

will abort the "list" if "more" is terminated to list the entire file system.

Note that you can also access reading and writing end of the pipe directly by name in a single line:

copy foo to PIPE: | type PIPE: hex

This "PIPE:" identifies (left) the writing and (right) the reading end of the pipe that is implicitly constructed by the shell for you.

tygre · 29 September 2023, 19:07

Hi Koobo and Thomas!
Very interesting, thanks for sharing!

DisasterIncarna · 29 September 2023, 21:13

great functions added to both aget and HippoPlayer, being able to stream audio direct from the net is 1 of the best most recent features, previously i used AmigaAmp to listen to the retro goodness on Slay Radio, now Hippo can do the same by using your "Radio Stations" option and entering the information contained in 1 of Slay Radio's .M3U files, i use http://relay1.slayradio.org:8000 and it plays just fine.

Perhaps that could be automated by getting Hippo to just outright read those address(s) from an M3U file when we add a NEW mod/file.

patrik · 30 September 2023, 10:50

@DisasterIncarna:

Why not just search for slay radio in the radio stations search?

Or is the question two questions:
- Can hippo be made to to add urls to play to the playlist manually, without searching for them?
- Can hippo be made to play .m3u files?

DisasterIncarna · 30 September 2023, 23:12

yeah probably thinking the load/play m3u option as i typically have a saved list of fave stations including slay/digitally imported fm/etc which is why i rarely use or overlooked the radio station search, which yeah also works fine as it defaults to the 128k stream, no idea how you pick the 56k or 24k streams without doing it manually tho. Just accustomed to using my saved file list is all

either way its 1 of the best new additions i use all the time.

Now that i think about it, maybe i could just slap a load of m3u links into a custom playlist and load that instead?

patrik · 01 October 2023, 12:03

@DisasterIncarna:

Continuing here asking things that are not obvious to me

- Where/how do you download your .m3u file collection?
- What is the actual use for the 56k or 24k streams?

DisasterIncarna · 02 October 2023, 00:27

the only use for different bitrates afaik is just if your using something that cant manage large internet speeds, emulated amiga and im all in on biggest bitrates, using real hardware or some fpga with a slow net capability then im using much, much lower bitrates.

Ive also defaulted to the lower bitrate M3U streams when the better/biggest ones go offline/die/arent available.

Saving an M3U was/is as easy as just rightmousing a stream on a website and saving the m3u, for example: on https://www.slayradio.org/home.php#news looking top/left in the "Listen to slay radio" box, you can just rightmouse the High/Medium/Low links, and each of those M3U's has a list of relays/mirrors for each of the 128k/56k/24k bitstreams.

ive done the same on other websites to get M3U's i need as i mainly used them on a radio station docker app on my NAS and since i had them i just defaulted to using them wherever i am able to.

koobo · 14 October 2023, 15:37

The plumber brothers are back with another pipely adventure!

I got an idea and decided to push CD-quality wav-sample through PIPE:. It was really slow. I mentioned this to @patrik and this caused him to lose two nights of sleep, after which he decided to do some scientific measuring:

PIPE performance

Needed to know queue-handler isolated performance, measured on a super serious A3000 68030@25Mhz workstation.

Code:

8.Ram Disk:> source ? 
DESTFILE/A,NUMWRITES/N,WRITESIZE/N: 
erfordrat argument saknas
8.Ram Disk:> sink ? 
SOURCEFILE/A,NUMREADS/N,READSIZE/N: 
erfordrat argument saknas

Sources 10x1MB writes from allocated buffer and sinks 10x1MB reads to allocated buffer by default.

Code:

3.2.1:
8.Ram Disk:> version full 
Kickstart 47.7, Workbench 47.3 (2021-07-13)
8.Ram Disk:> version full PIPE: 
queue-handler 45.2 (2017-10-15)
8.Ram Disk:> run sink PIPE:test 
[CLI 7]
8.Ram Disk:> uhc:c/time source PIPE:test 
14.431280s

0.69MBytes/sec

Code:

3.1:
11.Ram Disk:> version full
Kickstart 40.68, Workbench 40.42 (1994-02-18)
11.Ram Disk:> version full PIPE:
Could not find version information for 'PIPE:'
11.Ram Disk:> version full L:queue-handler 
Version 37.12 (1991-03-01)
11.Ram Disk:> run sink PIPE:test
[CLI 10]
11.Ram Disk:> uhc:c/time source PIPE:test
99.937655s

0.1MBytes/sec

so old queue-handler is approx 1/7 the speed of the new one

End of scientific section

A CD quality sample bandwidth is about 0.17MBytes/sec, so the old queue-handler can't handle it on the super serious workstation. Luckily the new one can also be used on older OS versions and kickstarts!

Observations anomalous

I was testing a pipeline where a WAV sample was produced into PIPE: and HippoPlayer was consuming it on the other end. I noticed that on my A1200/060 (OS 3.1) the WAV sample was intermittently destroyed and sounded like noise.

The conditions for this happening were:

queue-handler 37.12
Mungwall and Enforcer running
Pipe using default buffer settings

If I changed the pipe buffer settings to

65536/2

the problem went away, or altenatively removed Mungwall+Enforcer from the equation.

These conditions seemed to unearth some of the bugs apparently hiding in the old queue-handler. With queue-handler 45.2 this effect could not be reproduced.

DisasterIncarna · 14 October 2023, 15:51

Thomas Richter · 14 October 2023, 18:49

The V37 and V45 Queue handlers work by entirely different principles, the v45 is completely new. The V37 creates a new process for every PIPE: file you open, and then uses inter-process communication with PutMsg() to send the data from the writing PIPE:-process to the reading PIPE:Process, so inter-process communication goes over four processes: The writer, the writing pipe, the reading pipe, and the reader.

The V45 queue handler only creates a single instance of the PIPE that keeps lists of waiting and ready packets, and directly copies from the writing end to the reading target buffer, avoiding one stage of inter-process communication. That certainly helps to speed things up, but it primarily simplified the design.

The drawback is that you have a sequential bottleneck in the middle, i.e. if multiple processes want to talk to each other over the PIPE:, then a single PIPE process has to manage copying all data around. However, we only have a single CPU core on Amiga anyhow, so it is probably not quite that bad.

patrik · 14 October 2023, 21:03

@Thomas: Very interesting with the details about how the original queue-handler was implemented vs how the current is!

If anyone is interested in reproducing this PIPE: benchmark themselves, these are the components used:
http://megaburken.net/~patrik/SourceSink.lha
http://uhc.a1k.org/time

koobo · 17 October 2023, 19:49

Thanks @Thomas, interesting

I performed another measurement on the miggy (A1200/060, OS 3.1) with the following HippoPlayer-pipeline:

Input is a MIDI file (st2cred.mid, Turrican music, pretty simple for a MIDI tune)
Timidity convert this to WAV and writes to PIPE:hippo/65536/2
HippoPlayer reads PIPE:hippo
16-bit stereo WAV data is converted to 8-bit and fed to Paula
Music is output from the speakers

For a three minute run the average CPU usage:

Queue-handler V37: 37%
Queue-handler V45: 29%

Here about 170 kB/s of data goes through the pipe.

DisasterIncarna · 18 October 2023, 04:51

wonder if someone can abuse the same setup to make an online animation streaming program, even if it was just for tiny iff animations or something.

29 September 2023, 06:55	#1
koobo Registered User Join Date: Sep 2019 Location: Finland Posts: 365	Use for PIPE: HippoPlayer and aget Use for PIPE: HippoPlayer and aget Text by me & @patrik, early 2023. I thought I'd write something about this since it's the first time I've used the queue-handler provided PIPE: for anything and it was kind of an interesting AmigaOS learning experience. In fact I didn't know what the whole PIPE business was, even though I remember noticing the file "queue-handler" in the directory "l" back in the days. So, in the shell you can do: echo "hello" >PIPE: ...and then get it back with: type PIPE: Whee! Or even better, you can name your pipe: echo "hi again" >PIPE:SuperPipeline type PIPE:SuperPipeline If the pipe is empty, trying to read it will block until data is available. Googling about pipe usage results in some disconcerting information, such as: "Up to OS 3.1, the queue-handler has some bugs which don't allow a really foolproof transmission of data.". There are also some alternative implementations which purportedly fix bugs and generally work much better. During this experiment kickstart 3.0 and 3.2 were used for testing, no 3rd party queue-handlers were used. HippoPlayer uses DOS file handles for reading data (obviously), so replacing the handle with something that doesn't point to the local filesystem would probably work just fine, right? Let's grab the fine UHCTools package and the aget tool, which is able to do HTTP-transfers wonderfully. Then, redirect aget output into a named pipe and on the other end, make Hippo read it. We get an interprocess communication network data streamer thingy without having to implement a HTTP stack or internet connectivity, since the UHCTeam already did all of that. What could possibly go wrong? After a not insignificant amount of programming, testing and fine tuning, eventually it started to work. There were some gotchas on the way. Here HippoPlayer is the pipe consumer, and aget the pipe producer. Hippo passes a url to an mpeg audio stream to aget and asks aget to write the data into a named pipe. Hippo will then start consuming the data from the pipe and feed it to mpega.library for decoding, and after that to Paula or AHI. Radio streams work fine, as well as MP3 files without needing to download them first. Nice! When the pipe runs out of data, ie. all data has been transferred, the consumer end will see an end-of-file, similarly as with local files. If the consumer wants to stop reading the pipe, that is, the user presses the stop button, the process is more complicated. First, CTRL+C needs to be sent to the producer, aget in this case. Then the pipe must be flushed by the consumer. Aget will be able to complete the current write because the pipe is now empty and then notice the incoming break signal, and exit cleanly. Stream playback completed. It’s possible to configure the pipe buffers. We tried a lot of buffer combinations, different sized pipe buffers and aget buffers. Some radio stations were difficult as they provided data in throttled manner, the bytes came barely fast enough to sustain real time playback. In these cases it also helped to just wait a few seconds before starting to consume the pipe data to allow some data to be buffered. Hippo also had a small rounding error in the MP3 downsampling part which contributed to data being consumed ever so slightly too fast. Another thing to take into account is that pipe file handles can only be read sequentially, so the seek operation won’t work. We did not notice any problems with the pipe usage in terms of data loss or reliability. It’s notable that there are differences in how the pipe device works in different kickstart versions, details below. CTRL-C Example under 3.2.1 where the CTRL-C behaviour can be observed: Code: aget shell - starting aget: 7.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout Buffer size: 128kB Connected to se.aminet.net:80 Sent GET request! Server response: 200 OK Receiving file: 0% - [ ] - 100% (6.80MB) In second shell I use the uhc typeline command which just read lines, til it finds the requested line number, then closes and exits: Code: 8.Ram Disk:> uhc:c/typeline PIPE:agetout 1 \| Aminet index, created on 27-Sep-2023 8.Ram Disk:> …leads to this final aget shell: Code: 7.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout Buffer size: 128kB Connected to se.aminet.net:80 Sent GET request! Server response: 200 OK Receiving file: 0% - [ ] - 100% (6.80MB) Ctrl-C signal caught, ending transfer! 160.79kB received in 13.46s - 11.94kB/s 7.Ram Disk:> Example under 3.1: Aget shell - starting aget: Code: 11.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout Buffer size: 128kB Connected to se.aminet.net:80 Sent GET request! Server response: 200 OK Receiving file: 0% - [ ] - 100% (6.80MB) In the second shell I use the uhc typeline command as before, but as this just consumes the first two rows of the pipe (row 0 and 1) before closing it and the 3.1 queue-handler does not send ctrl-c and consume the last write, aget is still hanged and I need to manually do that: Code: 10.Ram Disk:> uhc:c/typeline PIPE:agetout 1 \| Aminet index, created on 27-Sep-2023 10.Ram Disk:> break `status COMMAND=aget` 10.Ram Disk:> type PIPE:agetout >NIL: 10.Ram Disk:> Which leads to the wanted and final aget shell result where it has exited: Code: 11.Ram Disk:> aget se.aminet.net/INDEX PIPE:agetout Buffer size: 128kB Connected to se.aminet.net:80 Sent GET request! Server response: 200 OK Receiving file: 0% - [ ] - 100% (6.80MB) Ctrl-C signal caught, ending transfer! 160.79kB received in 294.07s - 559B/s 11.Ram Disk:>

29 September 2023, 21:13	#4
DisasterIncarna Registered User Join Date: Oct 2021 Location: England Posts: 1,202	great functions added to both aget and HippoPlayer, being able to stream audio direct from the net is 1 of the best most recent features, previously i used AmigaAmp to listen to the retro goodness on Slay Radio, now Hippo can do the same by using your "Radio Stations" option and entering the information contained in 1 of Slay Radio's .M3U files, i use http://relay1.slayradio.org:8000 and it plays just fine. Perhaps that could be automated by getting Hippo to just outright read those address(s) from an M3U file when we add a NEW mod/file. Attached Thumbnails Last edited by DisasterIncarna; 29 September 2023 at 21:21.

30 September 2023, 10:50	#5
patrik Registered User Join Date: Jan 2005 Location: Umeå Age: 43 Posts: 926	@DisasterIncarna: Why not just search for slay radio in the radio stations search? Or is the question two questions: - Can hippo be made to to add urls to play to the playlist manually, without searching for them? - Can hippo be made to play .m3u files? Last edited by patrik; 30 September 2023 at 10:59.

30 September 2023, 23:12	#6
DisasterIncarna Registered User Join Date: Oct 2021 Location: England Posts: 1,202	yeah probably thinking the load/play m3u option as i typically have a saved list of fave stations including slay/digitally imported fm/etc which is why i rarely use or overlooked the radio station search, which yeah also works fine as it defaults to the 128k stream, no idea how you pick the 56k or 24k streams without doing it manually tho. Just accustomed to using my saved file list is all either way its 1 of the best new additions i use all the time. Now that i think about it, maybe i could just slap a load of m3u links into a custom playlist and load that instead? Last edited by DisasterIncarna; 30 September 2023 at 23:20.

14 October 2023, 15:37	#9
koobo Registered User Join Date: Sep 2019 Location: Finland Posts: 365	The plumber brothers are back with another pipely adventure! I got an idea and decided to push CD-quality wav-sample through PIPE:. It was really slow. I mentioned this to @patrik and this caused him to lose two nights of sleep, after which he decided to do some scientific measuring: PIPE performance Needed to know queue-handler isolated performance, measured on a super serious A3000 68030@25Mhz workstation. Code: 8.Ram Disk:> source ? DESTFILE/A,NUMWRITES/N,WRITESIZE/N: erfordrat argument saknas 8.Ram Disk:> sink ? SOURCEFILE/A,NUMREADS/N,READSIZE/N: erfordrat argument saknas Sources 10x1MB writes from allocated buffer and sinks 10x1MB reads to allocated buffer by default. Code: 3.2.1: 8.Ram Disk:> version full Kickstart 47.7, Workbench 47.3 (2021-07-13) 8.Ram Disk:> version full PIPE: queue-handler 45.2 (2017-10-15) 8.Ram Disk:> run sink PIPE:test [CLI 7] 8.Ram Disk:> uhc:c/time source PIPE:test 14.431280s 0.69MBytes/sec Code: 3.1: 11.Ram Disk:> version full Kickstart 40.68, Workbench 40.42 (1994-02-18) 11.Ram Disk:> version full PIPE: Could not find version information for 'PIPE:' 11.Ram Disk:> version full L:queue-handler Version 37.12 (1991-03-01) 11.Ram Disk:> run sink PIPE:test [CLI 10] 11.Ram Disk:> uhc:c/time source PIPE:test 99.937655s 0.1MBytes/sec so old queue-handler is approx 1/7 the speed of the new one End of scientific section A CD quality sample bandwidth is about 0.17MBytes/sec, so the old queue-handler can't handle it on the super serious workstation. Luckily the new one can also be used on older OS versions and kickstarts! Observations anomalous I was testing a pipeline where a WAV sample was produced into PIPE: and HippoPlayer was consuming it on the other end. I noticed that on my A1200/060 (OS 3.1) the WAV sample was intermittently destroyed and sounded like noise. The conditions for this happening were: queue-handler 37.12 Mungwall and Enforcer running Pipe using default buffer settings If I changed the pipe buffer settings to 65536/2 the problem went away, or altenatively removed Mungwall+Enforcer from the equation. These conditions seemed to unearth some of the bugs apparently hiding in the old queue-handler. With queue-handler 45.2 this effect could not be reproduced.

29 September 2023, 07:14	#2
Thomas Richter Registered User Join Date: Jan 2019 Location: Germany Posts: 3,247	The Ctrl-C behaivor was added in 3.2 to address the case of the reading end of the pipe terminating, thus leaving the writing end with a "broken pipe" left. Unixoid systems have their own signal for this case, terminating the writer. AmigaOs does not, so it instead sends a ^C to the writer and ignores all further incoming write requests. This is particularly useful if you use pipes along with "more" as a pager: list all \| more will abort the "list" if "more" is terminated to list the entire file system. Note that you can also access reading and writing end of the pipe directly by name in a single line: copy foo to PIPE: \| type PIPE: hex This "PIPE:" identifies (left) the writing and (right) the reading end of the pipe that is implicitly constructed by the shell for you.

29 September 2023, 19:07	#3
tygre Returning fan! Join Date: Jan 2011 Location: Montréal, QC, Canada Posts: 1,434	Hi Koobo and Thomas! Very interesting, thanks for sharing!

01 October 2023, 12:03	#7
patrik Registered User Join Date: Jan 2005 Location: Umeå Age: 43 Posts: 926	@DisasterIncarna: Continuing here asking things that are not obvious to me - Where/how do you download your .m3u file collection? - What is the actual use for the 56k or 24k streams?

02 October 2023, 00:27	#8
DisasterIncarna Registered User Join Date: Oct 2021 Location: England Posts: 1,202	the only use for different bitrates afaik is just if your using something that cant manage large internet speeds, emulated amiga and im all in on biggest bitrates, using real hardware or some fpga with a slow net capability then im using much, much lower bitrates. Ive also defaulted to the lower bitrate M3U streams when the better/biggest ones go offline/die/arent available. Saving an M3U was/is as easy as just rightmousing a stream on a website and saving the m3u, for example: on https://www.slayradio.org/home.php#news looking top/left in the "Listen to slay radio" box, you can just rightmouse the High/Medium/Low links, and each of those M3U's has a list of relays/mirrors for each of the 128k/56k/24k bitstreams. ive done the same on other websites to get M3U's i need as i mainly used them on a radio station docker app on my NAS and since i had them i just defaulted to using them wherever i am able to.

14 October 2023, 15:51	#10
DisasterIncarna Registered User Join Date: Oct 2021 Location: England Posts: 1,202

14 October 2023, 18:49	#11
Thomas Richter Registered User Join Date: Jan 2019 Location: Germany Posts: 3,247	The V37 and V45 Queue handlers work by entirely different principles, the v45 is completely new. The V37 creates a new process for every PIPE: file you open, and then uses inter-process communication with PutMsg() to send the data from the writing PIPE:-process to the reading PIPE:Process, so inter-process communication goes over four processes: The writer, the writing pipe, the reading pipe, and the reader. The V45 queue handler only creates a single instance of the PIPE that keeps lists of waiting and ready packets, and directly copies from the writing end to the reading target buffer, avoiding one stage of inter-process communication. That certainly helps to speed things up, but it primarily simplified the design. The drawback is that you have a sequential bottleneck in the middle, i.e. if multiple processes want to talk to each other over the PIPE:, then a single PIPE process has to manage copying all data around. However, we only have a single CPU core on Amiga anyhow, so it is probably not quite that bad.

14 October 2023, 21:03	#12
patrik Registered User Join Date: Jan 2005 Location: Umeå Age: 43 Posts: 926	@Thomas: Very interesting with the details about how the original queue-handler was implemented vs how the current is! If anyone is interested in reproducing this PIPE: benchmark themselves, these are the components used: http://megaburken.net/~patrik/SourceSink.lha http://uhc.a1k.org/time

17 October 2023, 19:49	#13
koobo Registered User Join Date: Sep 2019 Location: Finland Posts: 365	Thanks @Thomas, interesting I performed another measurement on the miggy (A1200/060, OS 3.1) with the following HippoPlayer-pipeline: Input is a MIDI file (st2cred.mid, Turrican music, pretty simple for a MIDI tune) Timidity convert this to WAV and writes to PIPE:hippo/65536/2 HippoPlayer reads PIPE:hippo 16-bit stereo WAV data is converted to 8-bit and fed to Paula Music is output from the speakers For a three minute run the average CPU usage: Queue-handler V37: 37% Queue-handler V45: 29% Here about 170 kB/s of data goes through the pipe.

18 October 2023, 04:51	#14
DisasterIncarna Registered User Join Date: Oct 2021 Location: England Posts: 1,202	wonder if someone can abuse the same setup to make an online animation streaming program, even if it was just for tiny iff animations or something.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
PIPE: already mounted	spawnterror	support.AmigaOS	16	20 August 2022 21:59
Pipe Mania (Empire)	Avanze	MarketPlace	0	11 April 2016 21:39
\\.\pipe\WinUAE	boir	support.WinUAE	17	23 October 2015 08:57
Amiga, vfork(), and pipe()	tygre	Coders. General	5	03 December 2011 01:35
Pipe Dream (original)	MethodGit	request.Old Rare Games	3	16 December 2010 22:17