hardfile speed problem

[Ed Cruse]

I did some more testing on regular hardfiles versus RDB hardfiles and there's a very large difference in speed. I added buffers, used ffs and sfs and it's made no difference. The average read and write speeds are a little less then a meg a second. Normally I can see the effects of the Windows harddrive cache on speeds, especially read speeds, but with the regular hardfiles this affect is very small. It's like the regular hardfiles aren't using the Window harddrive cache like the RDB ones do.

[Toni Wilen]

Originally Posted by Ed Cruse

I did some more testing on regular hardfiles versus RDB hardfiles and there's a very large difference in speed. I added buffers, used ffs and sfs and it's made no difference. The average read and write speeds are a little less then a meg a second. Normally I can see the effects of the Windows harddrive cache on speeds, especially read speeds, but with the regular hardfiles this affect is very small. It's like the regular hardfiles aren't using the Window harddrive cache like the RDB ones do.

They use the exact same code, with exact same open parameters (routine that opens the file does not even know if it is "regular" or RDB file)

Perhaps it is due to different "geometry"?

[Ed Cruse]

Originally Posted by Toni Wilen

They use the exact same code, with exact same open parameters (routine that opens the file does not even know if it is "regular" or RDB file)

Perhaps it is due to different "geometry"?

I really don't think the geometery would make that much of a difference, I'm talking over two minutes verses 5 seconds. I used to use regular hardfiles and they weren't slow, the only reason I went to RDB hardfiles is because I like doing the partitioning like a real amiga. I think that was about 6 months ago, that's as close as I can get.

I don't know what to tell you, with the disk speed program I'm using it really looks like there's no Windows harddrive cache being used on the regular hardfiles. The program writes a 1024000 byte block and then reads it, with my RDB hardfiles the read speed is way up in the gigs per second because it's reading from the cache. With the regular hardfile it's about 900k per sec. Could there be a real narrow bandwidth path the regular hardfiles are going through? If I watch on the screen as the files scroll up it looks like the data is being pushes through a straw.

I don't know what I could be doing wrong, setting up a regular hardfile is pretty easy, not very many options.

[Toni Wilen]

Originally Posted by Ed Cruse

Could there be a real narrow bandwidth path the regular hardfiles are going through?

No. How many times do I have to repeat this?

RDB or no RDB HDF uses EXACT SAME CODE. It _IS_ EXACT SAME thing. Only difference is RDB parsing, partition parameters are read from RDB, not from configuration file. (and there are some differences, in "geometry" and other things)

(and btw, when did this become 1.4.5-specific problem?)

[Ed Cruse]

Originally Posted by Toni Wilen

No. How many times do I have to repeat this?

RDB or no RDB HDF uses EXACT SAME CODE. It _IS_ EXACT SAME thing. Only difference is RDB parsing, partition parameters are read from RDB, not from configuration file. (and there are some differences, in "geometry" and other things)

(and btw, when did this become 1.4.5-specific problem?)

Maybe there's a bug in using the config file information, the code maybe basically the same but there's still a difference.

If I find a possible problem in 1.4.5 doesn't that make it a 1.4.5 specific problem regardless of when the possible problem actually occured?

[Toni Wilen]

Originally Posted by Ed Cruse

Maybe there's a bug in using the config file information, the code maybe basically the same but there's still a difference.

Thats possible but so far I haven't seen any proof either

If I find a possible problem in 1.4.5 doesn't that make it a 1.4.5 specific problem regardless of when the possible problem actually occured?

You never said this is 1.4.5 specific Usually it is good idea to say something like, "problem does not happen in official 1.4.4" etc..

and:

Allowed posts in beta thread (not yet posted):

- bug reports but make sure bug is NOT in previous offical version. (Old bug -> separate thread)

[Toni Wilen]

Originally Posted by Ed Cruse

Maybe there's a bug in using the config file information, the code maybe basically the same but there's still a difference.

Try this: format your "fast" RDB hardfile as a regular hardfile. (DO NOT re-create it, use existing file) Important question is: did it get slower?

(and more editing..)

Does diskspeed or similar software agree with speed differences?

[Ed Cruse]

Originally Posted by Toni Wilen

Try this: format your "fast" RDB hardfile as a regular hardfile. (DO NOT re-create it, use existing file) Important question is: did it get slower?

(and more editing..)

Does diskspeed or similar software agree with speed differences?

I took fast RDB hardfile that was created with 1.4.5, changed it back to a regular hardfile with the harddrive panel, formated it and it's much slower. As a RDB it did the test in 11 sec, as a regular it did the test 131 sec.

I don't have anything similar to diskspeed, but the copy command and diskspeed both agree that the regular hardfiles are slow. With the regular hardfiles, the files scrolling up the screen using the copy command jerk up about 1/4 screen then pause, then jerk up again and pause, this happens during whole copy process. With the RDB hardfiles it scrolls up smoothly with no pauses unless it's copying a big file like 20 megs.

If I didn't do the right thing with converting RDB to regular please advice.

[Ed Cruse]

Originally Posted by Toni Wilen

Thats possible but so far I haven't seen any proof either



You never said this is 1.4.5 specific Usually it is good idea to say something like, "problem does not happen in official 1.4.4" etc..

and:

I'm going to try and figure out when the problem with the slow hardfiles started. I'm thinking it started with 1.4.1.

Sorry, I'll try and be more specific in the future.

[Toni Wilen]

(moved from beta-thread)

[Ed Cruse]

I learned something today that you guys might find interesting or not.

For the last couple of days I've been trying to figure out why regular hardfiles are a lot slower then RDB ones. Well Toni straightened me out on that. Regular hardfiles are basically like floppies, they use the old filesystem (OFS). All you have to do when you get into the Amiga is format them using the FFS switch and you now have a regular hardfile that's as fast as a FFS RDB hardfile.

You're all probably going dah! Well I didn't know, I've never had to use the FFS switch when formating.

[Doc Mindie]

Ed, format from WB... as in select HD, use pulldown "Format" there you can select FFS and DCache ad such.

Personally, I never bother with the RDB..... not under WinUAE. On the A1200, I'm "forced" to care about it

[Ed Cruse]

Originally Posted by Doc Mindie

Ed, format from WB... as in select HD, use pulldown "Format" there you can select FFS and DCache ad such.

Personally, I never bother with the RDB..... not under WinUAE. On the A1200, I'm "forced" to care about it

I already know about WB format, I just didn't realized regular hardfiles used the OFS and it never accured to me to format them with FFS after I figured it out.

Personally I like the RDBs, I've used for years with real Amigas and it's something that I understand, and it makes WinUAE feel more like a real Amiga. I also run a full size screen so I pretty much forget I'm actually on a PC.

[YoJoe!]

Why format a hardfile with FFS?
Why not use SFS?

Could you run your timing tests again with SFS vs. FFS?

[Ed Cruse]

Originally Posted by YoJoe!

Why format a hardfile with FFS?
Why not use SFS?

Could you run your timing tests again with SFS vs. FFS?

I don't have it, is it something I can find on Aminet? If you select SFS when making a hardfile and don't have it, it will default to OFS. I learned that the hard way, I thought SFS was built into WinUAE, wrong.

[Ed Cruse]

Originally Posted by Ed Cruse

I don't have it, is it something I can find on Aminet? If you select SFS when making a hardfile and don't have it, it will default to OFS. I learned that the hard way, I thought SFS was built into WinUAE, wrong.

Originally Posted by YoJoe!

Why format a hardfile with FFS?
Why not use SFS?

Could you run your timing tests again with SFS vs. FFS?

I did something wrong, the above quotes came out in the wrong order.

I aswered my own question, yes. I downloaded sfs which was uploaded recently, I assume it's the latest. The times are listed below.

FFS = 11.5
SFS = 6.0
PFS2 = 4.5

FFS SFS are regular hardfiles, PFS2 is a RDB hardfile, you might be able to adjust the settings if you use SFS with a RDB hardfile and get more speed. All the speeds are the best after running them several times, and all were run on the same running of WinUAE. The times vary each time I bring up WinUAE, not much but a little.

[YoJoe!]

You forgot to list:
OFS = 131.0

[YoJoe!]

Have you any way of measuring CPU usage during the timing tests?

People keep saying that, yes SFS/PFS are "faster". but they use 95% CPU power compared to FFS 10% CPU power.

[Ed Cruse]

Originally Posted by YoJoe!

Have you any way of measuring CPU usage during the timing tests?

People keep saying that, yes SFS/PFS are "faster". but they use 95% CPU power compared to FFS 10% CPU power.

I'll try and do cpu usuage today, but I'm kind of busy building a robot at one of the local middle schools, so I may have to get back with you tomorrow.

To me it seems reasonable that a faster filing system would use more cpu time because it's faster and therefore is not waiting as much. PFS's main increase in speed comes from it updating the directory after it's done with a copy job, it updates when there's no activity for a couple of seconds, kind of dangerous but efficient. PFS really works good with my USB zip drive, it's a usb1 so the communications isn't real fast so by stuffing the files out first and then updating really helps with speed. I've done through-put tests with a single large file, pfs and ffs are basically the same in speed, it's just the file handling that's more efficient with pfs.

[Ed Cruse]

It didn't take as long as I thought, but I had to do the test on my A3000/060 because the cpu usuage program I use doesn't seem to get along with the JIT on WinUAE. Even though the program runs at a -128 task priority it still slows other programs down like filing systems. On a real amiga the program has no effect. The numbers I came up with are consistant with my last message.

FFS = 49 sec, 18% usuage
PFS = 26 sec, 31% usuage

PFS is about twice as fast and a little less then twice the cpu. Don't compare the times with the times I got earlier on WinUAE, the Work: partition on the A3000 is only 30meg, on WinUAE it's about 55megs.