PFS3AIO v3.1 Final

[TuKo]

I'll let Tony answer but I guess it will require a full format. Same does apply for modern systems on unvirtualized partitions.

That's a great news, I was dreaming of this on pfs3aio. This should bring some performance improvement. Is there any way to quantify those in current state ?

[Toni Wilen]

Originally Posted by alexh

Most filesystems I test the SSDs we design with use something like a 4Kbyte minimum block size. We announce we support 512 byte block size but internally we are 16Kbytes and anything less than that is sub optimal (i.e. require read-modify-write). This is because a typical NAND flash page is 16Kbytes

Yeah but 4096 block size is very common. (and it can be supported easily in pfs3, larger block sizes would require extra complexity)

Originally Posted by TuKo

I'll let Tony answer but I guess it will require a full format. Same does apply for modern systems on unvirtualized partitions.

That's a great news, I was dreaming of this on pfs3aio. This should bring some performance improvement. Is there any way to quantify those in current state ?

Yes, format is required. (and before that, setting higher block size using hdtoolbox)

Performance testing is someone else's problem
I guess not much difference when using emulator + HDF (where host OS handles caching) but there can be noticeably difference when using CF/SD/etc devices.

---

I also have related question: how many files do you have on your HDF/real Amiga harddrive partitions? I think current algorithm preallocates far too many pfs3 metadata blocks (*) when drive is relatively large (as in >8G). Allocation table for metadata blocks needs to be in RAM all the time and it is significant part of pfs3's memory usage. The larger the partition, the more metadata blocks it allocates.

*) pfs3 works differently than most other filesystem by preallocating blocks used for metadata at the beginning of partition which includes tree structures, file/dir blocks etc when formatted and it can't be increased without reformat.

[daxb]

My tools partition has 40.000 files and my small games partition 35.000. So I would guess people with a large games or whatever collection will have more then 100.000. But 1.000.000 should be very rare.

Edit: My Backup partition counts 120.000 files.

[TuKo]

I have 365'000 files in my data partition

[rutra80]

50000 files on 25 years old system.

[Toni Wilen]

Thanks. Only some extreme cases probably have more than few hundred thousand files.

Another test: pfsdoctor_experimental.zip has been updated. Added new command "info" that shows some pfs3 internal details. Important value is size of reserved blocks and amount of free reserved blocks. If everyone has free reserved blocks >50% (or so) compared to total reserved blocks, it confirms that preallocated reserved blocks area does not need to be that big. (Single reserved block can contain multiple file/dir entries)

https://download.abime.net/winuae/fi...perimental.zip

Output is something like this:

Autodetected disk access mode: Standard
Initializing
rootblock 100...
Checking rootblock
Rootblock found using 512 block size (block size was 1024)
Checking disk A500STORE
Option mask: 0000077f
Reserved blocksize: 1024
Reserved block cluster: 10
First reserved block: 2
Last reserved block: 70849
Reserved blocks: 35424
Free reserved blocks: 34985
Free blocks: 3078805
Always free blocks: 154209
Total blocks: 3155040
..
Checking rext
Max file name length: 32
Disk formatted with PFS-III 0.18

"rootblock found.." means partition was set with blocksize 1024 but default physical block size 512 was used. This is normal because previous pfs3 versions completely ignored hdtoolbox block size setting.

Some "block" values are in physical blocks (512), some in logical metadata blocks (1024) if you wonder why reserved block counts don't match.

EDIT: updated, other options possibly didn't work. (But I don't recommend using any fix options when using this version.. CHECK should be safe)

[TuKo]

Here is the output on my jumbo partition :

Autodetected disk access mode: NSD
Initializing
rootblock 100...
Checking rootblock
Rootblock found using 512 block size (block size was 1024)
Checking disk Programs
Option mask: 000007ff
Reserved blocksize: 1024
Reserved block cluster: 82
First reserved block: 2
Last reserved block: 654529
Reserved blocks: 327264
Free reserved blocks: 288392
Free blocks: 3757336
Always free blocks: 2313667
Total blocks: 46927872
..
Checking rext
Max file name length: 107
Disk formatted with PFS-III 0.19

[Jope]

Toni, the check argument runs info in this latest version.

[tomcat666]

My extremely jumbo partition (63 GB with only 9 GB free):

Workbench:> pfsdoctor_exp info ex1
Autodetected disk access mode: NSD
Initializing
rootblock 100...
Checking rootblock
Rootblock found using 512 block size (block size was 4096)
Checking disk Stuff
Option mask: 000007ff
Reserved blocksize: 1024
Reserved block cluster: 190
First reserved block: 2
Last reserved block: 1535425
Reserved blocks: 767712
Free reserved blocks: 712827
Free blocks: 25223898
Always free blocks: 5793014
Total blocks: 117395712
..
Checking rext
Max file name length: 107
Disk formatted with PFS-III 0.19
Workbench:>

[rutra80]

For some reason I don't get the "Rootblock found using..." line?

Autodetected disk access mode: Standard
Initializing
rootblock 100...
Checking rootblock
Checking disk HardDisk
Option mask: 0000077f
Reserved blocksize: 1024
Reserved block cluster: 12
First reserved block: 2
Last reserved block: 84929
Reserved blocks: 42464
Free reserved blocks: 37182
Free blocks: 1032861
Always free blocks: 200553
Total blocks: 4096000
..
Checking rext
Max file name length: 32
Disk formatted with PFS-III 0.19

[pcotter]

Same here. Missing the Rootblock found line.

Autodetected disk access mode: NSD
Initializing
rootblock 100...
Checking rootblock
Checking disk Stuff
Option mask: 000007ff
Reserved blocksize: 1024
Reserved block cluster: 184
First reserved block: 2
Last reserved block: 1489409
Reserved blocks: 744704
Free reserved blocks: 721116
Free blocks: 61910969
Always free blocks: 5611391
Total blocks: 113717248
..
Checking rext
Max file name length: 107
Disk formatted with PFS-III 0.19

DH1: /* Stuff */
Device = amithlon.device
Unit = 310
Flags = 0
SectorSize = 512
Surfaces = 8
SectorsPerBlock = 1 /* 512 bytes */
SectorsPerTrack = 256
Reserved = 2
PreAlloc = 0
Interleave = 0
LowCyl = 611
HighCyl = 56136 /* block 114968575 */
Buffers = 600
BufMemType = 0 /* ANY */
MaxTransfer = 0x0001FE00
Mask = 0x7FFFFFFE
BootPri = 0
DosType = 0x50445303 /* PDS3 */

[Toni Wilen]

Thanks. As I expected, huge amounts of free reserved blocks. It should be safe to half the amount of free blocks when partition is large enough (>5G or so). There is large jump around 10G+ size when using old PFS3 versions.

"root block found.." only appears if block size does not match. If pfs3 partition uses normal 512 logical block size, message does not appear. If partition logical block size is >512, pfsdoctor will show the message. (It failed to find root block when using configured block size, but found when retried with 512 block size. Because all pfs3 versions until now ignored logical block size)

Previously this was not a problem but now that logical block size is actually taken in account, pfsdoctor needed this backwards compatibility test.

I'll probably upload test version soon.

[Toni Wilen]

I did some "copy lots of random stuff to pfs3 partition, delete some of them, copy more random stuff until it is full, delete some, copy again" using 4096 block size. Seemed to work fine except after the whole operation pfsdoctor check reported that amount of free blocks was off by one.

Perhaps this is normal (as in some old bug when drive gets filled and part of file was already copied) or side-effect of block size update.

Technically this is not dangerous because free block size field is only used to quickly calculate free space (scanning whole bitmap etc would be really slow) but I need to debug this first.

Anyone else noticed something similar with previous pfs(aio) versions?

EDIT: copied 300 000 small files to 9G pfs3 HDF, worked without problems. Deleted and copied again, still no problems. Still wondering what was the problem with first test..

[Mathesar]

Originally Posted by alexh

Most filesystems I test the SSDs we design with use something like a 4Kbyte minimum block size. We announce we support 512 byte block size but internally we are 16Kbytes and anything less than that is sub optimal (i.e. require read-modify-write). This is because a typical NAND flash page is 16Kbytes

Does anyone really use a true SSD on Amiga's?
I guess most of us are using CompactFlash cards or IDE2SD and SCSI2SD contraptions. And most compactflash cards and SD cards are not so great when it comes to NAND flash management and wear leveling. So, I don't think it matters much on Amiga.

[Mathesar]

Originally Posted by Boing-Ball

Ummmm… Yes….Yes they do…..

But how? Last time I checked there were no SSD's with IDE or SCSI connections available.

[Aardvark]

Originally Posted by Mathesar

But how? Last time I checked there were no SSD's with IDE or SCSI connections available.

[Jope]

Originally Posted by Mathesar

But how? Last time I checked there were no SSD's with IDE or SCSI connections available.

They aren't that readily available any more, but they were available from the industrial side of things back when PATA was still a thing in embedded devices. Sizes went up to 64 or 128GB at least.

https://www.amazon.com/KingSpec-2-5-.../dp/B008RVN97A
https://solidstatedisks.co.uk/produc...-1mg3-p-detail
https://www.m4l.com/AF4GSSADHAAXX-ATP-Solid-State-Drive

[Mathesar]

Ah yes, those adapters could be used!
I am actually running a DIY SCSI2SD now with an industrial SD card from Kingston and PFS3AIO.
https://www.kingston.com/en/memory-c...crosd-uhs-i-u3
I actually wanted an SD card from Swissbit but they are too expensive for a hobby
But the Kingston's are nice as well. On my Amiga most data is actually mostly cold storage. Lot's of ISO images, ADF's, etc, etc. There is not much writing going on so not much wear and tear on the SD.
However, I do want my data to last! And these industrial SD cards operate TLC flash in SLC mode (pSLC)and they also monitor any read disturbances. So, I guess these are most suited for typical Amiga use.

[matt3k]

I have used Acard 2000 SUP SCSI to SATA with real SSD's for a very long time. Only used PFS over the years on them.

No issue, they just seem to work so I don't poke at them...

[technomancer]

Any way to manually flush pfs3 write cache?