PFS3 or PFS3 SCSI Direct

[mfilos]

My 16GB CF works just fine with PFS3ds (18.5) and scsi.device 43.47. Iirc my Work partition >4GB wasn't visible with Doobrey's 44.2 scsi.device. With Cosmos 43.47 it works like a charm.

[Toni Wilen]

Ok, all-in-one PFS3 attached for testing.

- TD (<4G), TD64, NSD and DirectSCSI supported, "best" method autodetected.
- 68000+ and KS 1.x+ compatible. (requesters only shown if 2.0+)

Autodetection:

If end of partition inside of first 4G:
- CMD_READ last block of partition.
- If failed, attempt same with Direct SCSI.
- If still failed: show requester and don't mount the drive.
(Above tests may need to be switched, there may be drivers that have wrap around problem when using CMD_READ, even if size is less than 4G..)

If end of partition is outside of first 4G:
- Test if NSD compatible (using NDK NSD-COMMANDSPECS test code), if ok, do read test using NSCMD_TD_READ64.
- If failed, TD64 test (using test suggested above by Thomas), if ok, TD_READ64 read test.
- If failed, Direct SCSI test.
- If still failed: show requester (different than <4G requester) and don't mount the drive.

Read test = fill buffer with same byte values, read block, read is considered ok if DoIO() didn't return error and at least one byte changed in buffer. If buffer is same (it can happen), fill buffer with value+1 and read same block again. If still not changed: driver is broken, return failed status.

I think this should be paranoid enough test

EDIT:

- >4G requester removed, should now work without any requesters if drive is not mounted (because of no TD64, NSD or DS)
- Always use standard CMD_READ/WRITE if drive is "small" (less than 128M).

EDIT2:

- DS test fixed.

[mfilos]

Just tested Toni's new PFS3 handler.
I updated my CF's RDB putting Toni's PFS3 instead of my previous 18.5 PFS3ds filesystem. From what I can understand, I don't have to re-partition/format my Hard drives to use the new version right?

- If I use the stock scsi.device from 3.1 kickstart then my 1023MB System (DH0) partition boots just fine giving me the requester that all methods failed and doesn't recognise my 15GB Work (DH1) partition.
- If I use Cosmos (43.47) or Doobrey's (44.2) scsi.device system boots just fine without a single issue!

[Toni Wilen]

No need to repartition, filesystem update does not modify partition data.

>4G mount failure dialog was removed in updated version (updated few hours after I posted it originally). Of course it still won't mount the second partition.

It is normal that 15G partition won't mount with standard scsi.device, it does not support TD64 or NSD and Direct SCSI can only access up to 8G.

Perhaps it also should limit max transfer to 128k if original max transfer was larger?

[mfilos]

Thanks Toni, I updated it again and now I had no requesters

Guess what...
Work partition is visible EVEN with older stock scsi.device (!?)

This is with stock 3.1 kickstart (you can check it out also from the icon borders lol




...and this is with my normal custom kickstart 3.9




Does the old scsi.device shows the size up to 8GB (total) due to DirectScsi?

[thomas]

The original PFS3 does not check if the end of the partition is accessible. You could create a 80 GB partition on a 20 GB drive. If only the directory area is accessible, the partition mounts correctly and can even be formatted (quick).

I would have expected the new PFS3 to fail because it checks access to end of partition, doesn't it?

[Toni Wilen]

Hmm.. I'll need to test what original scsi.device HD_SCSICMD returns when attempting to read block over 8G "border".

EDIT: HD_SCSICMD returns an error and partition won't mount. Tried with single 9G PFS3 partition.

[Toni Wilen]

Change of size (~5G vs 13G) probably means something really bad..

It looks like something truncates end of partition block to inside of 8G "border" which of course passes the end of partition check.

Could you attach RDB backup of your drive? (Recent enough hdtoolbox should have option for it)

EDIT: does it also mount if you cold boot without startup-sequence? What does c:info say?

[mfilos]

There you go Toni
https://dl.dropbox.com/u/939939/mfilos_16GB_RDB

[Toni Wilen]

RDB looks fine.. PFS3 (if formatted with PFS3 v16+) also stores size of disk in rootblock and it is checked before mounting. Mount will fail if dosenvec size != stored size.

Size change/mount succeeding won't make any sense..

(Check also my edit in previous post)

EDIT: Did you mean stock 3.1 ROM plus recent enough setpatch? Not just stock 3.1 ROM _and_ 3.1 WB?

[thomas]

Originally Posted by mfilos

...and this is with my normal custom kickstart 3.9

Does this custom Kickstart contain updated Workbench? Stock 3.1 Workbench does not support numbers bigger than 4GB. If you try to store 10 GB in bytes in 32 bits, only 2 GB will remain. Partition size didn't change, only the display is wrong. You see this by the "23% full". 3GB of 5GB would be 60%.

[mfilos]

Originally Posted by Toni Wilen

does it also mount if you cold boot without startup-sequence? What does c:info say?

If I load without the custom Kickstart with NO startup-sequence this is my output:

Originally Posted by Toni Wilen

Did you mean stock 3.1 ROM plus recent enough setpatch? Not just stock 3.1 ROM _and_ 3.1 WB?

Yeah exactly.
Loading OS3.9 without the ROM Updates (SetPatch NOROMUPDATE) but the old stock 3.1 Kickstart.

Originally Posted by thomas

Does this custom Kickstart contain updated Workbench? Stock 3.1 Workbench does not support numbers bigger than 4GB. If you try to store 10 GB in bytes in 32 bits, only 2 GB will remain. Partition size didn't change, only the display is wrong. You see this by the "23% full". 3GB of 5GB would be 60%.

You're right as well Thomas.
My custom kickstart contains Workbench.library 45.132 (changed 45.131 from PeterK)

Anywayz, I don't intend to run my partitions without my custom kickstart, so there is no problem. I just thought of posting it here just for info

[Toni Wilen]

Originally Posted by mfilos

If I load without the custom Kickstart with NO startup-sequence this is my output:

Interesting.. Perhaps ~8G IDE SCSICMD limit depends on "geometry" of drive and it isn't necessarily 8G? I don't see any other reasons anymore..

[thomas]

Info command asks DOS and DOS sees mounted partition. If PFS3 successfully mounts the partition, then it returns the size of the mounted partition, not limited to harddisk size.

Additionally PFS3 stores all directory information in the beginning of the drive. So even if the majority of the partition lies outside the harddrive's size, it can still read the entire directory. Only if you try to really access a file whose data is stored above the 8GB limit, only then you get an error.

The data on your drive (23% of 13GB + 0.9GB) does not even touch the 8GB limit. It does not even touch the 4GB limit.

[Toni Wilen]

Originally Posted by thomas

Info command asks DOS and DOS sees mounted partition. If PFS3 successfully mounts the partition, then it returns the size of the mounted partition, not limited to harddisk size.

Additionally PFS3 stores all directory information in the beginning of the drive. So even if the majority of the partition lies outside the harddrive's size, it can still read the entire directory. Only if you try to really access a file whose data is stored above the 8GB limit, only then you get an error.

The data on your drive (23% of 13GB + 0.9GB) does not even touch the 8GB limit. It does not even touch the 4GB limit.

Yes but it still don't explain why the last partition block check returns success. Only possibility is that some of the tests actually succeed without reading correct data (for example wrapped around data)

When using above special PFS3 version, if partion's last block is outside of 4G, partition won't mount if all read tests (NSD, TD64 and DirectSCSI) fail. I guess serial debug version is needed..

[Toni Wilen]

mfilos, attached is debug version, could do one more test?

When PFS3 volume is mounted, first it shows some information (device, partition start and end and some other info).

Then comes test requesters. ("Test x = y"). x = test mode (X:1="TD32",2=DS,3=TD64,4=NSD,-1=all failed. Y:0=failed,non-zero=ok)

Please include all information shown.

EDIT: attachment updated.

[mfilos]

There you go Toni:
https://dl.dropbox.com/u/939939/pfs3-debug.rar

I made a rar archive not to spam the thread with 7 pics

I tried this without Startup-Sequence and without the custom kick.
First 2 pics appeared before the CLI screen. then CLI screen came up and I brought the rest requesters from the back

[Toni Wilen]

Thanks.

Everything looks fine except last screen that says Direct SCSI test succeeded to read last block (31293359) which shouldn't be possible.

I'll need to think about this first..

[Toni Wilen]

Yet another test version attached.

Now SCSI Direct test executes Read Capacity SCSI command first and tests if returned last available block is same or larger than last block of partition. If not, SCSI Direct won't be used. If capacity is large enough, normal Read(10) test is done next.

Attach new read capacity dialog screenshot, thanks. (Nothing else is needed)

[mfilos]

There you go Toni mate:
https://dl.dropbox.com/u/939939/pfs3-debug2.rar