English Amiga Board


Go Back   English Amiga Board > Support > support.Apps

 
 
Thread Tools
Old 19 July 2018, 21:24   #1
Yulquen74
Registered User
 
Join Date: May 2013
Location: Kleppe / Norway
Posts: 198
PFS3 19.0 issue - > ALERT: Wrong index block ID when copying folder with many files

This is happening in WinUAE since I prefer it over my genuine machine when copying large amount of files into the CF card I use in my real A1200.

The last days I have unpacked thousands of LHA archives onto my CF card
in my emulation environment in WinUAE 4.0.0 (set up to resemble my real A1200, emulating Blizzard1260/132MB RAM, full 060 in compatible mode) without any issues, in fact I have had no (real) PFS issues since I started building up this CF card partition.

I downloaded an archive with close to 20000 game cover scans, intended for tinylauncher, where all files are contained within a single folder. As the archive was zipped, I unpacked the cover folder to my C:Temp drive (available as a "drive" in WinUAE), and when I try and copy this folder to CF-card from inside WinUAE, after a while it fails with the PFS error requester.

If I restart the emulated machine, everything is fine (pfsdoctor reports no errors), and it turns out 3500 - 4500 files was copied (I renamed the cover folder each time to see if the problem was bad sectors).

The problem is consistent for each attempt, and I have tried to remove all commodities (have also retired mcp for a while now), increase stack to huge value, increase partition buffers to 500, copy in workbench instead of dopus5, but it still fails.

The CF is a 16GB Kingston ultimate x266 card set up with one 7.8GB partition (just below the CHS limit), and now has about 1GB left. Total amount of files of the entire disk is approaching 250.000.

Is there a limit of how many files one can/should have in one single folder, or partition for that matter with PFS3, or does my problem lie elsewhere?

Thanks in advance for suggestions.
Yulquen74 is offline  
AdSense AdSense  
Old 20 July 2018, 00:09   #2
talybont
Registered User

 
Join Date: Oct 2016
Location: Norway
Posts: 96
http://eab.abime.net/showthread.php?t=90778


- Bug fixes, including finally fixed nasty bug causing Wrong Index Block ID
(and others) if partition was >=5G and lots of small files were created
very quickly, for example unpacking archive with thousands of small files.
This bug has most likely existed since PFS3 introduced >=5G partition
support.
talybont is offline  
Old 20 July 2018, 18:00   #3
Yulquen74
Registered User
 
Join Date: May 2013
Location: Kleppe / Norway
Posts: 198
I'm using the newest version, but to verify I re-downloaded the aminet version, updated the filesystem although the existing one was the same number of bytes, rebooted, and then did the same copy process over, but still failing, now at >9000 files.

Another observation, when rebooting and inspecting the destination folder, if I copy one of the images to RAM: and double-click it in dopus (lauches multiview) the picture displays fine. But if I double-click it in the >9000 file directory, sometimes I get a PFS3 error requester, and if it does load, it takes several seconds before the picture is shown on the screen (we are talking small <50KB iff images here).

Also, during copy process, it starts out fast, and then gets slower and slower to a point where it looks like it takes 1 second or more to copy each small picture before it ultimately fails.

The files I try and copy, is this archive:
ftp://ftp:amiga@grandis.nu/~Uploads/...rt-PAL-v0.3.7z
Yulquen74 is offline  
Old 20 July 2018, 18:07   #4
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 43
Posts: 21,500
Use hardfile (with same size as CF) and repeat the test.

Removes lots of unknown variables (bad CF card, bad adapter, unreliable USB and so on..)
Toni Wilen is online now  
Old 21 July 2018, 11:27   #5
Yulquen74
Registered User
 
Join Date: May 2013
Location: Kleppe / Norway
Posts: 198
I did, and it was able to copy all the files without errors.

But I had the same observation with regards to copy speed: At some point in the copy process, it went from many files per second to 1-2 files, but still faster than with CF card though. Doing copy tests from big folder on Amiga partition to RAM: showed that I was able to copy any file from the top of the list down to and including the file Indianapolis500_SCR4.iff fast, which is file no.9104 from the top.

All files from here down to the bottom copy very very slow, both with HDF file and CF card (Using HDF, 1-2 files per second, from CF card, less than 1 file per second).

Either something is very wrong in my setup, or PFS3 is just slow with that many files in the same folder.

With that said, if I copy files from the PC mounted folder inside WinUAE to RAM, no such speed problems exist.

I have tried with another CF card, and have the same result there.


Using the card in my real A1200 (060@85/fast-ata pio4), copying the big folder to somewhere else, the copy process starts to really slow down after a few thousand files, and looking at the HD LED it is lit almost continously during the copy process, even when copying only 1-2 files per second.

Does the PFS housekeeping increase that much when the number of files increase to a certain number?

Also, when copying files from the big folder to RAM I also experience the big slowdown in copying from file 9105 from the top, as in WinUAE (same image). Copy test done with dopus 5.91 and cli copy.
During copy, HD LED also implies heavy activity even when copying less than 1 small file per second.

With regards to PFS errors experienced in WinUAE, I have tried several CF cards, and something tells me it is not block errors on the cards themselves, at least not initially (unless the copy process wears it down in the process), but there may be something with the CF adapter or USB as you said.

All in all it seems having that many files in a single folder is a bad idea for now, but it would be nice to find out the cause of the problem.
Yulquen74 is offline  
Old 21 July 2018, 14:43   #6
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 43
Posts: 21,500
I don't know (or want to know) too much about PFS3 internals but I haven't heard any other similar issues but Amiga directory containing 10000+ files is very rare.

Does increasing buffers help? (What is current size? Check what addbuffers return, not that hdtoolbox shows because pfs3 does some adjustments)

WinUAE direct HD access is relatively slow because there is no caching. Increasing buffers to something like 1000-2000 should increase performance noticeably.
Toni Wilen is online now  
Old 21 July 2018, 16:16   #7
Yulquen74
Registered User
 
Join Date: May 2013
Location: Kleppe / Norway
Posts: 198
I had it set to 500 in RDB, but using addbuffers it maxes out at 600 if one tries to set it higher.
Yulquen74 is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
PFS3 Wrong Index Block ID vintageit support.Hardware 92 17 September 2016 12:38
Wrong Checksum on block 1694 skel2raw project.ClassicWB 3 12 February 2016 01:22
PFS3 continuosly giving "Block outside reserved area" -errors emuola project.ClassicWB 15 19 August 2012 19:47
Index files *again* 1time project.EAB File Server 7 06 October 2010 16:59
DMS-LHA CDs - Index files 1time project.EAB File Server 6 20 September 2010 18:59

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 11:21.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.
Page generated in 0.06371 seconds with 14 queries