7GB copy from Windows to AmigaOS
 

Hi,

I am trying to copy a 7GB+ directory from Windows directory to PFSAIO hardfile. It consists of ROMs, mostly Nintendo ones.

Whenever I copy the GoodNES/GoodGB uncompressed rom set to the hardfile, it copy each file from the rom set, very slowly really, which is not common, up to a point that the copy operation stops, and after that all Intuition gadgets stop responding. Intuition seems not responding any more messages and I need to reboot or quit WinUAE, because the file copy looks near to impossible. (Well, not so, because the Workbench windows itself continue working, albeit they dont update anymore!)

Albeit you said before, which the issue is AmigaOS stack overflow, I cannot believe, sorry, being the error repeated perfectly all the times. I dont understand, sorry, why the copy operation for the GoodNES/GoodGB directory, the biggest ones, takes so long for each file (1 seg for 2MB file), while for the smaller ones, like No-Intro Famicom, each file is copied very fast in chunks of ten or twenty! And why it gets slower and slower, up to a point that it stops altogether. And if it is a stack overflow really, why the stack overflow crashes justly Intuition everytime, and not other library or program?

I was thinking in posting logs, but the last log WinUAE created was over 2GB, and its very big for a simple attachment in a thread or the Zone.

What are you using to copy the files? DOpus, WB, DOS or? WB will have problems copying so many files. I found that DOpus usually copes fine with copying large stuff. Set the CPU to 060 JIT maximum speed to speed up the copy.

Ah, thanks demolition, I am using C:Copy command, I have tried many values for stack command, and about the Dopus, I will try it.

Sorry Toni, for asking this thing again, I dont want to prejudice you with my posts, albeit its looking like it was the intention, I think.

Recursive copy needs huge amount of stack and I am quite sure there is also path size limit that will caused crashes, sooner or later when copying very deep and long paths.

Suggesting CPU change is pointless. There is no speed difference whatsoever and setting 060 will only cause crashes if you don't have proper libraries..

addbuffers <destionation drive>: 1000 (or larger) may also increase copy speed.

The reason I suggested changing CPU is to be able to enable JIT which does make a big difference in speed. I expect you still cannot use JIT with 68000?

I assumed 68020+ config. Using 68000 would be even more pointless config for this use case :)

Thanks Toni. About the tip from demolition, I tried DOpus and it copied very fast, but Intuition stopped again, sooner or later it stops. The hardfile is new, its an AmigaOS 3.9 with BB1+2, only two patches in HD, and systempatch 3.3b and layers.library are disabled, and I enabled all CPU caches with the CPU command.

Is it possible that there are just too many files shown in a single window and you run out of chipram? I mean 7 GB of nintendo roms is quite a lot.

EDIT: even on GFX card(fastram) that would take a huge amount of ram to show the files in a wb window.

No, its not possible, because the window is closed, I tried copying with both Shell and DOpus, and both do it all wrong.

In my last tries, I tried to copy the GoodNES source directory over destination directory, and Directory Opus shown up a message saying: "file is in use" or something similar, and Intuition crashed again, and when I exited from WinUAE it created this dump-file:

JIT must be off (JIT on = can crash when Amiga programs do bad things and dump file is useless anyway)

Do you have some suspicious system patches?

I was not aware that JIT was activated, sorry.

Now I see the reason for that crash, WinUAE was reading the directory structure, but I stopped it prematurely by exitting the emulator. But the first error persists yet. Interestingly, Intuition stops when WinUAE is reading the directory, in the same way that I explained before. May that give some idea of whats wrong here, Toni?

About the patches, the only patches I have installed in the hardfile are SystemPatch 3.3b and layers.library, the latter loaded with C:LoadModule. But to be sincere, I have removed the two from my startup-sequence/user-startup, and it wont work.

Could it be that you are reaching some file system limit in regard to number of file system entries? I don't know what the limit is on PFS, but if it is all small files, that could be a problem?

Normal debugging steps needed: copy all files to ram disk (if error happens before all ram is used) or to directory filesystem. If it still happens: not filesystem problem. Basic stuff :)

What do you mean by "intuition stops"?

Dump is from non-official beta which makes it useless. Crash dumps must be from official betas (or final versions).

Ok, I will try it.

About Intuition, all programs that use it seems frozen, GadTools gadgets doesnt work anymore, specifically any kind of graphical control doesnt work, but top level windows themselves works and can be dragged, and send to back/front. Its very strange, really.

Ok, I will make one new. :)

I was having a similar issue copying 2 gig of data, using DM2 (Disk Master). It didn't seem to be cause by the size of the files, but rather the number of files.

Eventually, I reduced the copy operations to the folder level, and each copy was successful I would reset my WinUAE. Eventually I got everything copied.

PS - towards the end I switched to DOPUS and it seemed to be faster.