Advice needed on HDFs

[Foebane]

I have got a whole bunch of HDFs, some 200Mb, and some 400Mb, and whilst some are nearly full, others have hardly any stuff in them. When I zip up these files, they can become drastically much smaller in proportion to how much of their capacity is unused.

I am wondering if a single huge HDF is preferable to several small ones (like say 1Gb) or if there is a way to make the file size of the HDF vary with the contents. Is this what "Dynamic HDF" is? And how reliable is it? I would imagine something like that would get fragmented easily and cause problems, so what's the consensus on using variable-sizing HDFs?

EDIT: I'm going to experiment with ticking "Sparse File" and "Dynamic HDF" and see how I get on.

[DamienD]

Originally Posted by Foebane

or if there is a way to make the file size of the HDF vary with the contents. Is this what "Dynamic HDF" is?

Yes, though personally I don't use them

...no reason at all why you shouldn't if you are low on space.

[Foebane]

Originally Posted by DamienD

Yes, though personally I don't use them

...no reason at all why you shouldn't if you are low on space.

I have eight HDFs totalling around 2.5Gb, with only around 960Mb being used, and whilst I don't like the idea of putting "all of my eggs in one basket" and risking losing them all if the file becomes corrupt, on the other hand, it would make them easier to organise and be much more efficient.

I think I might go with the one giant HDF.

[thomas]

Dynamic HDF is what you want. It creates standard VHD files as used by many virtualisation softwares.

Don't forget to use RDB mode also.

And don't format a dynamic HDF. During formatting it will grow to its maximum size because each and every block is written to. Only do quick format to keep the file size small.

[Foebane]

Originally Posted by thomas

Dynamic HDF is what you want. It creates standard VHD files as used by many virtualisation softwares.

Don't forget to use RDB mode also.

And don't format a dynamic HDF. During formatting it will grow to its maximum size because each and every block is written to. Only do quick format to keep the file size small.

How efficient and compact have you found them? Can I use "Sparse File" AND "Dynamic HDF" when creating them?

[emufan]

Originally Posted by Foebane

I think I might go with the one giant HDF.

my main system consists from 2 HDFs, one for the system (SYS: with workbench, libs and
basic tools (200 MB only -> bit small, maybe 512 to 1gb is better),
and a 4GB Work: hdf, which holds all the installed apps and games.

most important, i have a backup of both of them in a working state,
updating them on a regular basis.

#1) on the ftp are some preinstalled 2GB PFS3 / FFS dynamic hardfiles.

[thomas]

A 4000 MB VHD takes up 34 KB when created. It grows with its contents.

A sparse VHD does not make sense because the VHD does not contain unused space.

A sparse file is a file system feature in NTFS. It appears as if it was a big file but does not take that space. Only areas of the file which have been written to are held on disk.

It's your choice, but as sparse files only work with NTFS, I would not use them. If you want to copy the hardfiles around, possibly to an USB drive with FAT or to a NAS drive, you would then expand the sparse files to their nominal (maximum) size while VHDs keep their small size.

[Foebane]

What I meant by "one giant HDF" is for the demos alone, I still have a 400Mb Games HDF and my main System HDF, which is 200Mb (probably from the same ClassicWB people as you, emufan?)

[emufan]

Originally Posted by Foebane

What I meant by "one giant HDF" is for the demos alone, I still have a 400Mb Games HDF and my main System HDF, which is 200Mb (probably from the same ClassicWB people as you, emufan?)

ah, ok. mine came from AmigaSYS

@thomas: thanks for explaining the sparse mode.

[Foebane]

Originally Posted by thomas

A 4000 MB VHD takes up 34 KB when created. It grows with its contents.

A sparse VHD does not make sense because the VHD does not contain unused space.

A sparse file is a file system feature in NTFS. It appears as if it was a big file but does not take that space. Only areas of the file which have been written to are held on disk.

It's your choice, but as sparse files only work with NTFS, I would not use them. If you want to copy the hardfiles around, possibly to an USB drive with FAT or to a NAS drive, you would then expand the sparse files to their nominal (maximum) size while VHDs keep their small size.

Thanks for the explanation, Thomas. I'm going to experiment with VHD files tomorrow, replace the RDB HDFs I've got with them, maybe. I'll let everyone know how I get on.

[Foebane]

So I've set up several VHD files of 800Mb each, although most will not get even quarter-filled, and I'm satisfied with the small size.

Curiously, I deleted the contents of one VHD to see if it would shrink in size, but no. Would it ever shrink, if say, defragmented? Is there such a tool in Workbench?

[thomas]

Would it ever shrink

No.

if say, defragmented?

Defragmentation would probably cause even more growth because defragmented files will be written to unused space and then the fragments will be declared unused. But new unused space will not be given back.


You can prepare a new VHD and then copy all files from the old to the new one. This will defragment as well as reorganise (shrink).

[Foebane]

Oh well, can't have everything.

I don't mind copying over everything. I go into the Shell and use "copy all to", which I only hope works even with an operating system like ClassicWB. I haven't had any problems so far.