Bug: Filenames containing national characters in Windows Directories

[arnljot]

Not sure if this is reported before, so here goes.

Symptom:
Certain national characters will give a "file not found" error in emmulated OS when used in filenames.

Host system:
Windows Vista Business SP2

Tested WinUAE versions
-1.5.2 "kinda works"
-1.6.1 "block"
-2.0.0 "block"

How to reproduce for 1.5.2:

1. Use an installed version of 3.x (I've tested with 3.9 and AmiKit)
2. An example of a directory that should be listed:
3. Host side: C:\AmiKit\AmiKitClean\Internet\AWeb_APL\Catalogs\espańol
4. WinUAE side: Not showing, hence "kinda works", no errors

How to reproduce for 1.6.1 and 2.0.0:

1. Do 1 to 3 like 1.5.2
2. WinUAE side: showing "espanol", notice the "normal n". Trying to access it will give an error.

It's only certain characters that WinUAE will barf at, most usual ones are fine, but this one and quite a few others it'll barf at.

To see more examples, try copying all files from AmiKit AK0 under WinUAE 1.6.1+ and see which files that it reports errors on.

I hope this is helpfull, and if there is any other way I can be of assistance, please do not hesitate to ask me for further help/debugging

EDIT:
I have tested the AmiKit.zip archive directly under WinUAE 1.5.2. The AmiKit.zip archive can be found here:
http://amikit.amiga.sk/download/linu...nux-latest.zip

Under 1.5.2 using the AmiKit.zip archive as a harddrive, the character bug is not present. Then it works just as it's supposed to. So it seems that the bug is "limited" to using directories on the host system.

[arnljot]

Isn't there any one else who can confirm this bug?

[TCD]

Tested with German 'special' characters and it works both if created within WinUAE or on Windows. If I try your example, it creates "espan'ol" here and it still works.

[arnljot]

Originally Posted by TheCyberDruid

Tested with German 'special' characters and it works both if created within WinUAE or on Windows. If I try your example, it creates "espan'ol" here and it still works.

Would you mind extracting the AmiKit.zip inside the archive I linked to above, and see if you can reproduce the error with those directories.

Like I mention, it's certain filenames that gives issues.

[TCD]

Originally Posted by arnljot

Would you mind extracting the AmiKit.zip inside the archive I linked to above, and see if you can reproduce the error with those directories.

Funnily enough the Windows package doesn't have that character. Will check with the Linux package now, but may I ask why you use it?

Edit : Extracting that folder from the Linux package with WinRAR result in 'espa±ol' as the directory name here. However, that name works then.

[gimbal]

Linux filesystems default to UTF-8, while windoze filesystems use that weird microsoft encoding thingie (iso-8859-1, quick check). Might be the cause of the weird characters.

[Toni Wilen]

It only happens with very few specific national characters with specific Windows languages/regional setting. No, I can't duplicate this, it seems to be limited to some specific languages or something.. (I use english Windows)

Reason: for some weird reason "ansi" (iso-8859-1 and other non-unicode charsets) to unicode conversion converts some national characters to "plain" version. (or perhaps it was unicode to "ansi", don't remember), for example ñ becomes n. (even if destination charset does support both..)

(btw, Windows internally uses UTF-16)

This was originally reported by AmiKit author but as usual, I "can't" fix anything unless I can duplicate it myself..

Does anything change if you add "win32.filesystem_codepage=0" to configuration file?

[arnljot]

I absolutely understand that you can't fix something that you can't reproduce. Because you cannot create a test to verify that the fix is done.

I code too so this is 100% understandable.

I'll try the codepage config and report back to you! :-)

Thanks.

[arnljot]

Originally Posted by Toni Wilen

Does anything change if you add "win32.filesystem_codepage=0" to configuration file?

It was already there

[Toni Wilen]

Originally Posted by arnljot

It was already there

Then replace it with 1252

[arnljot]

Originally Posted by Toni Wilen

Then replace it with 1252

I did.

Español works like it always has. Espanol with the accent(´) over the n doesn't work.

I think this is a shortcomming in conversion routine like you mentioned working together with a bug in the extraction routine of the AmiKit installer.

It's surely wrong that it's espańol and not Español.

[Toni Wilen]

Problem is that those routines are Windows functions..

What Windows language version do you have? What do you have in "Currrent language for non-Unicode programs"? (Somewhere in control panel, regional settings)

[arnljot]

It's "Norsk, bokmål (Norge)".

That's norwegian. We have two variants of norwegian in Norway, this is the most common one.

Should I try and set it to English?

EDIT:
fyi, changeing to british english didn't change it.

[Maccara]

Windows functions for character encodings are extremely sensitive to what language options you have set in the first place (and apps have to go "the extra mile" if they want to do anything about it).

If you try to show spanish text, it will get converted funnily if your locale settings are anything else but spanish (or a language with exactly the same accents). I.e., will not work ok in norwegian locale and neither in GB.

Toni, is WUAE coded with full unicode support (the char_w version of functions etc and not ANSI set in project settings)? If not, this is not "fixable" easily without jumping through hoops in code (I've had to port a couple of apps for this only).

[Toni Wilen]

Actually there are no problems. I think I confused this was another similar problem..

ń in espańol does not appear to be legal character in AmigaDOS (WB locale disk has only español)

<1.6.0 does not show it because ń does not exists in iso-8859-1 and WinUAE can't access it.

>=1.6.0 shows it because of unicode support but conversion to AmigaDOS path creates different name (ń becomes n because ń does not exist in Amiga charset)

Perhaps >=1.6.0 should also hide files if they contain illegal characters..

[arnljot]

Originally Posted by Toni Wilen

Perhaps >=1.6.0 should also hide files if they contain illegal characters..

It would make sense in one way. File operations on directories containing such files could be done undisturbed. On the other hand, there is no real way of detecting such mistakes then.

I think it's a matter of taste... unfortunately.

For AmiKit archive, it's gotta be correct then since WinUAE extracts it correctly. It's got to be the AmiKit installer which foobars it...

[Minuous]

I can confirm it, it's a longstanding bug.

[arnljot]

Originally Posted by Minuous

I can confirm it, it's a longstanding bug.

You confirm it's a bug in AmiKit installer, or a bug in WinUAE.

My opinion atm is that it's a bug in the AmiKit installer.