Odd problem reading a string containg non English characters

[chocsplease]

Hi,

Part of my code reads a file and then changes the apps messages to the ones stored there. In this way the user can localise the app simply by copying the prefs file, throwing it a google translate and then correcting any stupid and obvious mistakes. Quick and hopefully easy.

I'm testing the code using German but it's choking on non English characters.

The format of the lines is message=new message

Here's the code segment.

msg1=msg2=PrefsFile; /*give pointers default to the start of the prefs file*/
        
        printf("msg1=%s-\n",msg1);
        
        /*now read through prefs setting messages etc*/
        for (i=0;i<size;i++)
        {
            
            if ((i<size) && ((char)PrefsFile[i]==' ') && (msg1==&PrefsFile[i])) /*if not at end of file, and found space and at start of line*/
            {
                printf("found leading space, msg1 -%s- i=%d\n", msg1, i);
                while ((i<size) && ((char)PrefsFile[i]==' '))
                    i++; /*ignore leading spaces*/
                msg1=msg2=&PrefsFile[i]; /*update messages to be past spaces*/
            }    
            if (((char)PrefsFile[i]==';') && (msg1==&PrefsFile[i])) /*found comment and at start of a line*/
            {
                printf("FOUND COMMENT!\n");
                while ((i<size) && ((char)PrefsFile[i]!='\0'))
                    i++; /*scan to end of file or line ending null*/
            }
            
             if ((char)PrefsFile[i]=='=')
            {
                printf("prefs file equals found\n");
                /*temp change current pos to null and store the pos*/
                (char)PrefsFile[i]='\0';
                temp=strlen(msg1);
                printf("length of msg1 is %d\n",temp);
            
                /*make msg1 lowercase*/
                for (ii=0;ii<temp;ii++)
                {
                    msg1[ii]=tolower(msg1[ii]);
                }
                printf("msg1 is now -%s-\n",msg1);
                msg2=&(char)PrefsFile[i+1];    /*set replacement message to 1 char after equals*/
                printf("msg2 is now -%s-\n",msg2); /*<<< Its at this point that I find the problem*/
                err=update_message(msg1,msg2); /*change the required message or error if its not one we know*/
                (char)PrefsFile[i]='=';        /*put equals sign back*/
                if (err!=0) 
                {
                    Printf("Formatting error in prefs file - unknown message type?\n");
                    break;            /*got error so stop*/
                }    
            }
            else if ((i<size) && ((char)PrefsFile[i]=='\0')) /*at end of line so move msg pointers to new line, i will always be < size */
            {
                /*printf("updating msg pointers\n");*/
                msg1=msg2=&(char)PrefsFile[i+1];
                /*printf("updated ok!\n");*/
            }

This works until it hits a line like -

msg_baddisk = Datenträger kann nicht gelesen werden

When msg2 gets truncated to Datentr

The same thing happens whenever there is a German specific character in the new message. I know the PrefsFile is ok as just before this code I scan the whole thing and print out the characters and their ASCII values.

The code works fine on lines that do not contain a language specific character.

Could someone suggest what's going on? I'm once again baffled.

[meynaf]

This smells the signed char problem...
Does it still occur if you define char as unsigned ?

[demolition]

Is the file a true ASCII file, that is only using 7 bit for each character?
If the prefs file was made using a PC, then it might be some multi-char format. You could check in a hex-editor.

Edit: If it is written using ISO-8859-1, then perhaps the characters outside the lower 128 ones could cause problem when printed out, although they might still be correctly contained in the string array.

[chocsplease]

Hi and many thanks for both your replies

Originally Posted by meynaf

This smells the signed char problem...
Does it still occur if you define char as unsigned ?

Meynaf you were spot on, switching from a standard char to specifying unsigned chars for the PrefsFile and msg1 & msg2 has fixed the problem. I had no idea that C would store chars as -127 to +128 rather than 0 to 255.

You live and learn.

[meynaf]

Well, when accentuated characters start to fail in C, it's probably either charset problem (but you said you scanned the string before) or signed char...

C itself says nothing about signedness of char. It's up to the compiler. Some have an option for this.
That's for me one more reason to like asm - where this kind of thing never occurs

Anyway, it's a pleasure to have helped.