[RMXP] Need help with character encodings

[flowerthief]

Since somebody offered to translate a game of mine to Japanese, I'm attempting to make a version of the game that supports Japanese fonts. For the display of hard-coded text, there is no problem. However, this particular game does quite a lot of reading of text from files on disk. Since at the time I originally wrote this code I only had ascii text in mind, the Japanese text is being read as gibberish.

I don't know much about character encodings. Is there a way to transform the gibberish back into Japanese text, preferably in such a way that I don't have to rewrite all my code from before? I'm kind of lost right now so help would be appreciated.

 

[ForeverZer0]

Probably not converting it back after it has already been messed up, but make sure you read and write the text as Unicode, since it supports pretty much everything you will be using.

 

[flowerthief]

How do I read the text as unicode? Even before I've done any parsing, the text read from the file is gibberish. Like in the following code, the lines are printed as gibberish.

f = File.open(file, "r")
f.each do |one_line|
print one_line
end

What can be done to one_line to fix the encoding?

 

[ForeverZer0]

If I remember correctly it is:

file = File.open('filename.txt', 'r:utf-8')

You can also add a header to the text file so that it can be opened automatically that way, but I personally prefer this way.

 

[flowerthief]

I get "illegal access mode" error with that.

 

[ForeverZer0]

Hmmm. I must have the call wrong.
You can likely find the correct argument on google.

Another solution is to change the Ruby interpreter's default encoding.
[rgss]$KCODE='u'
[/rgss]

I think 1.8 supported that, which RMXP runs. I know that there are issues in older version of Ruby running a script (I think even 1.8.6 had the problem) with a BOM header ('\xef\xbb\xbf' in a header on a text file lets a program know the data is in UTF-8), but I don't think that was a problem with simply reading text from a file.

It might not hurt to open the file in Notepad and check the encoding there, and change it if it is in the wrong format, which you can see when you "Save As...".

If that doesn't help, send me a copy of the file you are trying to open, and I'll see what I can do. I take it you are trying to read text from a file and display it in a message?

 

[flowerthief]

Saving the file as utf-8 did the trick.

So it's reading lines from the file correctly now. As I feared, my entire parsing code, which is somewhat lengthy, is now broken. It turns out that in Ruby 1.8, strings are stored as bytes and now the bytes are all off. So I've got lots of code to rewrite, but that's my problem.

Thanks for your help! I might be asking you for help again later.

 

[zeus81]

You should split your string : charArray = str.split(//)
And use join when you want a piece : charArray[x, y].join('')

 

[flowerthief]

That will certainly help.
It's still a helluva pain to have to change everything. If this were Ruby 1.9 the job would be quick and easy. Almost makes me wish I were using VX.

(Opening a file with a given encoding like 'r:utf-8' was Ruby 1.9 only btw)