OT Unicode, Locales and Python (for a wxPy app)

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
Hello list,
I know this is not wx-oriented, but it's for my wxPython project and I am
not on other python-related lists, so I thought I'd ask here. It's to do
with unicide and i18n and l10n.

I don't want to use wx specific locale code because I output to the terminal
as well as the gui. If there is some advantage to them, please help me out.

Code-wise I am going to start the app something like this:
(minus the error traps etc.)
presLocale = locale.setlocale ( locale.LC_ALL,'')
encoding = locale.getlocale ( ) [ 1 ]

My tests seem to confirm that the system (or shell, like bash) supplies
command lines arguments as strings already encoded in the 'encoding' of the
locale. (Or Python makes it seem that way.)
So, in order to get unicode objects out of a cli argument, I must:

path = sys.argv[1]
d = os.listdir ( unicode ( path, encoding ) )

So os.listdir, should return a list of unicode objects.

Is this the right way to handle input from a user when you don't know who or
where they may be? I am not at all sure how to test this on my machine. I
have installed a few locales like 'fi_FI' and 'af_ZA' just so I could mess
with the LANG env-variable before I run test python apps, but they all end
in ".UTF-8". I am not sure how to change the encoding of the locale, so I
can test other kinds. (Am I right in seeing 'ascii' and '8859' as
encodings?)

Thanks for any u"help" :)

/d






Reply | Threaded
Open this post in threaded view
|

Re: OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
12 hour bump :D

/d



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Rune Devik
Hi
 
This article is pretty good and may give you some hints..
http://www.pyzine.com/Issue008/Section_Articles/article_Encodings.html
 
Regards,
--
Rune Devik
http://www.garageinnovation.org
 
On 12/14/06, Donn Ingle <[hidden email]> wrote:
12 hour bump :D

/d


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
> This article is pretty good and may give you some hints..
> http://www.pyzine.com/Issue008/Section_Articles/article_Encodings.html
Thanks for the link. For some reason that site refuses to load on my end.
I'll keep trying on and off.

/d



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Werner F. Bruhin
In reply to this post by Donn Ingle
Hi Donn,

Have you seen this wiki page:
http://wiki.wxpython.org/index.cgi/Internationalization?highlight=%28international%29

Also not specific to Unicode, it might still help with I18N stuff and
have you seen the new I18N demo in the 2.8.0.1 wxPython demo.

Werner
Donn Ingle wrote:

> 12 hour bump :D
>
> /d
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
>
>  




Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Werner F. Bruhin
In reply to this post by Donn Ingle
Donn Ingle wrote:
>> This article is pretty good and may give you some hints..
>> http://www.pyzine.com/Issue008/Section_Articles/article_Encodings.html
>>    
> Thanks for the link. For some reason that site refuses to load on my end.
> I'll keep trying on and off.
>  
It works for me, just tried and looks like it has some interesting info.
Werner



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
In reply to this post by Werner F. Bruhin
> Have you seen this wiki page:
>
http://wiki.wxpython.org/index.cgi/Internationalization?highlight=%28international%29
 
> Also not specific to Unicode, it might still help with I18N stuff and
> have you seen the new I18N demo in the 2.8.0.1 wxPython demo.

Yes, seen the link and No, haven't been able to upgrade to 2.8 yet - network
is too slow today, keeps failing to apt-get. I will check it out soonest.

I think I am going to go with the assumption that:
locale.setlocale (blah blah)
enc = locale.getlocale()[1]
enc is the encoding that the user is using. If I get any Unicode Decode
Errors then I am going to "fix" the strings by a process of 'ignore'.


/d



Reply | Threaded
Open this post in threaded view
|

Re: OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
In reply to this post by Donn Ingle
> I am not sure how to change the encoding of the
> locale, so I can test other kinds.

Any hints? I'm on Kubuntu 6.06

/d



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Werner F. Bruhin
Hi Donn,

Donn Ingle wrote:
>> I am not sure how to change the encoding of the
>> locale, so I can test other kinds.
>>    
>
> Any hints? I'm on Kubuntu 6.06
>  
I do this in my app:

if hasattr(sys, "frozen"):  #Py2Exe does not run Site.py
    sys.setdefaultencoding('iso-8859-1')
else:                       #The Python interpreter needs to reload the
function
    reload(sys)
    sys.setdefaultencoding('iso-8859-1')
    del sys.setdefaultencoding

Note that I am not using Unicode build (at least not yet) and my app is
supporting English, German and French.

I also needed to do this to ensure that the correct stuff was returned
by e.g. 'letters', 'lowercase' and 'uppercase' (all string constants).

import locale
locale.setlocale(locale.LC_ALL, '')

Werner



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Werner F. Bruhin
In reply to this post by Donn Ingle
Hi Donn,

Donn Ingle wrote:
>> I am not sure how to change the encoding of the
>> locale, so I can test other kinds.
>>    
>
> Any hints? I'm on Kubuntu 6.06
>  
One more thing, all your Python source code files should have a line
similar to this:

# -*- coding: iso-8859-1 -*-#

They defined the encoding used of strings within your source code.

Werner



Reply | Threaded
Open this post in threaded view
|

Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Donn Ingle
In reply to this post by Werner F. Bruhin
Werner,
Thanks for the setdefaultencoding idea!

BTW:
>     sys.setdefaultencoding('iso-8859-1')
>     del sys.setdefaultencoding
Why do you have that del ?

/d



Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: OT Unicode, Locales and Python (for a wxPy app)

Werner F. Bruhin
Hi Donn,

Donn Ingle wrote:

> Werner,
> Thanks for the setdefaultencoding idea!
>
> BTW:
>  
>>     sys.setdefaultencoding('iso-8859-1')
>>     del sys.setdefaultencoding
>>    
> Why do you have that del ?
>  
Frankly, I can't remember.  I must have thought if this is the normal
behaviour of Python and as I do it this way instead of using
sitecustomize.py I should go and delete it.

I can not remember how I found out about this, but here some links:
http://blog.ianbicking.org/illusive-setdefaultencoding.html
http://www.diveintopython.org/xml_processing/unicode.html

Werner