django - Python and character normalization

Question

Hello I retrieve text based utf8 data from a foreign source which contains special chars such as u"??ü?" while I want to normalize them to English such as "??ü?" -> "iouc" . What would be the best way to achieve this ?

Answer 1

I recommend using Unidecode module:

>>> from unidecode import unidecode
>>> unidecode(u'??ü?')
'iouc'

Note how you feed it a unicode string and it outputs a byte string. The output is guaranteed to be ASCII.