[Karnagious]

Hi, I am having a problem with the output of this query.

Code:

episode_request = {"jsonrpc": "2.0", "method": "VideoLibrary.GetEpisodes", "params":     {"properties": ["season","episode","runtime", "resume","playcount", "tvshowid", "lastplayed", "file"]}, "id": "allTVEpisodes"}

This is what I get:

Code:

[{u'tvshowid': 1, u'episode': 5, u'resume': {u'position': 0, u'total': 0}, u'season': 1, u'lastplayed': u'2013-10-01 15:25:54', u'episodeid': 1, u'label': u'1x05. Babylon', u'file': u'C:\\TV Shows\\Carniv\xe0le\\Season1\\Carniv\xe0le.S01E05.mkv', u'playcount': 0, u'runtime': 3600}]

The word is supposed to be "Carnivàle'. Whenever I try and use the 'file' entry I get this error:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 115: ordinal not in range(128)

The "solutions" I can find for this involve dealing with a string before it is used.

[Tolriq]

Well JSON use UTF8 not Ascii

[Karnagious]

Then what's going on? I can observe it in the db as "Carnivàle".

Oh and good luck with post 2,000.

[Tolriq]

Well you are trying to work with ascii values (check your error message).

You need to work with UTF to handle all special characters, doing so depends on what you use to code

[Karnagious]

OK, but I am simply using the output from the query as it is given. Does that mean it is being stored in the DB as ascii?

I have a fix for it, but it involves encode('latin-1'), which doesnt strike me as a robust solution.

[Tolriq]

I'll let you document yourself a little more on what is UTF 8 and how to handle it

I can't explain you more than the data you get is UTF encoded and that if you handle it as ascii data it won't work.

It's UTF in database too

[Karnagious]

I am not doing anything with ascii.

I cant see anywhere in my code that would make python to think that the string is ascii.

[Tolriq]

Well well well

I guess using Google is too complicated

Your error : UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 115: ordinal not in range(128)

I have never done any python but I clearly see that your unicode handler expect ascii and that it's not (Hey isn't that what I'm trying to explain you ? )

A 1 sec google gives :

http://docs.python.org/2/howto/unicode.h...de-support

With well all the same explanation I gave you and the solution ......

[Karnagious]

Of course I found that page. That's partly where my bodgy solution came from.

This was helpful as well, http://forum.xbmc.org/showthread.php?tid=144677

The confusion is that if its stored in utf-8 and provided by JSON in utf-8, then why cant it be decode("utf-8") and why does encode("utf-8") give me something different when it is already utf-8?

[sffjunkie]

This bit below shows you have a unicode value for the `file` member so the UTF-8 decoding of the returned JSON value has already taken place.

Code:

[{u'tvshowid': 1, u'episode': 5, u'resume': {u'position': 0, u'total': 0}, u'season': 1, u'lastplayed': u'2013-10-01 15:25:54', u'episodeid': 1, u'label': u'1x05. Babylon', u'file': u'C:\\TV Shows\\Carniv\xe0le\\Season1\\Carniv\xe0le.S01E05.mkv', u'playcount': 0, u'runtime': 3600}]

How are you trying to open the file name? Using the unicode string directly should work at least on an NTFS filesystem.

Code:

f = open(the_list[0][u'file'])

[Karnagious]

Unicode isnt utf. Sorry, thats my blind spot.

I was using the file name with the Playlist.Add method but before that I had to check whether the string contained a smaller string. Thats where it was producing the error.

[radu122]

I've had a similar problem with special characters, and it is not XBMC related. XBMC can handle UTF8 well.

My implementation uses Delphi, and the problem was that when sending and receiving messages on the TCP socket, the default encoding was Unicode. I've changed the encoding for sending and receiving a TCP message, and voila, solved.

You should check the same in your Python code (how are your TCP messages encoded/decoded).

Good luck!