Re: python mode shell and unicode

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: python mode shell and unicode

Andreas Röhler
Max Arnold wrote:

> On Sat, Mar 27, 2010 at 06:48:00PM +0100, Andreas Röhler wrote:
>>> My python-shell invoked via C-c C-c from python buffer can not print unicode characters (emits
>>> UnicodeEncodeError) and sys.stdout.encoding is empty. System wide LANG set to "ru_RU.UTF-8" and
>>> os.environ.get('LANG') in python shell confirms this.
>>>
>>> When python-shell invoked manually (M-x python-shell) there is no encoding issues. Any ideas?
>>>
>> What you get from
>> C-h v buffer-file-coding-system?
>
> ---
> buffer-file-coding-system is a variable defined in `C source code'.
> Its value is utf-8-unix
> Local in buffer test1.py; global value is utf-8-unix
> ...
> It does not apply to sending output to subprocesses, however.
> ---
>
> Python module contains encoding specification # -*- coding: utf-8 -*- at the beginning.
> Emacs version is 23.1.
>
>
>

Hmm, see inside `python-send-region'

  ;; Fixme: Write a `coding' header to the temp file if the region is
  ;; non-ASCII.

Maybe that indicates the cause?

You could try `py-execute-file' from python-mode.el

CC to mailing list there.


Andreas

--
https://code.launchpad.net/~a-roehler/python-mode
https://code.launchpad.net/s-x-emacs-werkstatt/



_______________________________________________
Python-mode mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-mode
Reply | Threaded
Open this post in threaded view
|

Re: python mode shell and unicode

Andreas Röhler
Max Arnold wrote:

> On Sun, Mar 28, 2010 at 09:25:02AM +0200, Andreas Röhler wrote:
>> Hmm, see inside `python-send-region'
>>
>>   ;; Fixme: Write a `coding' header to the temp file if the region is
>>   ;; non-ASCII.
>>
>> Maybe that indicates the cause?
>
> I think this is unrelated to the issue because my python code contains no
> unicode characters; they are received from parsed web page and then printed
> to stdout. The problem is with python process spawned by Emacs - its stdout
> encoding is not specified (sys.stdout.encoding is None).
>
> Maybe this is a python feature: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=415968
>
> But then I do not understand why manually invoked pyton-shell handles unicode
> characters just fine...

Looks like your problem is rather on the python than the emacs side.

If I change the function in the bugreport given above into

def output(s):
    # print s.decode('utf-8')
    print s

redirecting into FILE with > works


>
>> You could try `py-execute-file' from python-mode.el
>
> Sorry, didn't mentioned that I use python.el shipped with emacs. How to disable
> built-in one and enable python-mode.el?

Download it here

http://launchpad.net/python-mode/trunk/5.1.0/+download/python-mode.el

and get it loaded

 How to invoke py-execute-file from
> M-x prompt?

Just

M-x py-execute-file

should do it

Andreas

>
> BTW, print u'\xA9' is a nice test for this issue.
>
> Cheers, Max
>
>
>

_______________________________________________
Python-mode mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-mode
Reply | Threaded
Open this post in threaded view
|

Re: python mode shell and unicode

Andreas Röhler
Max Arnold wrote:

> On Sun, Mar 28, 2010 at 04:56:49PM +0200, Andreas Röhler wrote:
>> Looks like your problem is rather on the python than the emacs side.
>
>> Download it here
>> http://launchpad.net/python-mode/trunk/5.1.0/+download/python-mode.el
>>
>> M-x py-execute-file
>
> Ok, I placed (require 'python-mode) to init.el and it seems to be activated
> (autoloading didn't worked for me). But M-x shows no available completions
> for py-execute-file:
>
> Possible completions are:
> py-electric-backspace                  py-electric-colon
> py-electric-delete                     py-end-of-def-or-class
> py-execute-buffer                      py-execute-def-or-class
> py-execute-import-or-reload            py-execute-region
> py-execute-string
>
> Although C-h f py-execute-file shows it and says it is defined in python-mode.el.
>
>
> Next, quick test with print u'\xA9':
>
> 1. Invoke python shell manually:
>
> M-x py-shell
>>>> print u'\xA9'
> ©
>
>
> 2. Create new buffer containing the same print command, switch it to python-mode
> and use py-execute-buffer:
>
>>>> ## working on region in file /usr/tmp/python-9773IlV.py...
> ©
>

So that's what it should do(?)

>
> 3. Close python shell (opened at step 1) and invoke py-execute-buffer again:
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position 0: ordinal not in range(128)
>
>
> Is this really a python problem?  I think there is a difference in how Emacs spawns python
> process in each case.
>
>
>

Hmm, yes, get the same error.
However, if I re-start ipython-shell parallel

it works again

In [11]: ## working on region in file /usr/tmp/python-3766xFD.py...
©


Maybe just start a python-shell to have a work-around?

Sorry, I'm not able to dive into now.

It may help, if you make a bug-report, having it noticed at least here:

https://bugs.launchpad.net/python-mode

Thanks

Andreas




_______________________________________________
Python-mode mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-mode
Reply | Threaded
Open this post in threaded view
|

Re: python mode shell and unicode

Andreas Röhler
Max Arnold wrote:

> On Sun, Mar 28, 2010 at 06:34:06PM +0200, Andreas Röhler wrote:
>>> On Sun, Mar 28, 2010 at 04:56:49PM +0200, Andreas Röhler wrote:
>>>> Looks like your problem is rather on the python than the emacs side.
>>>> Download it here
>>>> http://launchpad.net/python-mode/trunk/5.1.0/+download/python-mode.el
>>>>
>>>> M-x py-execute-file
>>> Ok, I placed (require 'python-mode) to init.el and it seems to be activated
>>> (autoloading didn't worked for me). But M-x shows no available completions
>>> for py-execute-file:
>>>
>>> Possible completions are:
>>> py-electric-backspace                  py-electric-colon
>>> py-electric-delete                     py-end-of-def-or-class
>>> py-execute-buffer                      py-execute-def-or-class
>>> py-execute-import-or-reload            py-execute-region
>>> py-execute-string
>>>
>>> Although C-h f py-execute-file shows it and says it is defined in python-mode.el.
>>>
>>>
>>> Next, quick test with print u'\xA9':
>>>
>>> 1. Invoke python shell manually:
>>>
>>> M-x py-shell
>>>>>> print u'\xA9'
>>> ©
>>>
>>>
>>> 2. Create new buffer containing the same print command, switch it to python-mode
>>> and use py-execute-buffer:
>>>
>>>>>> ## working on region in file /usr/tmp/python-9773IlV.py...
>>> ©
>>>
>> So that's what it should do(?)
>
>
> Yes, (1) and (2) is expected behaviour, 0xA9 is the UTF-8 code for copyright symbol (C).
>
>>> 3. Close python shell (opened at step 1) and invoke py-execute-buffer again:
>>>
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position 0: ordinal not in range(128)
>>>
>>>
>>> Is this really a python problem?  I think there is a difference in how Emacs spawns python
>>> process in each case.
>>>
>>>
>> Hmm, yes, get the same error.
>> However, if I re-start ipython-shell parallel
>>
>> it works again
>>
>> In [11]: ## working on region in file /usr/tmp/python-3766xFD.py...
>> ©
>>
>>
>> Maybe just start a python-shell to have a work-around?
>
> Initially I discovered this problem using python.el. It spawns new python
> process upon C-c C-c even if there is existing one (looks like it launches
> one process per buffer). So this workaround applicable only for python-mode.el
>
>
>> Sorry, I'm not able to dive into now.
>>
>> It may help, if you make a bug-report, having it noticed at least here:
>>
>> https://bugs.launchpad.net/python-mode
>
> Ok, I'll do this.  Should I do the same for python.el somewhere?
>
>
>

Yes, thanks

M-x report-emacs-bug

will provide the destination.

Andreas
_______________________________________________
Python-mode mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-mode