Python log in the jupyter notebook appears only in the terminal, not in the output cell

[ziky90]

I am using jupyter notebook and since short time ago logs stopped to appear in output cells and started to appear only in the terminal.

I have described the problem here:
http://stackoverflow.com/questions/35936086/jupyter-notebook-does-not-print-logs-to-the-output-cell/35940918#35940918

Logging to output cell works correctly for ipykernel==4.2.2 and earlier, but since ipykernel==4.3.0 https://github.com/ipython/ipykernel/releases/tag/4.3.0 logs goes to terminal instead of the output cell.

I am not sure, but my guess is that the problem is caused by this commit: 65b6144

I have already opened one issue in the jupyter repository jupyter/jupyter#120, but I was advised to better open it here.

[updiversity]

I am having the same problem... I confirm that with jupyter 4.0.6 it was working fine.

[davegreenwood]

Hi, I am also having this problem, however I'm ssh tunnelling to a remote notebook server and don't have a corresponding console to capture the emits... The notebook fills with stack traces to this effect for each call to a logger in any module I've imported:

Traceback (most recent call last): File "/home/daveg/anaconda2/lib/python2.7/logging/__init__.py", line 874, in emit stream.write(fs % msg) IOError: [Errno 5] Input/output error Logged from file h5.py, line 62 Traceback (most recent call last): File "/home/daveg/anaconda2/lib/python2.7/logging/__init__.py", line 874, in emit stream.write(fs % msg) IOError: [Errno 5] Input/output error Logged from file h5.py, line 63 Traceback (most recent call last): File "/home/daveg/anaconda2/lib/python2.7/logging/__init__.py", line 874, in emit stream.write(fs % msg)

I can't comment on any previous version, but this is with ipykernel (4.3.1)

[ostrokach]

Ran into the same problem after upgrading to ipykernel=4.3.1.

Using ipykernel=4.2.2 until this is fixed.

[skywalker-lili]

Confirm the issue.
I am using Anaconda distribution 2.5.0 (Python 2.7.11) on Windows 10 Version 1607 (Build 14393.10). The ipython core is 4.0.3 and notebook server version is 4.1.0

[ostrokach]

Any idea what is causing this..? Spent half an hour trying to fix what I thought was a broken logging module before I remembered this issue.... :(

[ostrokach]

Added the following to my IPython notebooks, and the logging messages are back. Yay! :)

# Create logger
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

# Create STDERR handler
handler = logging.StreamHandler(sys.stderr)
# ch.setLevel(logging.DEBUG)

# Create formatter and add it to the handler
formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

# Set STDERR handler as the only handler 
logger.handlers = [handler]

[takluyver]

I think #127 should have fixed this, so when we release 4.4, workarounds should not be necessary.

I'll close this issue, but let us know if you can reproduce it with ipykernel master, or >= 4.4.

[ckhung]

Hi: I am a jupyter newbie. From "Help" => "About" I see: notebook server 4.3.2, Python 2.7.6, IPython 5.2.2 . I am running jupyter inside a docker. I have the same problem. Everything works fine until I execute reload(sys). After that, output goes to the console instead of cells. This simpler workaround works for me. However, executing help() makes the output disappear again, and the above workaround does not help.

[takluyver]

Why do you think you need reload(sys)? Doing that is generally not advised, and is expected to break things.

[ckhung]

I wanted to use graphviz, so I did:

with open('study-map.dot', 'r') as myfile:
    data=myfile.read()
from graphviz import Source
src = Source(data)
src

But got this error:

UnicodeDecodeError                        Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/core/formatters.pyc in __call__(self, obj)
    309             method = get_real_method(obj, self.print_method)
    310             if method is not None:
--> 311                 return method()
    312             return None
    313         else:

/usr/local/lib/python2.7/dist-packages/graphviz/files.pyc in _repr_svg_(self)
     81 
     82     def _repr_svg_(self):
---> 83         return self.pipe(format='svg').decode(self._encoding)
     84 
     85     def pipe(self, format=None):

/usr/local/lib/python2.7/dist-packages/graphviz/files.pyc in pipe(self, format)
     94             format = self._format
     95 
---> 96         data = text_type(self.source).encode(self._encoding)
     97 
     98         outs = backend.pipe(self._engine, format, data)

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 63: ordinal not in range(128)

<graphviz.files.Source at 0x7f0a744bff90>

So then I followed this and added this:

import sys
#stdout = sys.stdout
reload(sys)
sys.setdefaultencoding('utf8')
#sys.stdout = stdout

(The two commented lines were added afterwards, to take stdout back to jupyter cells from the console.)

What would be the correct way of asking jupyter to treat strings as utf8 after it has started then? Or can it only be done before jupyter is started? (I am not quite familiar with python, either ^_^)

[takluyver]

The error there is coming from inside the graphviz package. I suspect the author hasn't been very careful with bytes and unicode - you may want to file a bug on that project about it.

I'd guess that you can work around it by replacing your open() call with:

import io
with io.open('study-map.dot', 'r', encoding='utf-8') as myfile:

Then the data you read from the file will be a unicode string rather than a bytes string.

[ckhung]

Thank you, @takluyver. It works!

However, I find it rather confusing about the system default encoding and the enforcement against changing it. First, the error pops up when doing I/O, not when graphviz parsing the content, so I suspect it has more to do with I/O than with graphviz. Secondly, with 2.7.12 in my working env, sys.getdefaultencoding() prints 'ascii' and yet myfile.read() (without encoding='utf-8') was successful. On the other hand with jupyter in a docker, sys.getdefaultencoding() prints 'utf-8' and yet myfile.read() (without encoding='utf-8') was unsuccessful.

So I just have to leave the bug report to someone more knowledgeable and remember to add encoding='utf-8' from now on as a sure bet.

ps. I have carelessly destroyed my previous docker env since last comment. This comment refers to a different jupyter version: jupyter 4.3.1 Python 3.5.2 IPython 5.1.0 . Here is the new error message:

UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-22-f358fa348e73> in <module>()
      1 # https://github.com/rainyear/lolita/issues/30
      2 with open('study-map.dot', 'r') as myfile:
----> 3     data=myfile.read()
      4 from graphviz import Source
      5 src = Source(data)

/opt/conda/lib/python3.5/encodings/ascii.py in decode(self, input, final)
     24 class IncrementalDecoder(codecs.IncrementalDecoder):
     25     def decode(self, input, final=False):
---> 26         return codecs.ascii_decode(input, self.errors)[0]
     27 
     28 class StreamWriter(Codec,codecs.StreamWriter):

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 63: ordinal not in range(128)