Jupyter tricks

what is this

In Python3 Jupyter (IPython notebook), I will introduce the following method.

--Display pandas large DataFrame without omitting it --Deployment of generator -Wide the width of Notebook --Hide DataFrame index

Preparation

First, install more-itertools.

bash


pip install more-itertools

Then do the following: Alternatively, place a file with the following content in <user's directory> /.ipython/profile_default/startup.

jupyter


import IPython.core.getipython
from more_itertools import chunked
ip = IPython.core.getipython.get_ipython()
class ChunkedDataFrame:
    def __init__(self, *args): self.df, self.n, self.m = args
    def _repr_html_(self):
        return ('' if not isinstance(self.df, pd.DataFrame) else
            '\n'.join(self.df.iloc[:self.m, r]._repr_html_()
                      for r in chunked(range(self.df.shape[1]), self.n)))
def C_impl(self, args):
    return ChunkedDataFrame(_, *[int(i) for i in (args+' 12 5').split()[:2]])
def L_impl(self, args):
    print(list(_))
if ip:
    ip.define_magic('C', C_impl)
    ip.define_magic('L', L_impl)

Display large pandas DataFrame without omitting it

Let's display a dummy DataFrame.

jupyter


import numpy as np, pandas as pd
a = pd.DataFrame(np.random.randint(0, 100, (3, 24)))
a
0 1 2 3 4 5 6 7 8 9 ... 14 15 16 17 18 19 20 21 22 23
0 0 30 60 68 53 81 60 10 49 28 ... 49 84 24 92 6 54 52 60 52 71
1 18 86 70 14 10 51 94 10 9 46 ... 90 81 22 65 50 93 28 83 71 22
2 62 58 42 79 83 71 22 44 7 52 ... 54 63 93 14 8 54 55 73 0 26

The middle is omitted. Following the DataFrame display, execute as follows.

jupyter


%C
0 1 2 3 4 5 6 7 8 9 10 11
0 0 30 60 68 53 81 60 10 49 28 78 93
1 18 86 70 14 10 51 94 10 9 46 14 19
2 62 58 42 79 83 71 22 44 7 52 3 76
12 13 14 15 16 17 18 19 20 21 22 23
0 7 3 49 84 24 92 6 54 52 60 52 71
1 17 78 90 81 22 65 50 93 28 83 71 22
2 37 28 54 63 93 14 8 54 55 73 0 26

It is displayed in 12 columns at a time. If you want to change the number of columns, you can specify it as "% C 8". Let's display the DataFrame again and then try again.

jupyter


a

jupyter


%C 8
0 1 2 3 4 5 6 7
0 80 61 53 28 36 56 94 72
1 41 75 74 58 86 82 92 26
2 16 40 97 14 73 54 33 87
8 9 10 11 12 13 14 15
0 81 20 16 37 39 5 8 34
1 4 0 43 16 5 46 25 25
2 60 42 72 43 44 70 58 42
16 17 18 19 20 21 22 23
0 71 41 60 32 69 79 23 79
1 4 71 92 31 25 26 6 3
2 61 10 35 81 50 60 68 81

You can also change the number of lines with the second argument.

jupyter


a

jupyter


%C 8 2
0 1 2 3 4 5 6 7
0 80 61 53 28 36 56 94 72
1 41 75 74 58 86 82 92 26
8 9 10 11 12 13 14 15
0 81 20 16 37 39 5 8 34
1 4 0 43 16 5 46 25 25
16 17 18 19 20 21 22 23
0 71 41 60 32 69 79 23 79
1 4 71 92 31 25 26 6 3

Deploying the generator

If you look at the generator as it is, you cannot see the contents. It would be nice to use "list (_)", but I tried to use only 2 characters.

jupyter


(i for i in range(10))
>>>
<generator object <genexpr> at 0x000000000XXXX>

jupyter


%L
>>>
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Widen the width of the notebook

jupyter


%%HTML
<style>
    div#notebook-container    { width: 95%; }
    div#menubar-container     { width: 65%; }
    div#maintoolbar-container { width: 99%; }
</style>

Hide DataFrame index

jupyter


df.style.set_table_styles([{'selector': 'tbody th', 'props': [('visibility','hidden')]}])

reference

―― In IPython, when I tried to see the value, it was a generator, so I came up with it when I was frustrated. -Prevent the display from being omitted in pandas

that's all

Recommended Posts

Jupyter tricks
3 Jupyter notebook (Python) tricks
Jupyter begins
Jupyter Tips 4
Jupyter nbextensions
Jupyter Tips 5
Jupyter Tips 3
Jupyter Tips 2
Jupyter Notebook: 4 banal tips and tricks
Jupyter installation
Python tricks
Jupyter installation error
Jupyter Notebook memo
Introducing Jupyter Notebook
Jupyter study notes_006
Jupyter Lab begins
Jupyter process management
Powerful Jupyter Notebook
Golang on jupyter
[Python] Pre-processing tricks
Jupyter on AWS
Jupyter Study Note_002
Jupyter notebook password
Build Jupyter Hub
Jupyter Notebook memo
Jupyter study notes_008
Jupyter study notes_004
jupyter qtconsole config
Jupyter study notes_001
Jupyter Study Note_003
Jupyter Study Note_007
Jupyter Study Note_005