Simple way to measure cell execution time in ipython notebook

Question

I would like to get the time spent on the cell execution in addition to the original output from cell   To this end  I tried   timeit -r1 -n1 but it doesn t expose the variable defined within cell     time works for cell which only contains 1 statement   In 1     time        1 CPU times  user 4   s  sys  0 ns  total  4   s Wall time  5 96   s Out 1   1  In 2     time          Notice there is no out result in this case         x   1        x CPU times  user 3   s  sys  0 ns  total  3   s Wall time  5 96   s   What s the best way to do it   Update  I have been using Execute Time in Nbextension for quite some time now  It is great

User · Answer

time and  timeit now come part of ipython s built-in magic commands

User · Answer

I simply added   time at the beginning of the cell and got the time   You may use the same on Jupyter Spark cluster  Virtual environment using the same  Just add   time at the top of the cell and you will get the output  On spark cluster using Jupyter  I added to the top of the cell and I got output like below -    1     time      import pandas as pd      from pyspark ml import Pipeline      from pyspark ml classification import LogisticRegression      import numpy as np           code       Output  -  CPU times  user 59 8 s  sys  4 97 s  total  1min 4s Wall time  1min 18s

User · Answer

You can use timeit magic function for that    timeit CODE LINE   Or on the cell    timeit   SOME CELL CODE   Check more IPython magic functions at https   nbviewer jupyter org github ipython ipython blob 1 x examples notebooks Cell 20Magics ipynb

User · Answer

you may also want to look in to python s profiling magic command  prunwhich gives something like -   def sum of lists N       total   0     for i in range 5           L    j    j  gt  gt  i  for j in range N           total    sum L      return total   then    prun sum of lists 1000000    will return   14 function calls in 0 714 seconds    Ordered by  internal time        ncalls  tottime  percall  cumtime  percall filename lineno function      5    0 599    0 120    0 599    0 120  lt ipython-input-19 gt  4  lt listcomp gt       5    0 064    0 013    0 064    0 013  built-in method sum      1    0 036    0 036    0 699    0 699  lt ipython-input-19 gt  1 sum of lists      1    0 014    0 014    0 714    0 714  lt string gt  1  lt module gt       1    0 000    0 000    0 714    0 714  built-in method exec    I find it useful when working with large chunks of code

User · Answer

import time start   time time    the code you want to test stays here  end   time time   print end - start

User · Answer

An easier way is to use ExecuteTime plugin in jupyter contrib nbextensions package   pip install jupyter contrib nbextensions jupyter contrib nbextension install --user jupyter nbextension enable execute time ExecuteTime

User · Answer

If you want to print wall cell execution time here is a trick  use   time  lt --code goes here-- gt   but here make sure that  the   time is a magic function  so put it at first line in your code  if you put it after some line of your code it s going to give you usage error and not gonna work

User · Answer

Sometimes the formatting is different in a cell when using print res   but jupyter ipython comes with a display  See an example of the formatting difference using pandas below     time import pandas as pd  from IPython display import display  df   pd DataFrame   col0    a  0  b  0                  col1    a  1  b  1                  col2    a  2  b  2                    compare the following print df  display df    The display statement can preserve the formatting

User · Answer

This is not exactly beautiful but without extra software  class timeit        from datetime import datetime     def   enter   self           self tic   self datetime now       def   exit   self   args    kwargs           print  runtime      format self datetime now   - self tic     Then you can run it like   with timeit          your code  e g        print sum range int 1e7        49999995000000   runtime  0 00 00 338492

User · Answer

Use cell magic and this project on github by Phillip Cloud    Load it by putting this at the top of your notebook or put it in your config file if you always want to load it by default    install ext https   raw github com cpcloud ipython-autotime master autotime py  load ext autotime   If loaded  every output of subsequent cell execution will include the time in min and sec it took to execute it

User · Answer

The only way I found to overcome this problem is by executing the last statement with print   Do not forget that cell magic starts with    and line magic starts with       time clf   tree DecisionTreeRegressor   fit X train  y train  res   clf predict X test  print res    Notice that any changes performed inside the cell are not taken into consideration in the next cells  something that is counter intuitive when there is a pipeline

User · Answer

When in trouble what means what     timeit or   timeit  To get the details   Usage  in line mode     timeit  -n lt N gt  -r lt R gt   -t -c  -q -p lt P gt  -o  statement or in cell mode      timeit  -n lt N gt  -r lt R gt   -t -c  -q -p lt P gt  -o  setup code   code   code     Time execution of a Python statement or expression using the timeit module   This function can be used both as a line and cell magic   - In line mode you can time a single-line statement  though multiple   ones can be chained with using semicolons    - In cell mode  the statement in the first line is used as setup code    executed but not timed  and the body of the cell is timed   The cell   body has access to any variables created in the setup code

[python] Simple way to measure cell execution time in ipython notebook

Examples related to python

Examples related to ipython

Examples related to ipython-notebook

Examples related to jupyter