Step-by-step debugging with IPython

Question

From what I have read  there are two ways to debug code in Python   With a traditional debugger such as pdb or ipdb  This supports commands such as c for continue  n for step-over  s for step-into etc    but you don t have direct access to an IPython shell which can be extremely useful for object inspection   Using  IPython by embedding an IPython shell in your code  You can do from IPython import embed  and then use embed   in your code  When your program script hits an embed   statement  you are dropped into an IPython shell  This allows the full inspection of objects and testing of Python code using all the IPython goodies  However  when using embed   you can t step-by-step through the code anymore with handy keyboard shortcuts    Is there any way to combine the best of both worlds  I e   Be able to  step-by-step through your code with handy pdb ipdb keyboard shortcuts  At any such step  e g  on a given statement   have access to a full-fledged IPython shell   IPython debugging as in MATLAB  An example of this type of  quot enhanced debugging quot  can be found  in MATLAB  where the user always has full access to the MATLAB engine shell  and she can still step-by-step through her code  define conditional breakpoints  etc  From what I have discussed with other users  this is the debugging feature that people miss the most when moving from MATLAB to IPython  IPython debugging in Emacs and other editors  I don t want to make the question too specific  but I work mostly in Emacs  so I wonder if there is any way to bring this functionality into it  Ideally  Emacs  or the editor  would allow the programmer to set breakpoints anywhere on the code and communicate with the interpreter or debugger to have it stop in the location of your choice  and bring to a full IPython interpreter on that location

User · Answer

Developing New Code  Debugging inside IPython   Use Jupyter IPython cell execution to speed up experiment iterations Use   debug for step through   Cell Example     debug      for n in range 4           n gt 2   Debugging Existing Code  IPython inside debugging   Debugging a broken unit test  pytest     --pdbcls IPython terminal debugger TerminalPdb --pdb Debugging outside of test case  breakpoint    python -m ipdb  etc  IPython embed   for full IPython functionality where needed while in the debugger   Thoughts on Python  I agree with the OP that many things MATLAB does nicely Python still does not have and really should since just about everything in the language favors development speed over production speed  Maybe someday I will contribute more than trivial bug fixes to CPython   https   github com ipython ipython commit f042f3fea7560afcb518a1940daa46a72fbcfa68  See also Is it possible to run commands in IPython with debugging

User · Answer

You can start IPython session from pudb and go back to the debugging session as you like   BTW  ipdb is using IPython behind the scenes and you can actually use IPython functionality such as TAB completion and magic commands  the one starts with      If you are OK with ipdb you can start it from IPython using commands such as  run and  debug   ipdb session is actually better than plain IPython one in the sense you can go up and down in the stack trace etc   What is missing in ipdb for  object inspection    Also  python el bundled with Emacs    24 3 has nice ipdb support

User · Answer

One option is to use an IDE like Spyder which should allow you to interact with your code while debugging  using an IPython console  in fact   In fact  Spyder is very MATLAB-like  which I presume was intentional  That includes variable inspectors  variable editing  built-in access to documentation  etc

User · Answer

What about ipdb set trace     In your code   import ipdb  ipdb set trace   update  now in Python 3 7  we can write breakpoint    It works the same  but it also obeys to the PYTHONBREAKPOINT environment variable  This feature comes from this PEP  This allows for full inspection of your code  and you have access to commands such as c  continue   n  execute next line   s  step into the method at point  and so on  See the ipdb repo and a list of commands  IPython is now called  edit  part of  Jupyter   ps  note that an ipdb command takes precedence over python code  So in order to write list foo  you d need print list foo    or  list foo    Also  if you like the ipython prompt  its emacs and vim modes  history  completions      it s easy to get the same for your project since it s based on the python prompt toolkit

User · Answer

Did you try this tip      Or better still  use ipython  and call   from IPython Debugger import Tracer  debug here   Tracer         then you can just use  debug here         whenever you want to set a breakpoint

User · Answer

Looks like the approach in  gaborous s answer is deprecated   The new approach seems to be   from IPython core import debugger debug   debugger Pdb   set trace  def buggy method        debug

User · Answer

Running from inside Emacs  IPython-shell and breakpoint set via pdb set trace   should work   Checked with python-mode el  M-x ipython RET etc

User · Answer

You can start IPython from within ipdb  Induce the ipdb debugger1  import idpb  ipdb set trace    Enter IPython from within in the ipdb gt  console2  from IPython import embed  embed    Return to the ipdb gt  console from within IPython  exit   If you re lucky enough to be using Emacs  things can be made even more convenient  This requires using M-x shell   Using yasnippet and bm  define the following snippet   This will replace the text ipdb in the editor with the set-trace line   After inserting the snippet  the line will be highlighted so that it is easily noticeable and navigable   Use M-x bm-next to navigate    - - mode  snippet - -   name  ipdb   key  ipdb   expand-env    yas-after-exit-snippet-hook   bm-toggle     -- import ipdb  ipdb set trace     1 All on one line for easy deletion   Since imports only happen once  this form ensures ipdb will be imported when you need it with no extra overhead  2 You can save yourself some typing by importing IPython within your  pdbrc file  try      from IPython import embed except      pass  This allows you to simply call embed   from within ipdb  of course  only when IPython is installed

User · Answer

If you type exit   in embed   console the code continue and go to the next embed   line

User · Answer

the right  easy  cool  exact answer for the question is to use  run macro with -d flag   In  4   run -d myscript py NOTE  Enter  c  at the ipdb gt   prompt to continue execution           gt   cygdrive c Users mycodefolder myscript py 4  lt module gt          2                                                                   3                         ---- gt  4 a 1                                                   5 b 2

User · Answer

The Pyzo IDE has similar capabilities as the OP asked for  You don t have to start in debug mode  Similarly to MATLAB  the commands are executed in the shell  When you set up a break-point in some source code line  the IDE stops the execution there and you can debug and issue regular IPython commands as well   It does seem however that step-into doesn t  yet   work well  i e  stopping in one line and then stepping into another function  unless you set up another break-point   Still  coming from MATLAB  this seems the best solution I ve found

User · Answer

Prefixing an     symbol to commands you type in pdb seems to have the same effect as doing something in an IPython shell  This works for accessing help for a certain function  or even variable names  Maybe this will help you to some extent  For example   ipdb gt  help numpy transpose      No help on  numpy transpose    But  help numpy transpose  will give you the expected help page on numpy transpose  Similarly for variable names  say you have a variable l  typing  l  in pdb lists the code  but  l prints the value of l

User · Answer

Update on May 28  2016  Using RealGUD in Emacs For anyone in Emacs  this thread shows how to accomplish everything described in the OP  and more  using  a new important debugger in Emacs called RealGUD which can operate with any debugger  including ipdb   The Emacs package isend-mode   The combination of these two packages is extremely powerful and allows one to recreate exactly the behavior described in the OP and do even more  More info on the wiki article of RealGUD for ipdb   Original answer  After having tried many different methods for debugging Python  including everything mentioned in this thread  one of my preferred ways of debugging Python with IPython is with embedded shells  Defining a custom embedded IPython shell  Add the following on a script to your PYTHONPATH  so that the method ipsh   becomes available  import inspect    First import the embed function from IPython terminal embed import InteractiveShellEmbed from IPython config loader import Config    Configure the prompt so that I know I am in a nested  embedded  shell cfg   Config   prompt config   cfg PromptManager prompt config in template    N In  lt     gt     prompt config in2 template          D     prompt config out template    N Out lt     gt        Messages displayed when I drop into and exit the shell  banner msg     quot  n  Nested Interpreter  n quot   quot Hit Ctrl-D to exit interpreter and continue program  n quot   quot Note that if you use  kill embedded  you can fully deactivate n quot   quot This embedded instance so it will never turn on again quot      exit msg      Leaving Nested interpreter     Wrap it in a function that gives me more context  def ipsh        ipshell   InteractiveShellEmbed config cfg  banner1 banner msg  exit msg exit msg       frame   inspect currentframe   f back     msg      Stopped at  0 f code co filename  at line  0 f lineno   format frame         Go back one level         This is needed because the call to ipshell is inside the function ipsh       ipshell msg stack depth 2   Then  whenever I want to debug something in my code  I place ipsh   right at the location where I need to do object inspection  etc  For example  say I want to debug my function below Using it  def my function b     a   b   ipsh      lt - This will embed a full-fledged IPython interpreter   a   4  and then I invoke my function 2  in one of the following ways   Either by running a Python program that invokes this function from a Unix shell Or by invoking it directly from IPython  Regardless of how I invoke it  the interpreter stops at the line that says ipsh    Once you are done  you can do Ctrl-D and Python will resume execution  with any variable updates that you made   Note that  if you run the code from a regular IPython  the IPython shell  case 2 above   the new IPython shell will be nested inside the one from which you invoked it  which is perfectly fine  but it s good to be aware of  Eitherway  once the interpreter stops on the location of ipsh  I can inspect the value of a  which be 2   see what functions and objects are defined  etc  The problem  The solution above can be used to have Python stop anywhere you want in your code  and then drop you into  a fully-fledged IPython interpreter  Unfortunately it does not let you add or remove breakpoints once you invoke the script  which is highly frustrating  In my opinion  this is the only thing that is preventing IPython from becoming a great debugging tool for Python  The best you can do for now  A workaround is to place ipsh   a priori at the different locations where you want the Python interpreter to launch an IPython shell  i e  a breakpoint   You can then  quot jump quot  between different pre-defined  hard-coded  quot breakpoints quot  with Ctrl-D  which would exit the current embedded IPython shell and stop again whenever the interpreter hits the next call to ipsh    If you go this route  one way to exit   quot debugging mode quot  and ignore all subsequent breakpoints  is to use ipshell dummy mode   True which will make Python ignore any subsequent instantiations of the ipshell object that we created above

User · Answer

You can use IPython s  pdb magic  Just call  pdb in IPython and when an error occurs  you re automatically dropped to ipdb  While you don t have the stepping immediately  you re in ipdb afterwards   This makes debugging individual functions easy  as you can just load a file with  load and then run a function  You could force an error with an assert at the right position    pdb is a line magic  Call it as  pdb on   pdb 1   pdb off or  pdb 0  If called without argument it works as a toggle

User · Answer

From python 3 2  you have the interact command  which gives you access to the full python ipython command space

[python] Step-by-step debugging with IPython

Developing New Code

Debugging Existing Code

Thoughts on Python

Examples related to python

Examples related to debugging

Examples related to emacs

Examples related to ipython

Examples related to pdb