How to tell if tensorflow is using gpu acceleration from inside python shell

Question

I have installed tensorflow in my ubuntu 16 04 using the second answer here with ubuntu s builtin apt cuda installation   Now my question is how can I test if tensorflow is really using gpu  I have a gtx 960m gpu  When I import tensorflow this is the output  I tensorflow stream executor dso loader cc 105  successfully opened CUDA library libcublas so locally I tensorflow stream executor dso loader cc 105  successfully opened CUDA library libcudnn so locally I tensorflow stream executor dso loader cc 105  successfully opened CUDA library libcufft so locally I tensorflow stream executor dso loader cc 105  successfully opened CUDA library libcuda so 1 locally I tensorflow stream executor dso loader cc 105  successfully opened CUDA library libcurand so locally   Is this output enough to check if tensorflow is using gpu

User · Answer

For TF2 4  listed as the  quot official quot  way on tensorflow website to check if TF is using GPU or Not  gt  gt  gt  import tensorflow as tf  gt  gt  gt  print  quot Num GPUs Available   quot   len tf config experimental list physical devices  GPU     Num GPUs Available   2

User · Answer

I found below snippet is very handy to test the gpu      Tensorflow 2 0 Test  import tensorflow as tf import tensorflow compat v1 as tf tf disable v2 behavior   with tf device   gpu 0        a   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  2  3   name  a       b   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  3  2   name  b       c   tf matmul a  b   with tf Session   as sess      print  sess run c     Tensorflow 1 Test  import tensorflow as tf with tf device   gpu 0        a   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  2  3   name  a       b   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  3  2   name  b       c   tf matmul a  b   with tf Session   as sess      print  sess run c

User · Answer

UPDATE FOR TENSORFLOW  gt   2 1  The recommended way in which to check if TensorFlow is using GPU is the following  tf config list physical devices  GPU     As of TensorFlow 2 1  tf test gpu device name   has been deprecated in favour of the aforementioned  Then  in the terminal you can use nvidia-smi to check how much GPU memory has been alloted  at the same time  using watch -n K nvidia-smi would tell you for example every K seconds how much memory you are using  you may want to use K   1 for real-time  If you have multiple GPUs and you want to use multiple networks  each one on a separated GPU  you can use   with tf device   GPU 0          neural network 1   initialize network 1    with tf device   GPU 1          neural network 2   initialize network 2

User · Answer

if you are using tensorflow 2 x use   sess   tf compat v1 Session config tf compat v1 ConfigProto log device placement True

User · Answer

I prefer to use nvidia-smi to monitor GPU usage  if it goes up significantly when you start you program  it s a strong sign that your tensorflow is using GPU

User · Answer

For Tensorflow 2 0 import tensorflow as tf  tf test is gpu available      cuda only False      min cuda compute capability None    source here other option is  tf config experimental list physical devices  GPU

User · Answer

No  I don t think  open CUDA library  is enough to tell  because different nodes of the graph may be on different devices   To find out which device is used  you can enable log device placement like this   sess   tf Session config tf ConfigProto log device placement True     Check your console for this type of output

User · Answer

If you are using TensorFlow 2 0  you can use this for loop to show the devices   with tf compat v1 Session   as sess    devices   sess list devices   devices

User · Answer

This should give the list of devices available for Tensorflow  under Py-3 6    tf   tf Session config tf ConfigProto log device placement True   tf list devices      DeviceAttributes  job localhost replica 0 task 0 device CPU 0  CPU  268435456

User · Answer

Run this command in Jupyter or your IDE to check if Tensorflow is using a GPU or not  tf config list physical devices  GPU

User · Answer

You have some options to test whether GPU acceleration is being used by your TensorFlow installation   You can type in the following commands in three different platforms   import tensorflow as tf sess   tf Session config tf ConfigProto log device placement True      Jupyter Notebook - Check the console which is running the Jupyter Notebook  You will be able to see the GPU being used  Python Shell - You will be able to directly see the output   Note- do not assign the output of the second command to the variable  sess   if that helps   Spyder - Type in the following command in the console   import tensorflow as tf tf test is gpu available

User · Answer

gt  gt  gt  import tensorflow as tf   gt  gt  gt  tf config list physical devices  GPU    2020-05-10 14 58 16 243814  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcuda so 1 2020-05-10 14 58 16 262675  I tensorflow stream executor cuda cuda gpu executor cc 981  successful NUMA node read from SysFS had negative value  -1   but there must be at least one NUMA node  so returning NUMA node zero 2020-05-10 14 58 16 263119  I tensorflow core common runtime gpu gpu device cc 1555  Found device 0 with properties  pciBusID  0000 01 00 0 name  GeForce GTX 1060 6GB computeCapability  6 1 coreClock  1 7715GHz coreCount  10 deviceMemorySize  5 93GiB deviceMemoryBandwidth  178 99GiB s 2020-05-10 14 58 16 263143  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcudart so 10 1 2020-05-10 14 58 16 263188  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcublas so 10 2020-05-10 14 58 16 264289  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcufft so 10 2020-05-10 14 58 16 264495  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcurand so 10 2020-05-10 14 58 16 265644  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcusolver so 10 2020-05-10 14 58 16 266329  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcusparse so 10 2020-05-10 14 58 16 266357  I tensorflow stream executor platform default dso loader cc 44  Successfully opened dynamic library libcudnn so 7 2020-05-10 14 58 16 266478  I tensorflow stream executor cuda cuda gpu executor cc 981  successful NUMA node read from SysFS had negative value  -1   but there must be at least one NUMA node  so returning NUMA node zero 2020-05-10 14 58 16 266823  I tensorflow stream executor cuda cuda gpu executor cc 981  successful NUMA node read from SysFS had negative value  -1   but there must be at least one NUMA node  so returning NUMA node zero 2020-05-10 14 58 16 267107  I tensorflow core common runtime gpu gpu device cc 1697  Adding visible gpu devices  0  PhysicalDevice name   physical device GPU 0   device type  GPU      As suggested by  AmitaiIrron   This section indicates that a gpu was found  2020-05-10 14 58 16 263119  I tensorflow core common runtime gpu gpu device cc 1555  Found device 0 with properties   pciBusID  0000 01 00 0 name  GeForce GTX 1060 6GB computeCapability  6 1 coreClock  1 7715GHz coreCount  10 deviceMemorySize  5 93GiB deviceMemoryBandwidth  178 99GiB s   And here that it got added as an available physical device  2020-05-10 14 58 16 267107  I tensorflow core common runtime gpu gpu device cc 1697  Adding visible gpu devices  0   PhysicalDevice name   physical device GPU 0   device type  GPU

User · Answer

In addition to other answers  the following should help you to make sure that your version of tensorflow includes GPU support   import tensorflow as tf print tf test is built with cuda

User · Answer

Following piece of code should give you all devices available to tensorflow   from tensorflow python client import device lib print device lib list local devices         Sample Output       name    cpu 0    device type   CPU    memory limit  268435456   locality         incarnation  4402277519343584096       name    gpu 0    device type   GPU    memory limit  6772842168   locality       bus id  1       incarnation  7471795903849088328   physical device desc   device  0  name  GeForce GTX 1070  pci bus id  0000 05 00 0

User · Answer

The following will also return the name of your GPU devices   import tensorflow as tf tf test gpu device name

User · Answer

This will confirm that tensorflow using GPU while training also    Code  sess   tf Session config tf ConfigProto log device placement True     Output  I tensorflow core common runtime gpu gpu device cc 885  Found device 0 with properties   name  GeForce GT 730 major  3 minor  5 memoryClockRate  GHz  0 9015 pciBusID 0000 01 00 0 Total memory  1 98GiB Free memory  1 72GiB I tensorflow core common runtime gpu gpu device cc 906  DMA  0  I tensorflow core common runtime gpu gpu device cc 916  0    Y  I tensorflow core common runtime gpu gpu device cc 975  Creating TensorFlow device   gpu 0  - gt   device  0  name  GeForce GT 730  pci bus id  0000 01 00 0  Device mapping   job localhost replica 0 task 0 gpu 0 - gt  device  0  name  GeForce GT 730  pci bus id  0000 01 00 0 I tensorflow core common runtime direct session cc 255  Device mapping   job localhost replica 0 task 0 gpu 0 - gt  device  0  name  GeForce GT 730  pci bus id  0000 01 00 0

User · Answer

With tensorflow 2 0  gt   import tensorflow as tf sess   tf compat v1 Session config tf compat v1 ConfigProto log device placement True

User · Answer

Tensorflow 2 1  A simple calculation that can be verified with nvidia-smi for memory usage on the GPU   import tensorflow as tf   c1      n   10  def matpow M  n       if n  lt  1   Abstract cases where n  lt  1         return M     else          return tf matmul M  matpow M  n-1    with tf device   gpu 0        a   tf Variable tf random uniform shape  10000  10000    name  a       b   tf Variable tf random uniform shape  10000  10000    name  b       c1 append matpow a  n       c1 append matpow b  n

User · Answer

Ok  first launch an ipython shell from the terminal and import TensorFlow     ipython --pylab Python 3 6 5  Anaconda custom  64-bit    default  Apr 29 2018  16 14 56   Type  copyright    credits  or  license  for more information IPython 6 4 0 -- An enhanced Interactive Python  Type     for help  Using matplotlib backend  Qt5Agg  In  1   import tensorflow as tf   Now  we can watch the GPU memory usage in a console using the following command     realtime update for every 2s   watch -n 2 nvidia-smi   Since we ve only imported TensorFlow but have not used any GPU yet  the usage stats will be     Notice how the GPU memory usage is very less    700MB   Sometimes the GPU memory usage might even be as low as 0 MB     Now  let s load the GPU in our code  As indicated in tf documentation  do   In  2   sess   tf Session config tf ConfigProto log device placement True     Now  the watch stats should show an updated GPU usage memory as below     Observe now how our Python process from the ipython shell is using   7 GB of the GPU memory     P S  You can continue watching these stats as the code is running  to see how intense the GPU usage is over time

User · Answer

I find just querying the gpu from the command line is easiest   nvidia-smi   -----------------------------------------------------------------------------    NVIDIA-SMI 384 98                 Driver Version  384 98                       ------------------------------- ---------------------- ----------------------    GPU  Name        Persistence-M  Bus-Id        Disp A   Volatile Uncorr  ECC     Fan  Temp  Perf  Pwr Usage Cap          Memory-Usage   GPU-Util  Compute M                                                                                        0  GeForce GTX 980 Ti  Off    00000000 02 00 0  On                    N A     22    33C    P8    13W   250W     5817MiB    6075MiB        0       Default    ------------------------------- ---------------------- ----------------------    -----------------------------------------------------------------------------    Processes                                                        GPU Memory      GPU       PID   Type   Process name                             Usage                                                                                             0      1060      G    usr lib xorg Xorg                            53MiB        0     25177      C   python                                      5751MiB    -----------------------------------------------------------------------------    if your learning is a background process the pid from  jobs -p should match the pid from nvidia-smi

User · Answer

You can check if you are currently using the GPU by running the following code     import tensorflow as tf tf test gpu device name     If the output is     it means you are using CPU only  If the output is something like that  device GPU 0  it means GPU works       And use the following code to check which GPU you are using     from tensorflow python client import device lib  device lib list local devices

User · Answer

This is the line I am using to list devices available to tf session directly from bash   python -c  import os  os environ  TF CPP MIN LOG LEVEL      3   import tensorflow as tf  sess   tf Session     print x  for x in sess list devices     print tf   version        It will print available devices and tensorflow version  for example    DeviceAttributes  job localhost replica 0 task 0 device CPU 0  CPU  268435456  10588614393916958794   DeviceAttributes  job localhost replica 0 task 0 device XLA GPU 0  XLA GPU  17179869184  12320120782636586575   DeviceAttributes  job localhost replica 0 task 0 device XLA CPU 0  XLA CPU  17179869184  13378821206986992411   DeviceAttributes  job localhost replica 0 task 0 device GPU 0  GPU  32039954023  12481654498215526877  1 14 0

User · Answer

Apart from using sess   tf Session config tf ConfigProto log device placement True   which is outlined in other answers as well as in the official TensorFlow documentation  you can try to assign a computation to the gpu and see whether you have an error   import tensorflow as tf with tf device   gpu 0        a   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  2  3   name  a       b   tf constant  1 0  2 0  3 0  4 0  5 0  6 0   shape  3  2   name  b       c   tf matmul a  b   with tf Session   as sess      print  sess run c     Here      cpu 0   The CPU of your machine    gpu 0   The GPU of your machine  if you have one    If you have a gpu and can use it  you will see the result  Otherwise you will see an error with a long stacktrace  In the end you will have something like this       Cannot assign a device to node  MatMul   Could not satisfy explicit   device specification   device GPU 0  because no devices matching that   specification are registered in this process     Recently a few helpful functions appeared in TF    tf test is gpu available tells if the gpu is available tf test gpu device name returns the name of the gpu device   You can also check for available devices in the session   with tf Session   as sess    devices   sess list devices     devices will return you something like    DeviceAttributes  job tpu worker replica 0 task 0 device CPU 0  CPU  -1  4670268618893924978     DeviceAttributes  job tpu worker replica 0 task 0 device XLA CPU 0  XLA CPU  17179869184  6127825144471676437     DeviceAttributes  job tpu worker replica 0 task 0 device XLA GPU 0  XLA GPU  17179869184  16148453971365832732     DeviceAttributes  job tpu worker replica 0 task 0 device TPU 0  TPU  17179869184  10003582050679337480     DeviceAttributes  job tpu worker replica 0 task 0 device TPU 1  TPU  17179869184  5678397037036584928

User · Answer

Tensorflow 2 0  Sessions are no longer used in 2 0  Instead  one can use tf test is gpu available   import tensorflow as tf  assert tf test is gpu available   assert tf test is built with cuda     If you get an error  you need to check your installation

User · Answer

Run the following in Jupyter   import tensorflow as tf sess   tf Session config tf ConfigProto log device placement True     If you ve set up your environment properly  you ll get the following output in the terminal where you ran  jupyter notebook    2017-10-05 14 51 46 335323  I c  tf jenkins home workspace release-win m windows-gpu py 35 tensorflow core common runtime gpu gpu device cc 1030  Creating TensorFlow device   gpu 0  - gt   device  0  name  Quadro K620  pci bus id  0000 02 00 0  Device mapping   job localhost replica 0 task 0 gpu 0 - gt  device  0  name  Quadro K620  pci bus id  0000 02 00 0 2017-10-05 14 51 46 337418  I c  tf jenkins home workspace release-win m windows-gpu py 35 tensorflow core common runtime direct session cc 265  Device mapping   job localhost replica 0 task 0 gpu 0 - gt  device  0  name  Quadro K620  pci bus id  0000 02 00 0   You can see here I m using TensorFlow with an Nvidia Quodro K620

User · Answer

In the new versions of TF  gt 2 1  the recommended way for checking whether TF is using GPU is  tf config list physical devices  GPU

User · Answer

With the recent updates of Tensorflow  you can check it as follow     tf test is gpu available  cuda only False  min cuda compute capability None    This will return True if GPU is being used by Tensorflow  and return False otherwise    If you want device device name you can type   tf test gpu device name    Get more details from here

User · Answer

Put this near the top of your jupyter notebook   Comment out what you don t need     confirm TensorFlow sees the GPU from tensorflow python client import device lib assert  GPU  in str device lib list local devices       confirm Keras sees the GPU  for TensorFlow 1 X   Keras  from keras import backend assert len backend tensorflow backend  get available gpus     gt  0    confirm PyTorch sees the GPU from torch import cuda assert cuda is available   assert cuda device count    gt  0 print cuda get device name cuda current device         NOTE  With the release of TensorFlow 2 0  Keras is now included as part of the TF API   Originally answerwed here

User · Answer

I think there is an easier way to achieve this   import tensorflow as tf if tf test gpu device name        print  Default GPU Device      format tf test gpu device name     else      print  Please install GPU version of TF     It usually prints like   Default GPU Device   device GPU 0   This seems easier to me rather than those verbose logs

[python] How to tell if tensorflow is using gpu acceleration from inside python shell?

For TF2.4+ listed as the "official" way on tensorflow website to check if TF is using GPU or Not

Examples related to python

Examples related to tensorflow

Examples related to ubuntu

Examples related to gpu