[python] How do I use TensorFlow GPU?

How do I use TensorFlow GPU version instead of CPU version in Python 3.6 x64?

import tensorflow as tf

Python is using my CPU for calculations.
I can notice it because I have an error:

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

I have installed tensorflow and tensorflow-gpu.

How do I switch to GPU version?

This question is related to python tensorflow gpu

The answer is

Follow this tutorial Tensorflow GPU I did it and it works perfect.

Attention! - install version 9.0! newer version is not supported by Tensorflow-gpu


  1. Uninstall your old tensorflow
  2. Install tensorflow-gpu pip install tensorflow-gpu
  3. Install Nvidia Graphics Card & Drivers (you probably already have)
  4. Download & Install CUDA
  5. Download & Install cuDNN
  6. Verify by simple program

from tensorflow.python.client import device_lib print(device_lib.list_local_devices())

The 'new' way to install tensorflow GPU if you have Nvidia, is with Anaconda. Works on Windows too. With 1 line.

conda create --name tf_gpu tensorflow-gpu 

This is a shortcut for 3 commands, which you can execute separately if you want or if you already have a conda environment and do not need to create one.

  1. Create an anaconda environment conda create --name tf_gpu

  2. Activate the environment activate tf_gpu

  3. Install tensorflow-GPU conda install tensorflow-gpu

You can use the conda environment.

First you need to install tensorflow-gpu, because this package is responsible for gpu computations. Also remember to run your code with environment variable CUDA_VISIBLE_DEVICES = 0 (or if you have multiple gpus, put their indices with comma). There might be some issues related to using gpu. if your tensorflow does not use gpu anyway, try this

Follow the steps in the latest version of the documentation. Note: GPU and CPU functionality is now combined in a single tensorflow package

pip install tensorflow

# OLDER VERSIONS pip install tensorflow-gpu


This is a great guide for installing drivers and CUDA if needed: https://www.quantstart.com/articles/installing-tensorflow-22-on-ubuntu-1804-with-an-nvidia-gpu/

I tried following the above tutorial. Thing is tensorflow changes a lot and so do the NVIDIA versions needed for running on a GPU. The next issue is that your driver version determines your toolkit version etc. As of today this information about the software requirements should shed some light on how they interplay:

NVIDIA® GPU drivers —CUDA 9.0 requires 384.x or higher.
CUDA® Toolkit —TensorFlow supports CUDA 9.0.
CUPTI ships with the CUDA Toolkit.
cuDNN SDK (>= 7.2) Note: Make sure your GPU has compute compatibility >3.0
(Optional) NCCL 2.2 for multiple GPU support.
(Optional) TensorRT 4.0 to improve latency and throughput for inference on some models.

And here you'll find the up-to-date requirements stated by tensorflow (which will hopefully be updated by them on a regular basis).

Strangely, even though the tensorflow website 1 mentions that CUDA 10.1 is compatible with tensorflow-gpu-1.13.1, it doesn't work so far. tensorflow-gpu gets installed properly though but it throws out weird errors when running.

So far, the best configuration to run tensorflow with GPU is CUDA 9.0 with tensorflow_gpu-1.12.0 under python3.6.

Following this configuration with the steps mentioned in https://stackoverflow.com/a/51307381/2562870 (the answer above), worked for me :)

Uninstall tensorflow and install only tensorflow-gpu; this should be sufficient. By default, this should run on the GPU and not the CPU. However, further you can do the following to specify which GPU you want it to run on.

If you have an nvidia GPU, find out your GPU id using the command nvidia-smi on the terminal. After that, add these lines in your script:

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = #GPU_ID from earlier

config = tf.ConfigProto()
sess = tf.Session(config=config)

For the functions where you wish to use GPUs, write something like the following:

with tf.device(tf.DeviceSpec(device_type="GPU", device_index=gpu_id)):

Questions with python tag:

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation Upgrade to python 3.8 using conda Unable to allocate array with shape and data type How to fix error "ERROR: Command errored out with exit status 1: python." when trying to install django-heroku using pip How to prevent Google Colab from disconnecting? "UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure." when plotting figure with pyplot on Pycharm How to fix 'Object arrays cannot be loaded when allow_pickle=False' for imdb.load_data() function? "E: Unable to locate package python-pip" on Ubuntu 18.04 Tensorflow 2.0 - AttributeError: module 'tensorflow' has no attribute 'Session' Jupyter Notebook not saving: '_xsrf' argument missing from post How to Install pip for python 3.7 on Ubuntu 18? Python: 'ModuleNotFoundError' when trying to import module from imported package OpenCV TypeError: Expected cv::UMat for argument 'src' - What is this? Requests (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.") Error in PyCharm requesting website How to setup virtual environment for Python in VS Code? Pylint "unresolved import" error in Visual Studio Code Pandas Merging 101 Numpy, multiply array with scalar What is the meaning of "Failed building wheel for X" in pip install? Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed Could not install packages due to an EnvironmentError: [Errno 13] OpenCV !_src.empty() in function 'cvtColor' error ConvergenceWarning: Liblinear failed to converge, increase the number of iterations How to downgrade python from 3.7 to 3.6 I can't install pyaudio on Windows? How to solve "error: Microsoft Visual C++ 14.0 is required."? Iterating over arrays in Python 3 How do I install opencv using pip? How do I install Python packages in Google's Colab? How do I use TensorFlow GPU? How to upgrade Python version to 3.7? How to resolve TypeError: can only concatenate str (not "int") to str How can I install a previous version of Python 3 in macOS using homebrew? Flask at first run: Do not use the development server in a production environment TypeError: only integer scalar arrays can be converted to a scalar index with 1D numpy indices array What is the difference between Jupyter Notebook and JupyterLab? Pytesseract : "TesseractNotFound Error: tesseract is not installed or it's not in your path", how do I fix this? Could not install packages due to a "Environment error :[error 13]: permission denied : 'usr/local/bin/f2py'" How do I resolve a TesseractNotFoundError? Trying to merge 2 dataframes but get ValueError Authentication plugin 'caching_sha2_password' is not supported Python Pandas User Warning: Sorting because non-concatenation axis is not aligned

Questions with tensorflow tag:

Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation Module 'tensorflow' has no attribute 'contrib' Tensorflow 2.0 - AttributeError: module 'tensorflow' has no attribute 'Session' Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: How do I use TensorFlow GPU? Which TensorFlow and CUDA version combinations are compatible? Could not find a version that satisfies the requirement tensorflow pip3: command not found How to import keras from tf.keras in Tensorflow? Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 Tensorflow import error: No module named 'tensorflow' How to downgrade tensorflow, multiple versions possible? Can I run Keras model on gpu? How to check if keras tensorflow backend is GPU or CPU version? how to install tensorflow on anaconda python 3.6 What is the role of "Flatten" in Keras? Running Tensorflow in Jupyter Notebook Update TensorFlow Tensorflow r1.0 : could not a find a version that satisfies the requirement tensorflow ImportError: No module named tensorflow On Windows, running "import tensorflow" generates No module named "_pywrap_tensorflow" error Keras, How to get the output of each layer? What is the meaning of the word logits in TensorFlow? How to compile Tensorflow with SSE4.2 and AVX instructions? Installing TensorFlow on Windows (Python 3.6.x) Can Keras with Tensorflow backend be forced to use CPU or GPU at will? How to get Tensorflow tensor dimensions (shape) as int values? TensorFlow ValueError: Cannot feed value of shape (64, 64, 3) for Tensor u'Placeholder:0', which has shape '(?, 64, 64, 3)' Deep-Learning Nan loss reasons Cannot import keras after installation TensorFlow not found using pip How to get current available GPUs in tensorflow? How to find which version of TensorFlow is installed in my system? How to tell if tensorflow is using gpu acceleration from inside python shell? How can I run Tensorboard on a remote server? Tensorflow set CUDA_VISIBLE_DEVICES within jupyter Using Keras & Tensorflow with AMD GPU What is the difference between 'SAME' and 'VALID' padding in tf.nn.max_pool of tensorflow? How to run Tensorflow on CPU TensorFlow, "'module' object has no attribute 'placeholder'" Installing tensorflow with anaconda in windows How to get the dimensions of a tensor (in TensorFlow) at graph construction time? How to convert numpy arrays to standard TensorFlow format? In Tensorflow, get the names of all the Tensors in a graph TensorFlow: "Attempting to use uninitialized value" in variable initialization Disable Tensorflow debugging information What is logits, softmax and softmax_cross_entropy_with_logits? How to assign a value to a TensorFlow variable? How to prevent tensorflow from allocating the totality of a GPU memory? Convert a tensor to numpy array in Tensorflow?

Questions with gpu tag:

How do I use TensorFlow GPU? How to check if pytorch is using the GPU? NVIDIA NVML Driver/library version mismatch NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver How to get current available GPUs in tensorflow? How to tell if tensorflow is using gpu acceleration from inside python shell? Using Java with Nvidia GPUs (CUDA) Is it possible to run CUDA on AMD GPUs? How do I choose grid and block dimensions for CUDA kernels? Can I run CUDA on Intel's integrated graphics processor? Monitor the Graphics card usage Intro to GPU programming