[python] ImportError: No module named google.protobuf

I am following this guide (https://developers.google.com/protocol-buffers/docs/pythontutorial) and using the exact sample of addressbook.proto.

Post the content of compiler generated addressbook_pb2.py file as well. When I run the following simple program, there is error saying, cannot find google.protobuf, any ideas how to resolve this issue? Thanks.

BTW, using Python 2.7 on Mac OSX.

from addressbook_pb2 import Person

p = Person()
p.email = "abc"

print p.email

Here is the automated generated file addressbook_pb2.py,

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: addressbook.proto

import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor.FileDescriptor(
  name='addressbook.proto',
  package='tutorial',
  syntax='proto2',
  serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\"\xda\x01\n\x06Person\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\n\n\x02id\x18\x02 \x02(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12+\n\x05phone\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x1aM\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x02(\t\x12.\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType:\x04HOME\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06person\x18\x01 \x03(\x0b\x32\x10.tutorial.Person')
)
_sym_db.RegisterFileDescriptor(DESCRIPTOR)



_PERSON_PHONETYPE = _descriptor.EnumDescriptor(
  name='PhoneType',
  full_name='tutorial.Person.PhoneType',
  filename=None,
  file=DESCRIPTOR,
  values=[
    _descriptor.EnumValueDescriptor(
      name='MOBILE', index=0, number=0,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='HOME', index=1, number=1,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='WORK', index=2, number=2,
      options=None,
      type=None),
  ],
  containing_type=None,
  options=None,
  serialized_start=207,
  serialized_end=250,
)
_sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE)


_PERSON_PHONENUMBER = _descriptor.Descriptor(
  name='PhoneNumber',
  full_name='tutorial.Person.PhoneNumber',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
      number=2, type=14, cpp_type=8, label=1,
      has_default_value=True, default_value=1,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=128,
  serialized_end=205,
)

_PERSON = _descriptor.Descriptor(
  name='Person',
  full_name='tutorial.Person',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='name', full_name='tutorial.Person.name', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='id', full_name='tutorial.Person.id', index=1,
      number=2, type=5, cpp_type=1, label=2,
      has_default_value=False, default_value=0,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='email', full_name='tutorial.Person.email', index=2,
      number=3, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='phone', full_name='tutorial.Person.phone', index=3,
      number=4, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[_PERSON_PHONENUMBER, ],
  enum_types=[
    _PERSON_PHONETYPE,
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=32,
  serialized_end=250,
)


_ADDRESSBOOK = _descriptor.Descriptor(
  name='AddressBook',
  full_name='tutorial.AddressBook',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='person', full_name='tutorial.AddressBook.person', index=0,
      number=1, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=252,
  serialized_end=299,
)

_PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
_PERSON_PHONENUMBER.containing_type = _PERSON
_PERSON.fields_by_name['phone'].message_type = _PERSON_PHONENUMBER
_PERSON_PHONETYPE.containing_type = _PERSON
_ADDRESSBOOK.fields_by_name['person'].message_type = _PERSON
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK

Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict(

  PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
    DESCRIPTOR = _PERSON_PHONENUMBER,
    __module__ = 'addressbook_pb2'
    # @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
    ))
  ,
  DESCRIPTOR = _PERSON,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.Person)
  ))
_sym_db.RegisterMessage(Person)
_sym_db.RegisterMessage(Person.PhoneNumber)

AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
  DESCRIPTOR = _ADDRESSBOOK,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.AddressBook)
  ))
_sym_db.RegisterMessage(AddressBook)


# @@protoc_insertion_point(module_scope)

Edit 1,

Tried pip install protobuf, met with the following error,

Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)

Here is the output of python version,

python -V
Python 2.7.11 :: Continuum Analytics, Inc.

**Edit 2 **

Post exact error message,

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Users/foo/personal/featureExtraction/protobuf_test.py", line 1, in <module>
    from addressbook_pb2 import Person
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/foo/personal/featureExtraction/addressbook_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
ImportError: No module named google.protobuf

Edit 3,

error message when import google,

Edit 4,

Output of which pip,

which pip
/Users/foo/miniconda2/bin/pip

Output of sys.executable,

/Users/foo/anaconda/bin/python

Edit 5,

foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install protobuf
Password:
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)
foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install google
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): google in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /Users/foo/miniconda2/lib/python2.7/site-packages (from google)

I had this issue when using the Python wrapper for DGraph DB, which was somehow fixed by this commit (perhaps of use to someone).

To find where the name google clashes .... try this:

python3

then >>> help('google')

... I got info about google-auth:

NAME
    google

PACKAGE CONTENTS
    auth (package)
    oauth2 (package)

Also then try

pip show google-auth

Then

sudo pip3 uninstall google-auth

... and re-try >>> help('google')

I then see protobuf:

NAME
    google

PACKAGE CONTENTS
    protobuf (package)

I also have this issue and have been looking into it for a long time. It seems that there is no such problem on python 3+. The problem is actually on google.protobuf

Solution 1:

pip uninstall protobuf
pip uninstall google
pip install google
pip install protobuf
pip install google-cloud

Solution 2:

create an __init__.py in "google" folder.

cd /path/to/your/env/lib/python2.7/site-packages/google
touch __init__.py

Hopefully it will work.

The reason for this would be mostly due to the evil command pip install google. I was facing a similar issue for google-cloud, but the same steps are true for protobuf as well. Both of our issues deal with a namespace conflict over the 'google' namespace.

If you executed the pip install google command like I did then you are in the correct place. The google package is actually not owned by Google which can be confirmed by the command pip show google which outputs:

 Name: google
 Version: 1.9.3
 Summary: Python bindings to the Google search engine.
 Home-page: http://breakingcode.wordpress.com/
 Author: Mario Vilas
 Author-email: [email protected]
 License: UNKNOWN
 Location: <Path where this package is installed>
 Requires: beautifulsoup4

Because of this package, the google namespace is reserved and coincidentally google-cloud also expects namespace google > cloud and it results in a namespace collision for these two packages.

See in below screenshot namespace of google-protobuf as google > protobuf

Solution :- Unofficial google package need to be uninstalled which can be done by using pip uninstall google after this you can reinstall google-cloud using pip install google-cloud or protobuf using pip install protobuf

FootNotes :- Assuming you have installed the unofficial google package by mistake and you don't actually need it along with google-cloud package. If you need both unofficial google and google-cloud above solution won't work.

Furthermore, the unofficial 'google' package installs with it 'soupsieve' and 'beautifulsoup4'. You may want to also uninstall those packages.

Let me know if this solves your particular issue.

This solved my problem with google.protobuf import in Tensorflow and Python 3.7.5 that i had yesterday.

Check where is protobuf

pip show protobuf

If it is installed you will get something like this

Name: protobuf
Version: 3.6.1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: None
Author-email: None
License: 3-Clause BSD License
Location: /usr/lib/python3/dist-packages
Requires: 
Required-by: tensorflow, tensorboard

(If not, run pip install protobuf )

Now move into the location folder.

cd /usr/lib/python3/dist-packages

Now run

touch google/__init__.py

You should run:

pip install protobuf

That will install Google protobuf and after that you can run that Python script.

As per this link.

On Mac OS - Installing tensorflow 1.3 - it will automatically remove other protobuf installs and install protobuf 3.4. However, this does not work and neither does installing or downgrading to any other protobuf version.

However I found a solution. Not sure why this works - but on Mac OS this solved it.

pip install google

I got the same error message as in the title, but in my case import google was working and import google.protobuf wasn't (on python3.5, ubuntu 16.04).

It turned out that I've installed python3-google-apputils package (using apt) and it was installed to '/usr/lib/python3/dist-packages/google/apputils/', while protobuf (which was installed using pip) was in "/usr/lib/python3.5/dist-packages/google/protobuf/" - and it was a "google" namespace collapse.

Uninstalling google-apputils (from apt, and reinstalling it using pip) solved the problem.

sudo apt remove python3-google-apputils
sudo pip3 install google-apputils

if protobuf is installed then import it like this

pip install protobuf

import google.protobuf

If you are a windows user and try to start py-script in cmd - don't forget to type python before filename.

python script.py

I have "No module named google" error if forget to type it.

I encountered the same situation. And I find out it is because the pip should be updated. It may be the same reason for your problem.

When pip tells you that you already have protobuf, but PyCharm (or other) tells you that you don't have it, it means that pip and PyCharm are using a different Python interpreter. This is a very common issue, especially on a Mac, with no standard Python package management.

The best way to completely eliminate such issues is using a virtualenv per Python project, which is essentially a directory of Python packages and environment variable settings to isolate the Python environment of the project from everything else.

Create a virtualenv for your project like this:

cd project
virtualenv --distribute virtualenv -p /path/to/python/executable

This creates a directory called virtualenv inside your project. (Make sure to configure your VCS (for example Git) to ignore this directory.)

To install packages in this virtualenv, you need to activate the environment variable settings:

. virtualenv/bin/activate

Verify that pip will use the right Python executable inside the virtualenv, by running pip -V. It should tell you the Python library path used, which should be inside the virtualenv.

Now you can use pip to install protobuf as you did.

And finally, you need to make PyCharm use this virtualenv instead of the system libraries. Somewhere in the project settings you can configure an interpreter for the project, select the Python executable inside the virtualenv.