Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Needs protobuf minimum version 3.12.0 #305

Closed
anistark opened this issue Oct 7, 2020 · 6 comments · Fixed by #306
Closed

Needs protobuf minimum version 3.12.0 #305

anistark opened this issue Oct 7, 2020 · 6 comments · Fixed by #306
Assignees
@yoshi-automation
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: question Request for information or clarification. Not an issue.

Comments

@anistark
Copy link
Contributor

anistark commented Oct 7, 2020

If you are still having issues, please be sure to include as much information as possible:

Environment details

  • OS type and version: MacOS
  • Python version: python --version 3.8
  • pip version: pip --version 20.2.3
  • google-cloud-bigquery version: pip show google-cloud-bigquery 2.0.0

Steps to reproduce

  1. Just install the package, import somewhere and run it.

Code example

from google.cloud import bigquery

Stack trace

File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 570, in _GetFieldByName
    return message_descriptor.fields_by_name[field_name]
KeyError: 'proto3_optional'

...

   from google.cloud import bigquery
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
    from google.cloud.bigquery.client import Client
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/client.py", line 57, in <module>
    from google.cloud.bigquery import _pandas_helpers
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 36, in <module>
    from google.cloud.bigquery import schema
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery/schema.py", line 19, in <module>
    from google.cloud.bigquery_v2 import types
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/__init__.py", line 19, in <module>
    from .types.encryption_config import EncryptionConfiguration
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/types/__init__.py", line 18, in <module>
    from .encryption_config import EncryptionConfiguration
  File "/.../venv/lib/python3.8/site-packages/google/cloud/bigquery_v2/types/encryption_config.py", line 29, in <module>
    class EncryptionConfiguration(proto.Message):
  File "/.../venv/lib/python3.8/site-packages/proto/message.py", line 215, in __new__
    field=[i.descriptor for i in fields],
  File "/.../venv/lib/python3.8/site-packages/proto/message.py", line 215, in <listcomp>
    field=[i.descriptor for i in fields],
  File "/.../venv/lib/python3.8/site-packages/proto/fields.py", line 104, in descriptor
    self._descriptor = descriptor_pb2.FieldDescriptorProto(
  File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 509, in init
    field = _GetFieldByName(message_descriptor, field_name)
  File "/.../venv/lib/python3.8/site-packages/google/protobuf/internal/python_message.py", line 572, in _GetFieldByName
    raise ValueError('Protocol message %s has no "%s" field.' %
ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Oct 7, 2020
This was referenced Oct 7, 2020
Closed
Merged
@HemangChothani
Copy link
Contributor

@anistark google-cloud-bigquery has dependency of proto-plus as you can see below which use the protobuf.

python-bigquery/setup.py

Line 33 in d1eb8b3

"proto-plus >= 1.10.0",

proto-plus library has already mentioned a minimum version of protobuf, you can find here https://github.com/googleapis/proto-plus-python/blob/d045cbf058cbb8f4ca98dd06741270fcaee865be/setup.py#L39

@anistark
Copy link
Contributor Author

anistark commented Oct 7, 2020

And yet it somehow didn't install for me.

@HemangChothani
Copy link
Contributor

@anistark i have tried to install with fresh env and it's working fine, even i tried to install proto-plus and 'google-cloud-bigquery' with already installed older version of protobuf, it successfully uninstall the older version and install the latest version of library.

@HemangChothani HemangChothani added the type: question Request for information or clarification. Not an issue. label Oct 7, 2020
gcf-merge-on-green bot pushed a commit that referenced this issue Oct 16, 2020
@anistark
deps: add protobuf dependency (#306)
cebb5e0 
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes #305
gcf-merge-on-green bot pushed a commit that referenced this issue Oct 19, 2020
@release-please
chore: release 2.2.0 (#321)
82290c3 
🤖 I have created a release \*beep\* \*boop\* 
---
## [2.2.0](https://www.github.com/googleapis/python-bigquery/compare/v2.1.0...v2.2.0) (2020-10-19)


### Features

* add method api_repr for table list item ([#299](https://www.github.com/googleapis/python-bigquery/issues/299)) ([07c70f0](https://www.github.com/googleapis/python-bigquery/commit/07c70f0292f9212f0c968cd5c9206e8b0409c0da))
* add support for listing arima, automl, boosted tree, DNN, and matrix factorization models ([#328](https://www.github.com/googleapis/python-bigquery/issues/328)) ([502a092](https://www.github.com/googleapis/python-bigquery/commit/502a0926018abf058cb84bd18043c25eba15a2cc))
* add timeout paramter to load_table_from_file and it dependent methods ([#327](https://www.github.com/googleapis/python-bigquery/issues/327)) ([b0dd892](https://www.github.com/googleapis/python-bigquery/commit/b0dd892176e31ac25fddd15554b5bfa054299d4d))
* add to_api_repr method to Model ([#326](https://www.github.com/googleapis/python-bigquery/issues/326)) ([fb401bd](https://www.github.com/googleapis/python-bigquery/commit/fb401bd94477323bba68cf252dd88166495daf54))
* allow client options to be set in magics context ([#322](https://www.github.com/googleapis/python-bigquery/issues/322)) ([5178b55](https://www.github.com/googleapis/python-bigquery/commit/5178b55682f5e264bfc082cde26acb1fdc953a18))


### Bug Fixes

* make TimePartitioning repr evaluable ([#110](https://www.github.com/googleapis/python-bigquery/issues/110)) ([20f473b](https://www.github.com/googleapis/python-bigquery/commit/20f473bfff5ae98377f5d9cdf18bfe5554d86ff4)), closes [#109](https://www.github.com/googleapis/python-bigquery/issues/109)
* use version.py instead of pkg_resources.get_distribution ([#307](https://www.github.com/googleapis/python-bigquery/issues/307)) ([b8f502b](https://www.github.com/googleapis/python-bigquery/commit/b8f502b14f21d1815697e4d57cf1225dfb4a7c5e))


### Performance Improvements

* add size parameter for load table from dataframe and json methods ([#280](https://www.github.com/googleapis/python-bigquery/issues/280)) ([3be78b7](https://www.github.com/googleapis/python-bigquery/commit/3be78b737add7111e24e912cd02fc6df75a07de6))


### Documentation

* update clustering field docstrings ([#286](https://www.github.com/googleapis/python-bigquery/issues/286)) ([5ea1ece](https://www.github.com/googleapis/python-bigquery/commit/5ea1ece2d911cdd1f3d9549ee01559ce8ed8269a)), closes [#285](https://www.github.com/googleapis/python-bigquery/issues/285)
* update snippets samples to support version 2.0 ([#309](https://www.github.com/googleapis/python-bigquery/issues/309)) ([61634be](https://www.github.com/googleapis/python-bigquery/commit/61634be9bf9e3df7589fc1bfdbda87288859bb13))


### Dependencies

* add protobuf dependency ([#306](https://www.github.com/googleapis/python-bigquery/issues/306)) ([cebb5e0](https://www.github.com/googleapis/python-bigquery/commit/cebb5e0e911e8c9059bc8c9e7fce4440e518bff3)), closes [#305](https://www.github.com/googleapis/python-bigquery/issues/305)
* require pyarrow for pandas support ([#314](https://www.github.com/googleapis/python-bigquery/issues/314)) ([801e4c0](https://www.github.com/googleapis/python-bigquery/commit/801e4c0574b7e421aa3a28cafec6fd6bcce940dd)), closes [#265](https://www.github.com/googleapis/python-bigquery/issues/265)
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
@aylusltd
Copy link

Clean install on AWS Sagemaker CondaPython3 Kernel. Looks like same bug.

Collecting google-cloud-bigquery
  Downloading google_cloud_bigquery-2.6.1-py2.py3-none-any.whl (211 kB)
     |████████████████████████████████| 211 kB 15.9 MB/s eta 0:00:01
Requirement already satisfied: six<2.0.0dev,>=1.13.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (1.14.0)
Collecting proto-plus>=1.10.0
  Downloading proto-plus-1.13.0.tar.gz (44 kB)
     |████████████████████████████████| 44 kB 3.5 MB/s  eta 0:00:01
Collecting google-resumable-media<2.0dev,>=0.6.0
  Downloading google_resumable_media-1.2.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 6.2 MB/s  eta 0:00:01
Requirement already satisfied: protobuf>=3.12.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (3.12.2)
Collecting google-cloud-core<2.0dev,>=1.4.1
  Downloading google_cloud_core-1.5.0-py2.py3-none-any.whl (27 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.23.0
  Downloading google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
     |████████████████████████████████| 92 kB 14.7 MB/s eta 0:00:01
Collecting google-crc32c<2.0dev,>=1.0; python_version >= "3.5"
  Downloading google_crc32c-1.1.0-cp36-cp36m-manylinux2010_x86_64.whl (39 kB)
Requirement already satisfied: setuptools in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from protobuf>=3.12.0->google-cloud-bigquery) (46.1.3.post20200330)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
     |████████████████████████████████| 100 kB 13.2 MB/s ta 0:00:01
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.23.0)
Requirement already satisfied: pytz in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2019.3)
Collecting google-auth<2.0dev,>=1.21.1
  Downloading google_auth-1.24.0-py2.py3-none-any.whl (114 kB)
     |████████████████████████████████| 114 kB 68.9 MB/s eta 0:00:01
Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc"
  Downloading grpcio-1.34.0-cp36-cp36m-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 62.5 MB/s eta 0:00:01
Requirement already satisfied: cffi>=1.0.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)
Requirement already satisfied: idna<3,>=2.5 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.0.4)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 64.1 MB/s eta 0:00:01
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.0-py3-none-any.whl (12 kB)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.4.2)
Requirement already satisfied: pycparser in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (0.4.8)
Building wheels for collected packages: proto-plus
  Building wheel for proto-plus (setup.py) ... done
  Created wheel for proto-plus: filename=proto_plus-1.13.0-py3-none-any.whl size=41591 sha256=185a880a276da72cb83ea7d6a1abf2b33be68ff357c6fe11e2b324d313f683ba
  Stored in directory: /home/ec2-user/.cache/pip/wheels/5a/76/e1/dd2dc8192e9a6b687d8c7550acc7c8edf059a4f16296e7ea89
Successfully built proto-plus
Installing collected packages: proto-plus, google-crc32c, google-resumable-media, googleapis-common-protos, pyasn1-modules, cachetools, google-auth, grpcio, google-api-core, google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-4.2.0 google-api-core-1.24.1 google-auth-1.24.0 google-cloud-bigquery-2.6.1 google-cloud-core-1.5.0 google-crc32c-1.1.0 google-resumable-media-1.2.0 googleapis-common-protos-1.52.0 grpcio-1.34.0 proto-plus-1.13.0 pyasn1-modules-0.2.8
WARNING: You are using pip version 20.0.2; however, version 20.3.3 is available.
You should consider upgrading via the '/home/ec2-user/anaconda3/envs/python3/bin/python -m pip install --upgrade pip' command.

ValueError                                Traceback (most recent call last)
<ipython-input-4-b3914349486e> in <module>
----> 1 from google.cloud import bigquery
      2 query_string = """
      3 SELECT * FROM `<---REDACTED DB NAME--->` LIMIT 10
      4 """
      5 client = bigquery.Client()

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     55 from google.cloud.bigquery._helpers import _verify_job_config_type
     56 from google.cloud.bigquery._http import Connection
---> 57 from google.cloud.bigquery import _pandas_helpers
     58 from google.cloud.bigquery.dataset import Dataset
     59 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 from six.moves import collections_abc
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model_reference import ModelReference
     20 from .standard_sql import (

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.

@anzen-l
Copy link

anzen-l commented Dec 28, 2020

Clean install on AWS Sagemaker CondaPython3 Kernel. Looks like same bug.

Collecting google-cloud-bigquery
  Downloading google_cloud_bigquery-2.6.1-py2.py3-none-any.whl (211 kB)
     |████████████████████████████████| 211 kB 15.9 MB/s eta 0:00:01
Requirement already satisfied: six<2.0.0dev,>=1.13.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (1.14.0)
Collecting proto-plus>=1.10.0
  Downloading proto-plus-1.13.0.tar.gz (44 kB)
     |████████████████████████████████| 44 kB 3.5 MB/s  eta 0:00:01
Collecting google-resumable-media<2.0dev,>=0.6.0
  Downloading google_resumable_media-1.2.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 6.2 MB/s  eta 0:00:01
Requirement already satisfied: protobuf>=3.12.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-cloud-bigquery) (3.12.2)
Collecting google-cloud-core<2.0dev,>=1.4.1
  Downloading google_cloud_core-1.5.0-py2.py3-none-any.whl (27 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.23.0
  Downloading google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
     |████████████████████████████████| 92 kB 14.7 MB/s eta 0:00:01
Collecting google-crc32c<2.0dev,>=1.0; python_version >= "3.5"
  Downloading google_crc32c-1.1.0-cp36-cp36m-manylinux2010_x86_64.whl (39 kB)
Requirement already satisfied: setuptools in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from protobuf>=3.12.0->google-cloud-bigquery) (46.1.3.post20200330)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
     |████████████████████████████████| 100 kB 13.2 MB/s ta 0:00:01
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.23.0)
Requirement already satisfied: pytz in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2019.3)
Collecting google-auth<2.0dev,>=1.21.1
  Downloading google_auth-1.24.0-py2.py3-none-any.whl (114 kB)
     |████████████████████████████████| 114 kB 68.9 MB/s eta 0:00:01
Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc"
  Downloading grpcio-1.34.0-cp36-cp36m-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 62.5 MB/s eta 0:00:01
Requirement already satisfied: cffi>=1.0.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)
Requirement already satisfied: idna<3,>=2.5 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.0.4)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 64.1 MB/s eta 0:00:01
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.0-py3-none-any.whl (12 kB)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (3.4.2)
Requirement already satisfied: pycparser in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= "3.5"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.21.1->google-api-core[grpc]<2.0.0dev,>=1.23.0->google-cloud-bigquery) (0.4.8)
Building wheels for collected packages: proto-plus
  Building wheel for proto-plus (setup.py) ... done
  Created wheel for proto-plus: filename=proto_plus-1.13.0-py3-none-any.whl size=41591 sha256=185a880a276da72cb83ea7d6a1abf2b33be68ff357c6fe11e2b324d313f683ba
  Stored in directory: /home/ec2-user/.cache/pip/wheels/5a/76/e1/dd2dc8192e9a6b687d8c7550acc7c8edf059a4f16296e7ea89
Successfully built proto-plus
Installing collected packages: proto-plus, google-crc32c, google-resumable-media, googleapis-common-protos, pyasn1-modules, cachetools, google-auth, grpcio, google-api-core, google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-4.2.0 google-api-core-1.24.1 google-auth-1.24.0 google-cloud-bigquery-2.6.1 google-cloud-core-1.5.0 google-crc32c-1.1.0 google-resumable-media-1.2.0 googleapis-common-protos-1.52.0 grpcio-1.34.0 proto-plus-1.13.0 pyasn1-modules-0.2.8
WARNING: You are using pip version 20.0.2; however, version 20.3.3 is available.
You should consider upgrading via the '/home/ec2-user/anaconda3/envs/python3/bin/python -m pip install --upgrade pip' command.

ValueError                                Traceback (most recent call last)
<ipython-input-4-b3914349486e> in <module>
----> 1 from google.cloud import bigquery
      2 query_string = """
      3 SELECT * FROM `<---REDACTED DB NAME--->` LIMIT 10
      4 """
      5 client = bigquery.Client()

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     55 from google.cloud.bigquery._helpers import _verify_job_config_type
     56 from google.cloud.bigquery._http import Connection
---> 57 from google.cloud.bigquery import _pandas_helpers
     58 from google.cloud.bigquery.dataset import Dataset
     59 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 from six.moves import collections_abc
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model_reference import ModelReference
     20 from .standard_sql import (

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.

I have the same problem with big query & jupyter notebooks

@Tsakunelson
Copy link

Tsakunelson commented Mar 17, 2021 •
edited
Loading

Hi All,
I am still having this same error, though have all the dependencies installed on a Jupyter Notebook in AWS Sagemaker including:

Protobuf = 3.13.0```

For Google BigQuery 2.12.0

But I get the following error:

```---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-017a44a94d83> in <module>
----> 1 df = gbq.read_gbq('SELECT * FROM `cmg-consumer-analytics.122142903.ga_sessions_20210314` LIMIT 10', project_id='cmg-consumer-analytics')

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/io/gbq.py in read_gbq(query, project_id, index_col, col_order, reauth, auth_local_webserver, dialect, location, configuration, credentials, use_bqstorage_api, private_key, verbose, progress_bar_type)
    161     DataFrame.to_gbq : Write a DataFrame to Google BigQuery.
    162     """
--> 163     pandas_gbq = _try_import()
    164 
    165     kwargs: Dict[str, Union[str, bool]] = {}

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/io/gbq.py in _try_import()
     15         "See the docs: https://pandas-gbq.readthedocs.io."
     16     )
---> 17     pandas_gbq = import_optional_dependency("pandas_gbq", extra=msg)
     18     return pandas_gbq
     19 

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/compat/_optional.py in import_optional_dependency(name, extra, raise_on_missing, on_version)
     87     )
     88     try:
---> 89         module = importlib.import_module(name)
     90     except ImportError:
     91         if raise_on_missing:

~/anaconda3/envs/python3/lib/python3.6/importlib/__init__.py in import_module(name, package)
    124                 break
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 
    128 

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _gcd_import(name, package, level)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _find_and_load(name, import_)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)

~/anaconda3/envs/python3/lib/python3.6/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas_gbq/__init__.py in <module>
----> 1 from .gbq import to_gbq, read_gbq, Context, context  # noqa
      2 
      3 from ._version import get_versions
      4 
      5 versions = get_versions()

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas_gbq/gbq.py in <module>
     10 try:
     11     from google.api_core import exceptions as google_exceptions
---> 12     from google.cloud import bigquery
     13 except ImportError:  # pragma: NO COVER
     14     bigquery = None

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = bigquery_version.__version__
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/client.py in <module>
     54 from google.cloud.bigquery._helpers import _verify_job_config_type
     55 from google.cloud.bigquery._http import Connection
---> 56 from google.cloud.bigquery import _pandas_helpers
     57 from google.cloud.bigquery.dataset import Dataset
     58 from google.cloud.bigquery.dataset import DatasetListItem

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     34     pyarrow = None
     35 
---> 36 from google.cloud.bigquery import schema
     37 
     38 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery/schema.py in <module>
     17 import collections
     18 
---> 19 from google.cloud.bigquery_v2 import types
     20 
     21 

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/__init__.py in <module>
     17 
     18 
---> 19 from .types.encryption_config import EncryptionConfiguration
     20 from .types.model import DeleteModelRequest
     21 from .types.model import GetModelRequest

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/__init__.py in <module>
     16 #
     17 
---> 18 from .encryption_config import EncryptionConfiguration
     19 from .model import (
     20     DeleteModelRequest,

~/anaconda3/envs/python3/lib/python3.6/site-packages/google/cloud/bigquery_v2/types/encryption_config.py in <module>
     27 
     28 
---> 29 class EncryptionConfiguration(proto.Message):
     30     r"""
     31 

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in __new__(mcls, name, bases, attrs)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/message.py in <listcomp>(.0)
    212         desc = descriptor_pb2.DescriptorProto(
    213             name=name,
--> 214             field=[i.descriptor for i in fields],
    215             oneof_decl=[
    216                 descriptor_pb2.OneofDescriptorProto(name=i) for i in oneofs.keys()

~/anaconda3/envs/python3/lib/python3.6/site-packages/proto/fields.py in descriptor(self)
    109                 type_name=type_name,
    110                 json_name=self.json_name,
--> 111                 proto3_optional=self.optional,
    112             )
    113 

ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.```



Why does the error still prevail? am I missing something else? Feedback appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yoshi-automation yoshi-automation

Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

deps: add protobuf dependency anistark/python-bigquery
6 participants
@aylusltd @anistark @Tsakunelson @yoshi-automation @HemangChothani @anzen-l