Bigtable system tests passing against V2 endpoints #1930

tseaver · 2016-06-28T20:08:46Z

/cc @garye, @lesv, @sduskis, @tswast, @mbrukman, @jgeewax

See reopened #1918.

Note that 'location' is a required argument for new instances: we need it to pass to the automatically-created cluster belonging to a newly-created instance. Non-default 'serve_nodes' can be passed in the automatically-created cluster belonging to a newly-created instance.

The 'PartialRowsData' has state: 'Table.read_row()' ensures it is in the correct state before emitting the PRD.

gcloud/bigtable/client.py

        """Factory to create a instance associated with this client.

        :type instance_id: str
        :param instance_id: The ID of the instance.

+        :type location: string
+        :param location: location name, in form
+                         ``projects/<project>/locations/<location>``; used to


Addresses: #1930 (comment)

gcloud/bigtable/instance.py

@@ -199,17 +212,28 @@ class Instance(object):
    :param client: The client that owns the instance. Provides
                   authorization and a project ID.

+    :type location: string
+    :param location: location name, in form


It can synthesize the path using its client's project ID. Addresses: #1930 (comment)

tswast · 2016-06-28T20:58:26Z

@tseaver Once this goes in, can you start a PyPi release? Then I will be able to update the code samples to use this version.

tseaver · 2016-06-28T21:22:49Z

@tswast This PR is still targeting the 'bigtable-v2' feature branch. I also need to get the gcloud.bigtable.happybase code working, including its separate system tests, before merging the feature branch to master and making a release.

tswast · 2016-06-28T21:23:22Z

I get an error when I try to use the code from this branch.

$ python main.py swast-bigtable-examples my-cluster us-central1-c
D0628 14:22:01.934576886    6633 ev_posix.c:101]             Using polling engine: poll
Traceback (most recent call last):
  File "main.py", line 29, in <module>
    from gcloud import bigtable
  File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/__init__.py", line 18, in <module>
    from gcloud.bigtable.client import Client
  File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/client.py", line 34, in <module>
    from gcloud.bigtable._generated_v2 import (
  File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/_generated_v2/bigtable_instance_admin_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
ImportError: cannot import name descriptor

garye · 2016-06-28T21:29:17Z

Do you need a more recent version of google.protobuf?

On Tue, Jun 28, 2016 at 5:23 PM Tim Swast [email protected] wrote:

I get an error when I try to use the code from this branch.

$ python main.py swast-bigtable-examples my-cluster us-central1-c
D0628 14:22:01.934576886 6633 ev_posix.c:101] Using polling engine: poll
Traceback (most recent call last):
File "main.py", line 29, in
from gcloud import bigtable
File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/init.py", line 18, in
from gcloud.bigtable.client import Client
File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/client.py", line 34, in
from gcloud.bigtable._generated_v2 import (
File "/usr/local/google/home/swast/src/gcloud-python/gcloud/bigtable/_generated_v2/bigtable_instance_admin_pb2.py", line 6, in
from google.protobuf import descriptor as _descriptor
ImportError: cannot import name descriptor

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1930 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AACTr-qCHqJKAd8znyMxy0AiiNP7rjKhks5qQZDRgaJpZM4JAe64
.

tswast · 2016-06-28T21:31:58Z

@garye This is from a fresh virtualenv. I installed via python setup.py develop.

$ pip freeze | grep protobuf
protobuf==3.0.0b3

Version 3.0.0b3 appears to be the latest release.

tseaver · 2016-06-29T20:44:21Z

@tswast I cannot reproduce with the current HEAD of the bigtable-v2 branch:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/issue_1930
$ cd /tmp/issue_1930
$ mkdir src; cd src
$ git clone -b bigtable-v2 --depth 1 [email protected]:GoogleCloudPlatform/gcloud-python
Cloning into 'gcloud-python'...
remote: Counting objects: 409, done.
remote: Compressing objects: 100% (384/384), done.
remote: Total 409 (delta 77), reused 103 (delta 21), pack-reused 0
Receiving objects: 100% (409/409), 5.76 MiB | 392.00 KiB/s, done.
Resolving deltas: 100% (77/77), done.
Checking connectivity... done.
$ cd ..
$ bin/pip install -e src/gcloud-python/
Obtaining file:///tmp/tswast/src/gcloud-python
Collecting httplib2>=0.9.1 (from gcloud==0.16.0)
Collecting googleapis-common-protos (from gcloud==0.16.0)
Collecting oauth2client>=2.0.1 (from gcloud==0.16.0)
Collecting protobuf!=3.0.0.b2.post1,>=3.0.0b2 (from gcloud==0.16.0)
  Using cached protobuf-3.0.0b3-py2-none-any.whl
Collecting six (from gcloud==0.16.0)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting grpcio>=0.14.0 (from gcloud==0.16.0)
  Using cached grpcio-0.15.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax>=0.12.0 (from gcloud==0.16.0)
Collecting gax-google-pubsub-v1 (from gcloud==0.16.0)
Collecting rsa>=3.1.4 (from oauth2client>=2.0.1->gcloud==0.16.0)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.1->gcloud==0.16.0)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.1->gcloud==0.16.0)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages (from protobuf!=3.0.0.b2.post1,>=3.0.0b2->gcloud==0.16.0)
Collecting enum34>=1.0.4 (from grpcio>=0.14.0->gcloud==0.16.0)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio>=0.14.0->gcloud==0.16.0)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax>=0.12.0->gcloud==0.16.0)
Collecting grpc-google-pubsub-v1>=0.7.10 (from gax-google-pubsub-v1->gcloud==0.16.0)
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, enum34, futures, grpcio, ply, google-gax, grpc-google-pubsub-v1, gax-google-pubsub-v1, gcloud
  Running setup.py develop for gcloud
Successfully installed enum34-1.1.6 futures-3.0.5 gax-google-pubsub-v1-0.7.10 gcloud google-gax-0.12.1 googleapis-common-protos-1.1.0 grpc-google-pubsub-v1-0.7.10 grpcio-0.15.0 httplib2-0.9.2 oauth2client-2.2.0 ply-3.8 protobuf-3.0.0b3 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gcloud import bigtable
D0629 16:41:43.553773202   30989 ev_posix.c:101]             Using polling engine: poll
>>>

tseaver added 7 commits June 28, 2016 13:23

Work around borked gRPC generation for g.longrunning.operations.

3467d99

See reopened #1918.

Revise Bigtable system tests to fit instance-based patterns.

784fe01

CreateInstance operation name includes location.

5e85b5d

'Row/ColumnFamily._table' has '_instance'.

d2c9daf

V2 UpdateTable no longer returns operation.

a01d02e

'PartialRowData' no longer keeps 'committed' state.

9bdeef7

The 'PartialRowsData' has state: 'Table.read_row()' ensures it is in the correct state before emitting the PRD.

tseaver added the api: bigtable Issues related to the Bigtable API. label Jun 28, 2016

tseaver assigned daspecster and dhermes Jun 28, 2016

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jun 28, 2016

garye reviewed Jun 28, 2016
View reviewed changes

tseaver mentioned this pull request Jun 28, 2016

Implement Bigtable v2 API #1850

Closed

Typo fix.

893d451

Addresses: #1930 (comment)

mbrukman reviewed Jun 28, 2016
View reviewed changes

Switch to passing location ID, rather than full path, to instance.

d8a01c2

It can synthesize the path using its client's project ID. Addresses: #1930 (comment)

tseaver mentioned this pull request Jun 28, 2016

Convert gcloud.bigtable.happybase to V2 patterns #1931

Merged

Lint fixes.

6f4aced

tseaver merged commit 0c625c5 into googleapis:bigtable-v2 Jun 29, 2016

tseaver deleted the bigtable-v2-system_tests branch June 29, 2016 00:26

tseaver mentioned this pull request Jun 29, 2016

Prepare 0.17.0 release. #1940

Merged

theacodes unassigned daspecster Sep 28, 2018

theacodes unassigned dhermes Sep 28, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bigtable system tests passing against V2 endpoints #1930

Bigtable system tests passing against V2 endpoints #1930

tseaver commented Jun 28, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

tswast commented Jun 28, 2016

tseaver commented Jun 28, 2016

tswast commented Jun 28, 2016

garye commented Jun 28, 2016

tswast commented Jun 28, 2016

tseaver commented Jun 29, 2016

Bigtable system tests passing against V2 endpoints #1930

Bigtable system tests passing against V2 endpoints #1930

Conversation

tseaver commented Jun 28, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

tswast commented Jun 28, 2016

tseaver commented Jun 28, 2016

tswast commented Jun 28, 2016

garye commented Jun 28, 2016

tswast commented Jun 28, 2016

tseaver commented Jun 29, 2016