Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bigtable table.getRows() silently failing on queries > 4000 cells #1443

Closed
imathews opened this issue Jul 21, 2016 · 8 comments · Fixed by #1444
Closed

Bigtable table.getRows() silently failing on queries > 4000 cells #1443

imathews opened this issue Jul 21, 2016 · 8 comments · Fixed by #1444
Assignees
Labels
api: bigtable Issues related to the Bigtable API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@imathews
Copy link

I'm fairly new to Bigtable, and my apologies if this is an issue on my end. I have a table where all rows have 84 cells; when I submit a getRows request with a limit >= 48 (total > 4000 cells) I get no error and no response. With a lower limit, everything works perfectly. Looking at my networking, it seems like there is data coming in, but otherwise I don't have much else to go off of.

This is happening both when I use the streaming and the callback APIs. Passing in a filter to reduce the number of returned columns doesn't change this behavior.

Many thanks for getting the Bigtable support out the door!

Environment details

OSX @10.11.4
node @6.2.0
npm @3.8.9
gcloud-node @0.37.0

Steps to reproduce

const table = bigtable.table('data');
const rows = [];
const rowCount = 48; //Works <= 47, silently fails >= 48. Each row has 84 cells.

table.getRows({
    limit: rowCount,
})
    .on('error', (err) => {
        console.error(err);
    })
    .on('data', (row) => {
        console.log('got row!');
        rows.push(row);
    })
    .on('end', () => {
        console.log(`Complete! Retrieved ${rows.length}.`);
        //Do something with rows
    });

// Also fails when using callback API
@callmehiphop callmehiphop added the api: bigtable Issues related to the Bigtable API. label Jul 21, 2016
@callmehiphop callmehiphop self-assigned this Jul 21, 2016
@callmehiphop
Copy link
Contributor

@imathews thanks for reporting this! I've managed to reproduce this locally, so definitely not an issue on your end.

@murgatroid99 not sure if this is a implementation error on our side or not, but from what I can tell the ClientReadableStream#_read method is hanging when we have 48 or more results. I'm going to continue to dig in and hopefully provide more details, but any assistance you can provide in the meantime would be greatly appreciated!

@murgatroid99
Copy link

Can you provide the output of running the client with the environment variable GRPC_TRACE=all? (using only test data, because that dumps everything).

@callmehiphop
Copy link
Contributor

@murgatroid99 actually it appears to be an issue with one of our dependencies, I made a few local changes and grpc seems to be functioning as expected - my apologies for looping you in prematurely 😦

@murgatroid99
Copy link

No problem. I'm glad to hear it's resolved.

@callmehiphop
Copy link
Contributor

@murgatroid99 is there any specific reason why grpc does not emit a status event (or similar) when we call startBatch within a server stream request? A metadata event is emitted, but it doesn't contain a status code, etc. letting me know the request was a success.

@murgatroid99
Copy link

I'm not exactly sure which startBatch call you're referring to, but that metadata event is triggered at the beginning of the call, before the server sends any responses. The status, on the other hand, comes in at the end of the call, after the server has finished sending responses.

@callmehiphop
Copy link
Contributor

@murgatroid99 Gotcha, thanks! I apologize if my questions seem off, as I am not super familiar with the grpc code base. Essentially I'm looking for a way to confirm that the request went through successfully and that the server is going to process results, I think my issue was that I assumed the startBatch calls within makeServerStreamRequest made a request to the server (which looks to be incorrect?).

@murgatroid99
Copy link

So, startBatch is used for any instance of sending anything, including metadata, messages, and the status. There are at least a couple of calls to it in or associated with makeServerStreamRequest.

In general, I think receiving metadata is a good signal that the server has started to process the request.

@callmehiphop callmehiphop added the type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. label Jul 21, 2016
@callmehiphop callmehiphop mentioned this issue Jul 21, 2017
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants