Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing metrics in BigTable #1314

Open
lum opened this issue Aug 14, 2018 · 1 comment
Open

Missing metrics in BigTable #1314

lum opened this issue Aug 14, 2018 · 1 comment
Labels

Comments

@lum
Copy link

lum commented Aug 14, 2018

Hi,

I currently have an environment that uses OpenTSDB and HBase. I wanted to try out using OpenTSDB with BigTable.
I've replicated my metric setup but instead of having OpenTSDB write to HBase, I have it writing to a BigTable instance.

My current opentsdb.conf looks like:
google.bigtable.project.id = xxx
google.bigtable.instance.id = xxx
google.bigtable.zone.id = xxx
hbase.client.connection.impl = com.google.cloud.bigtable.hbase1_2.BigtableConnection
google.bigtable.auth.service.account.enable = true
google.bigtable.auth.json.keyfile = xxx
google.bigtable.app_profile.id = xxx
google.bigtable.grpc.channel.count = 8
tsd.network.port = 4242
tsd.core.auto_create_metrics = true
tsd.http.request.enable_chunked = true
tsd.storage.fix_duplicates = true
tsd.storage.enable_compaction = false
tsd.storage.max_tags = 12
tsd.http.staticroot = /root/opentsdb/build/staticroot
tsd.http.cachedir = /tmp/opentsdb
tsd.network.keep_alive = true
tsd.core.meta.enable_realtime_ts = true
tsd.core.meta.enable_realtime_uid = true
tsd.core.meta.enable_tsuid_tracking = true
tsd.http.request.cors_domains = *
tsd.http.request.max_chunk = 5242880

When using this configuration, I noticed that my Write requests / sec to BigTabe is very low compared to what I am seeing with HBase.
When trying to use the OpenTSDB Web UI to query metrics in BigTable, I am either seeing no data points returned or missing data points when compared to the same query using OpenTSDB against HBase.

I've also tried updating the BigTable client that OpenTSDB is using as suggested by GoogleCloudPlatform/opentsdb-bigtable#2 , but still no luck.

Thanks,

Steve

@manolama
Copy link
Member

Any log files or data? We've recently updated to a new AsyncBigtable client and that should have helped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants