(latest version) write method does not work #467

tcloaa · 2017-06-16T01:47:02Z

I have the issue here..

I only would like to write single point, so I may need the write method. (instead of write_points)

And my JSON body is like:

json_body = {
                "measurement": "message",
                "tags": {
                    "tag1": "99999",
                    "tag2": "99999"
                },
                "fields": {
                    "field1": 9999,
                    "field2": 99999
                },
                "time": 1496625370893587000
            }

With calling method:

client.write(json_body, protocol=u'json')

And I got error:

  File "/usr/local/lib/python2.7/dist-packages/influxdb/line_protocol.py", line 121, in make_lines
    for point in data['points']:
KeyError: u'points'

The text was updated successfully, but these errors were encountered:

tcloaa · 2017-06-16T02:12:45Z

It is really weird that def write() has to call:

if protocol == 'json':
    data = make_lines(data, precision).encode('utf-8')

where make_lines:

for point in data['points']:

If I would only want to write 1 single point, then my json(dict) shall naturally look like:

 json_body = {
                "measurement": "message",
                "tags": {
                    "tag1": "99999",
                    "tag2": "99999"
                },
                "fields": {
                    "field1": 9999,
                    "field2": 99999
                },
                "time": 1496625370893587000
            }

sebito91 · 2017-06-16T17:59:50Z

Thanks for raising this, and sorry for the delay in getting back to you. Will look at this and address ASAP.

tcloaa · 2017-06-17T01:10:14Z

Thanks,

Currently, for your write, it says it requires a dict, so if we try this:

json_body = {
                "measurement": "message",
                "tags": {
                    "tag1": "99999",
                    "tag2": "99999"
                },
                "fields": {
                    "field1": 9999,
                    "field2": 99999
                },
                "time": 1496625370893587000
            }

Then in client.write(), we shall call this function client.write([json_body]) (it works!), which becomes a list again...

sebito91 · 2017-06-17T05:41:56Z

For sanity, can you please confirm the version of the app you're using? I've dug into this some more and the main issue is the order of your data entry...to be fair, this is not documented properly at the moment, sorry! :/

The new syntax is expected as follows, where the tags are listed outside of all points for which they are applicable (which include the required fields and optional time):

    json_body = {
        "tags": {
            "empty_tag": "",
            "none_tag": None,
            "integer_tag": 2,
            "string_tag": "hello"
            },
        "points": [
            {
                "measurement": "test",
                "fields": {
                    "string_val": "hello!",
                    "int_val": 1,
                    "float_val": 1.1,
                    "none_field": None,
                    "bool_val": True,
                    },
                "time": 1497678083000000000,
                }
            ]
        }

    ret = newClient.write(json_body, params={'db': 'sebtest'})
    print ret

Also, you don't need to specify the protocol unless you're sending the official line protocol; the default is expecting json data.

Do you want to give this another shot with the revised format and let us know?

tcloaa · 2017-06-17T06:40:37Z

I think I am using the latest version of the client. (apt-get install)．

IMO, the best format shall be:

json_body = {
                "measurement": "message",
                "tags": {
                    "tag1": "99999",
                    "tag2": "99999"
                },
                "fields": {
                    "field1": 9999,
                    "field2": 99999
                },
                "time": 1496625370893587000
            }

just like a normal json (dict).

And therefore we can use newClient.write(json_body). (for now I have to call it like oldClient.write([json_body]) even for writing 1 record.

To be frankly speaking, the design of

    json_body = {
        "tags": {
            "empty_tag": "",
            "none_tag": None,
            "integer_tag": 2,
            "string_tag": "hello"
            },
        "points": [
            {
                "measurement": "test",
                "fields": {
                    "string_val": "hello!",
                    "int_val": 1,
                    "float_val": 1.1,
                    "none_field": None,
                    "bool_val": True,
                    },
                "time": 1497678083000000000,
                }
            ]
        }

is confusing since in InfluxDB, we only have time, tags, field, measurement. (no bother to add points and make access to key-value difficult here. )

sebito91 · 2017-06-17T14:31:27Z

The standard, supported method for writing data to InfluxDB (and the TICK stack) is to write line protocol. The JSON endpoint is deprecated, but support remains in clients like influxdb-python to help users seamlessly cut over. This is why you see the client calling make_lines before actually formatting the http request. The limitation of the old schema is the ability to write multiple points, aka lines or groups of measurements in one json_body. With the new setup, you can define multiple points that get converted to proper line protocol and get sent to the backend. In all cases, tags are optional but points are required; the new structure would allow you to define a particular tagset across a heap of measurements if you wanted.

In either scenario you should not need to pass your json_body as a list. The default expectation is json and will parse the data accordingly (provided it's valid syntax and matches the new structure).

tcloaa · 2017-06-26T10:22:56Z

Hi,
Thanks for the reply.

May I ask that,

How can I upgrade InfluxDB-Python so that I can catch up with this new setup?
for method write, is it always connecting and disconnecting the DB all the time? If I may have loop to sequentially insert data into InfluxDB, will it be too inefficient?

Thanks,

sebito91 · 2017-07-06T04:30:56Z

We've actually added the instructions to update in the README, but you can also do it as listed here.

For your second question, the connections that are established are not long-running TCP connections. Instead you're creating an HTTP POST that has the keep-alive header set but only for the duration of the write itself. Each call to the write function is idempotent and returns when a retcode is received for the HTTP POST. In InfluxDB, the retcode is any one of the following:

Here is an example:

23:21:30.086209 IP (tos 0x0, ttl 64, id 663, offset 0, flags [DF], proto TCP (6), length 432)
    localhost.localdomain.52844 > localhost.localdomain.d-s-n: Flags [P.], cksum 0xffa4 (incorrect -> 0xdf53), seq 381:761, ack 174, win 350, options [nop,nop,TS val 3759584023 ecr 3759530578], length 380
E.....@[email protected].....^.......^.......
.......RPOST /write?db=sebtest HTTP/1.1
Host: localhost:8086
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: text/plain
User-Agent: python-requests/2.17.3
Content-type: application/octet-stream
Content-Length: 115
Authorization: Basic cm9vdDpyb290

test,integer_tag=2,string_tag=hello bool_val=True,float_val=1.1,int_val=1i,string_val="hello!" 1497678083000000000

23:21:30.100862 IP (tos 0x0, ttl 64, id 59040, offset 0, flags [DF], proto TCP (6), length 225)
    localhost.localdomain.d-s-n > localhost.localdomain.52844: Flags [P.], cksum 0xfed5 (incorrect -> 0x5b62), seq 174:347, ack 761, win 359, options [nop,nop,TS val 3759584038 ecr 3759584023], length 173
E.....@[email protected].......
...&....HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 95ce2f30-6202-11e7-8016-000000000000
X-Influxdb-Version: 1.2.4
Date: Thu, 06 Jul 2017 04:21:30 GMT


23:21:30.100912 IP (tos 0x0, ttl 64, id 664, offset 0, flags [DF], proto TCP (6), length 52)
    localhost.localdomain.52844 > localhost.localdomain.d-s-n: Flags [.], cksum 0xfe28 (incorrect -> 0x51dc), seq 761, ack 347, win 359, options [nop,nop,TS val 3759584038 ecr 3759584038], length 0
E..4..@.@.:*.........l.........F...g.(.....
...&...&

sebito91 · 2017-07-06T04:31:41Z

@aviau READY TO CLOSE

ferhatguneri · 2018-12-19T14:15:20Z

Hello,

I'v tried a lot of times but I still get error Below. Could you please help me about that?

Here is code and error

payload = {
"measurement": "virtualmachine",
"tags": {
    "name": name,
    "instanceUuid": instanceUuid
},
"time": datetime.datetime.now().isoformat(),
"fields": {
            "version": version,
            "template": template,
            "memorySizeMB": memorySizeMB,
            "numCpu":numCpu,
            "numEthernetCards": numEthernetCards,
            "numVirtualDisks": numVirtualDisks,
            "committedStorage": committedStorage,
            "uncommittedStorage": uncommittedStorage,
            "powerState":powerState,
            "tools_version": tools_version,
            "ip_address": ip_address,
            "hostName": hostName,
            "vmPathName": vmPathName,
            "guestFullName": guestFullName
}

}
print payload
client.write([payload])

Traceback (most recent call last):
File "influxdb/test.py", line 107, in
print_vm_info(child)
File "influxdb/test.py", line 92, in print_vm_info
client.write([payload])
File "/usr/local/lib/python2.7/dist-packages/influxdb/client.py", line 315, in write
data = make_lines(data, precision).encode('utf-8')
File "/usr/local/lib/python2.7/dist-packages/influxdb/line_protocol.py", line 126, in make_lines
static_tags = data.get('tags')
AttributeError: 'list' object has no attribute 'get'

a-l-t-e-r · 2019-08-08T21:36:00Z

Same problem here at latest version. No fix yet.

tcloaa changed the title ~~write method does not work~~ (latest version) write method does not work Jun 16, 2017

This was referenced Jun 21, 2017

InfluxDB-Python client not working for Telegraf v1.3.2 influxdata/telegraf#2948

Closed

influxdb-python client not working for Telegraf v1.3.2 #472

Closed

aviau closed this as completed Jul 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(latest version) write method does not work #467

(latest version) write method does not work #467

tcloaa commented Jun 16, 2017

tcloaa commented Jun 16, 2017 •

edited

Loading

sebito91 commented Jun 16, 2017

tcloaa commented Jun 17, 2017 •

edited

Loading

sebito91 commented Jun 17, 2017

tcloaa commented Jun 17, 2017 •

edited

Loading

sebito91 commented Jun 17, 2017

tcloaa commented Jun 26, 2017

sebito91 commented Jul 6, 2017

sebito91 commented Jul 6, 2017

ferhatguneri commented Dec 19, 2018

a-l-t-e-r commented Aug 8, 2019

(latest version) write method does not work #467

(latest version) write method does not work #467

Comments

tcloaa commented Jun 16, 2017

tcloaa commented Jun 16, 2017 • edited Loading

sebito91 commented Jun 16, 2017

tcloaa commented Jun 17, 2017 • edited Loading

sebito91 commented Jun 17, 2017

tcloaa commented Jun 17, 2017 • edited Loading

sebito91 commented Jun 17, 2017

tcloaa commented Jun 26, 2017

sebito91 commented Jul 6, 2017

sebito91 commented Jul 6, 2017

ferhatguneri commented Dec 19, 2018

a-l-t-e-r commented Aug 8, 2019

tcloaa commented Jun 16, 2017 •

edited

Loading

tcloaa commented Jun 17, 2017 •

edited

Loading

tcloaa commented Jun 17, 2017 •

edited

Loading