Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65 #1

Open
ugoviti opened this issue Apr 29, 2019 · 19 comments
Open

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65 #1

ugoviti opened this issue Apr 29, 2019 · 19 comments

Comments

@ugoviti
Copy link

ugoviti commented Apr 29, 2019

Hi,

very nice project!

The k8s_discovery.pl works good, but I have a problem with k8s_collect.pl

I double checked the user and password API, but it doesn't works:

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65.

this URL is correct:
my $APIURL = 'http://127.0.0.1/zabbix/api_jsonrpc.php';

and logging in the gui with the specified APIUSER and APIPASS works without problems (the specified user is Super Admin with read/write access to all Zabbix hostgroups).

How can I debug the Zabbix API HTTP Connection Error?

I'm using Zabbix 4.2.1

Kind regards
@Frintrop
Copy link
Contributor

Frintrop commented May 3, 2019 via email

@ugoviti
Copy link
Author

ugoviti commented May 3, 2019

Hi,

thank you for the reply.

Yes, Zabbix web server is listening on local loopback:

[zabbix@mon01 ~]$ curl http://127.0.0.1/zabbix/
<!DOCTYPE html>
<html>
	<head>
		<meta http-equiv="X-UA-Compatible" content="IE=Edge"/>
		<meta charset="utf-8" />
		<meta name="viewport" content="width=device-width, initial-scale=1">
		<meta name="Author" content="Zabbix SIA" />
		<title>izMon: Zabbix</title>
		<link rel="icon" href="favicon.ico">

same result using http://localhost/zabbix/

running by hand I got the following error:

[zabbix@mon01 ~]$ /usr/lib/zabbix/externalscripts/k8s_collect.pl 127.0.0.1 /var/lib/zabbix/.kube/config test
Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65.

[root@mon01 ~]# netstat -atpun | grep zabbix
tcp        0      0 0.0.0.0:10050           0.0.0.0:*               LISTEN      17605/zabbix_agentd 
tcp        0      0 0.0.0.0:10051           0.0.0.0:*               LISTEN      17690/zabbix_server 
tcp6       0      0 :::10050                :::*                    LISTEN      17605/zabbix_agentd 
tcp6       0      0 :::10051                :::*                    LISTEN      17690/zabbix_server

zabbix-kubernetes is compatible with Zabbix 4.2?
Can I look into or debug other options? How can I debug the API accesso from python?

Thank you for the help, keep up the good work.

Kind regards

@Frintrop
Copy link
Contributor

Frintrop commented May 3, 2019 via email

@MogiePete
Copy link

@Frintrop

I am experiencing the same issue. I am running 4.0

@Frintrop
Copy link
Contributor

Frintrop commented May 4, 2019 via email

@MogiePete
Copy link

Discovery is working.

Using the code provided by Zabbix I am able to authenticate with the account created to monitor k8s so the account is working and the server is taking connections.


Getting started with Zabbix API

[root@svrzbx01 ~]# cat auth.pl
#!/usr/bin/perl

use 5.010;
use strict;
use warnings;
use JSON::RPC::Legacy::Client;
use Data::Dumper;

# Authenticate yourself
my $client = new JSON::RPC::Legacy::Client;
my $url = 'http://127.0.0.1/zabbix/api_jsonrpc.php';
my $authID;
my $response;

my $json = {
jsonrpc => "2.0",
method => "user.login",
params => {
user => "zabbixapi",
password => "zabbixapipass"
},
id => 1
};

$response = $client->call($url, $json);

# Check if response was successful
die "Authentication failed\n" unless $response->content->{'result'};

$authID = $response->content->{'result'};
print "Authentication successful. Auth ID: " . $authID . "\n";
[root@svrzbx01 ~]# ./auth.pl
Authentication successful. Auth ID: 121cd383ee6a40c6d9589fea5b3b7b75

And here is the code if I try to log in using curl.

 [root@zabbixserver ~]# curl -i -X POST -H 'Content-type:application/json' -d '{"jsonrpc":"2.0","method":"user.login","params":{ "user":"zabbixapi","password":"zabbixapipass"},"auth":null,"id":0}' http://127.0.0.1/zabbix/api_jsonrpc.php
 HTTP/1.1 200 OK
 Date: Sun, 05 May 2019 06:53:09 GMT
 Server: Apache/2.4.6 (Red Hat Enterprise Linux) PHP/5.4.16
 X-Powered-By: PHP/5.4.16
 Access-Control-Allow-Origin: *
 Access-Control-Allow-Headers: Content-Type
 Access-Control-Allow-Methods: POST
 Access-Control-Max-Age: 1000
 Content-Length: 68
 Content-Type: application/json
 
{"jsonrpc":"2.0","result":"ea54cb61db9a7d0a864565dd6a84c53d","id":0}

@mfocuz
Copy link
Contributor

mfocuz commented May 5, 2019

Hi All,

Please pay attention that discovery do not use Zabbix API, so there is no "dependency" between "discovery works" and "collector doesnt work".

Your issue with API connection most probbaly related to Zabbix API User. If username and password is correct and URL "localhost" is accessible from script(that means you run script from zabbix WEB server) - THEN YOU NEED TO CHECK THE FOLLOWING:

  1. Zabbix User has access to API
  2. Zabbix API User has at least ADMIN ROLE. Better to set SUPER ADMIN for troubleshooting, after you verify that script works correctly, reduce role to ADMIN. (I'm 99% sure its your case root cause).

ADMIN permissions required for reading item's keys.

Maxim

@ugoviti
Copy link
Author

ugoviti commented May 5, 2019

Hi, I double checked the user password and privileges (testing as Super Admin right now).

And tested with script auth.pl of @MogiePete and it's works good using the same url user and password of k8s_collect.pl:

$ ./auth.pl
Authentication successful. Auth ID: 867559bdf856c121dba44bad6b413795

I'm running it into a CentOS 7 system with official perl modules installed.

$ rpm -qa | grep -e LWP -e JSON
perl-LWP-MediaTypes-6.02-2.el7.noarch
perl-LWP-Protocol-https-6.04-4.el7.noarch
perl-JSON-RPC-1.03-5.el7.noarch
perl-JSON-2.59-2.el7.noarch

What perl version are you using?

Kind regards

@mfocuz
Copy link
Contributor

mfocuz commented May 5, 2019 •
edited
Loading

Please try to add response printing adter line 59 with:
print Dumper($respAuth->content);
and uncomment
use Dumper; on line 6

Its very rare case when perl libs versions are root cause, also I dont remember versions I checked. If you have successfully logged in with custom API script, all you need to troubleshoot is to find HTTP request differences between you custom script and script in this repo. Any HTTP proxy you have exp with will help.

@ugoviti
Copy link
Author

ugoviti commented May 5, 2019

Thankyou, made the change, now I got:

$ /usr/lib/zabbix/externalscripts/k8s_collect.pl 127.0.0.1 /var/lib/zabbix/.kube/config test
Global symbol "$response" requires explicit package name at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 61.
Execution of /usr/lib/zabbix/externalscripts/k8s_collect.pl aborted due to compilation errors.

@mfocuz
Copy link
Contributor

mfocuz commented May 5, 2019

please check again, I've edited comment in few secodns, you read too fast :D
print Dumper($respAuth->content);

@MogiePete
Copy link

@mfocuz

I have it dump the $respAuth and the content and this is the results:

$VAR1 = bless( {
                 '_msg' => 'Precondition Failed',
                 '_content' => '',
                 '_protocol' => 'HTTP/1.0',
                 '_headers' => bless( {
                                        'connection' => 'close',
                                        'access-control-allow-headers' => 'Content-Type',
                                        'x-powered-by' => 'PHP/5.4.16',
                                        'client-response-num' => 1,
                                        'access-control-allow-origin' => '*',
                                        'date' => 'Sun, 05 May 2019 16:09:11 GMT',
                                        'access-control-allow-methods' => 'POST',
                                        'client-peer' => '127.0.0.1:80',
                                        'content-length' => '0',
                                        '::std_case' => {
                                                          'access-control-allow-headers' => 'Access-Control-Allow-Headers',
                                                          'x-powered-by' => 'X-Powered-By',
                                                          'client-response-num' => 'Client-Response-Num',
                                                          'access-control-allow-origin' => 'Access-Control-Allow-Origin',
                                                          'access-control-allow-methods' => 'Access-Control-Allow-Methods',
                                                          'client-peer' => 'Client-Peer',
                                                          'client-date' => 'Client-Date',
                                                          'access-control-max-age' => 'Access-Control-Max-Age'
                                                        },
                                        'client-date' => 'Sun, 05 May 2019 16:09:11 GMT',
                                        'content-type' => 'text/html; charset=UTF-8',
                                        'access-control-max-age' => '1000',
                                        'server' => 'Apache/2.4.6 (Red Hat Enterprise Linux) PHP/5.4.16 mod_perl/2.0.10 Perl/v5.16.3'
                                      }, 'HTTP::Headers' ),
                 '_rc' => '412',
                 '_request' => bless( {
                                        '_content' => '{"params":{"password":"zabbixapipass","user":"zabbixapi"},"jsonrpc":"2.0","id":1,"method":"user.login"}',
                                        '_uri' => bless( do{\(my $o = 'http://127.0.0.1/zabbix/api_jsonrpc.php')}, 'URI::http' ),
                                        '_headers' => bless( {
                                                               'user-agent' => 'libwww-perl/6.05',
                                                               'content-type' => 'application/x-www-form-urlencoded',
                                                               'content-length' => 103
                                                             }, 'HTTP::Headers' ),
                                        '_method' => 'POST',
                                        '_uri_canonical' => $VAR1->{'_request'}{'_uri'}
                                      }, 'HTTP::Request' )
               }, 'HTTP::Response' );
$VAR1 = '';
Zabbix API HTTP Connection Error at ./k8s_collect.pl line 66.

@mfocuz
Copy link
Contributor

mfocuz commented May 5, 2019

you have HTTP response code 412, no idea why. I recommend to use HTTP proxy or tcpdump to verify difference between 2 HTTP requests.

@posuch
Copy link

posuch commented Aug 9, 2019 •
edited
Loading

got the same thing - zabbix 4.0.x on centos 7, anybody was able to fix it ?

@posuch
Copy link

posuch commented Aug 12, 2019 •
edited
Loading

so I guess what helped me with the authentification error is that in collect script in config section the IPs/hosts needed to be the same after that the script can connect to the API

see IP in $APIURL and $ZABBIXSERVER

CONFIG

my $APIURL = 'http://10.0.2.15/zabbix/api_jsonrpc.php';
my $APIUSER = 'api';
my $APIPASS = 'api';
my $ZABBIX_SENDER = '/usr/bin/zabbix_sender';
my $ZABBIX_SERVER = '10.0.2.15';
my $ZABBIX_PORT = '10051';
my $KUBECTL = '/usr/bin/kubectl';
my $TMP_FILE_PATH = '/tmp/send_to_zabbix_'. int(rand(1000000000)) .'.data';

the thing is now that the data are not comming to zabbix only if I run it by hand than some data comes in to it ( can anybody repeat this ? )

from the zabbix log ( no data sent ) in lastest data still zabbix api connection error

18969:20190812:091944.651 End of substitute_key_macros():SUCCEED data:'k8s_collect.pl[10.0.2.15,/root/.minikube/config/config.json,kube-system]'
18969:20190812:091944.651 In get_value_external() key:'k8s_collect.pl[10.0.2.15,/root/.minikube/config/config.json,kube-system]'
18969:20190812:091944.651 In zbx_popen() command:'/usr/lib/zabbix/externalscripts/k8s_collect.pl '10.0.2.15' '/root/.minikube/config/config.json' 'kube-system''

zabbix_api_error

but if I run this command by hand I get at least some data in the console ( works if run as root )

[root@centos7 externalscripts]# /usr/lib/zabbix/externalscripts/k8s_collect.pl '10.0.2.15' '/root/.minikube/config/config.json' 'kube-system'
info from server: "processed: 46; failed: 0; total: 46; seconds spent: 0.002099"
sent: 46; skipped: 0; total: 46

@ninjageek-zcs
Copy link

It worked for me after using JSON :: RPC :: Legacy :: Client for both authentication
how to get the list of items. The error 412 may be due to parameters being out of order.

@gabovos
Copy link

gabovos commented Apr 11, 2020

hi @ninjageek-zcs ,can you upload your k8s_collect.pl file after modifying the JSON :: RPC :: Legacy :: Client

@gabovos
Copy link

gabovos commented Apr 11, 2020

hello @MogiePete , I have the same problem as you, I run auth.pl and it works for me but k8s_collect.pl still fails me with "Zabbix API HTTP Connection Error at ./k8s_collect.pl line 66.", did you get to solve it?

@b2ronn
Copy link

b2ronn commented May 15, 2020 •
edited
Loading

$ua-> default_header('Content-Type' => 'application/json-rpc');
does not work.
I added
my @content_type = ( 'content-type', 'application/json', );
.......
my $respAuth = $ua->post($APIURL, @content_type, Content => $authRequestJson);

and in each POST

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@MogiePete @mfocuz @Frintrop @ugoviti @b2ronn @gabovos @posuch @ninjageek-zcs