Skip to content
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.

[7.x] backport #537 #551

Closed
wants to merge 21 commits into from
Closed

[7.x] backport #537 #551

wants to merge 21 commits into from

Conversation

mdelapenya
Copy link
Contributor

@mdelapenya mdelapenya commented Jan 4, 2021

What does this PR do?

It cherry-picks all commits in that PR, manually resolving conflicts.

Why is it important?

Keep 7.x branch up-to-date

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have run the Unit tests for the CLI, and they are passing locally
  • I have run the End-2-End tests for the suite I'm working on, and they are passing locally
  • I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

  • [ ]

Related issues

Follow-ups

Once merged using squash, try to backport that resulting commit to 7.10.x too

@mdelapenya mdelapenya self-assigned this Jan 4, 2021
@mdelapenya mdelapenya requested a review from a team January 4, 2021 15:48
@mdelapenya mdelapenya changed the base branch from master to 7.x January 4, 2021 15:49
@@ -35,6 +35,7 @@ pipeline {
string(name: 'SLACK_CHANNEL', defaultValue: 'observablt-bots', description: 'The Slack channel(s) where errors will be posted. For multiple channels, use a comma-separated list of channels')
string(name: 'ELASTIC_AGENT_DOWNLOAD_URL', defaultValue: '', description: 'If present, it will override the download URL for the Elastic agent artifact. (I.e. https://snapshots.elastic.co/7.11.0-d52afba4/downloads/beats/elastic-agent/elastic-agent-7.11.0-SNAPSHOT-linux-x86_64.tar.gz')
string(name: 'ELASTIC_AGENT_VERSION', defaultValue: '7.x-SNAPSHOT', description: 'SemVer version of the stand-alone elastic-agent to be used for Fleet tests. You can use here the tag of your PR to test your changes')
string(name: 'ELASTIC_AGENT_STALE_VERSION', defaultValue: '7.10.0', description: 'SemVer version of the stale stand-alone elastic-agent to be used for Fleet upgrade tests.')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalpristas what version should we use as stale in 7.x? Same question will come up for the backport to 7.10.x

@elasticmachine
Copy link
Contributor

elasticmachine commented Jan 4, 2021

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Started by user Manuel de la Peña

  • Start Time: 2021-01-04T16:01:35.676+0000

  • Duration: 16 min 6 sec

Test stats 🧪

Test Results
Failed 3
Passed 18
Skipped 0
Total 21

Test errors 3

Expand to view the tests failures

Initializing / End-To-End Tests / fleet_agent_endpoint_integration / [empty] – TEST-fleet.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/outputs/TEST-fleet.xml was length 0 
    

Initializing / End-To-End Tests / fleet_fleet_mode_agent / [empty] – TEST-fleet.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/outputs/TEST-fleet.xml was length 0 
    

Initializing / End-To-End Tests / fleet_stand_alone_agent / [empty] – TEST-fleet.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/outputs/TEST-fleet.xml was length 0 
    

Steps errors 7

Expand to view the steps failures

Run functional tests for fleet:agent_endpoint_integration
  • Took 5 min 44 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "agent_endpoint_integration" "7.x-SNAPSHOT" "7.x-SNAPSHOT"
Archive the artifacts
  • Took 0 min 0 sec . View more details on here
  • Description: [2021-01-04T16:09:14.273Z] Archiving artifacts hudson.AbortException: script returned exit code 1
Run functional tests for fleet:stand_alone_agent
  • Took 5 min 44 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "stand_alone_agent" "7.x-SNAPSHOT" "7.x-SNAPSHOT"
Archive the artifacts
  • Took 0 min 0 sec . View more details on here
  • Description: [2021-01-04T16:09:15.712Z] Archiving artifacts hudson.AbortException: script returned exit code 1
Run functional tests for fleet:fleet_mode_agent
  • Took 5 min 43 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "fleet_mode_agent" "7.x-SNAPSHOT" "7.x-SNAPSHOT"
Archive the artifacts
  • Took 0 min 0 sec . View more details on here
  • Description: [2021-01-04T16:09:14.871Z] Archiving artifacts hudson.AbortException: script returned exit code 1
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: hudson.AbortException: script returned exit code 1

Log output

Expand to view the last 100 lines of log output

[2021-01-04T16:15:58.747Z] time="2021-01-04T16:15:58Z" level=debug msg="Index Alias deleted using Elasticsearch Go client" indexAlias=metricbeat-7.x-mysql-percona-5.7.24-2ivq0xev status="400 Bad Request"
[2021-01-04T16:15:59.316Z] Pulling mysql (docker.elastic.co/integrations-ci/beats-mysql:percona-8.0.13-4-1)...
[2021-01-04T16:15:59.885Z] percona-8.0.13-4-1: Pulling from integrations-ci/beats-mysql
[2021-01-04T16:16:09.866Z] Digest: sha256:0d94341e7101c5720237ab262d7664ddf809eadea63b58ebd901d034fe516493
[2021-01-04T16:16:09.866Z] Status: Downloaded newer image for docker.elastic.co/integrations-ci/beats-mysql:percona-8.0.13-4-1
[2021-01-04T16:16:09.866Z] metricbeat_elasticsearch_1 is up-to-date
[2021-01-04T16:16:09.866Z] Creating metricbeat_mysql_1 ... 
[2021-01-04T16:16:12.664Z] 
Creating metricbeat_mysql_1 ... done
time="2021-01-04T16:16:12Z" level=debug msg="Docker compose executed." cmd="[up -d]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/mysql/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false MYSQL_PATH:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/mysql MYSQL_VARIANT:percona MYSQL_VERSION:8.0.13-4 indexName:metricbeat-7.x-mysql-percona-5.7.24-2ivq0xev logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:5.7.24 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:16:44.763Z] Found orphan containers (metricbeat_mysql_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[2021-01-04T16:16:44.763Z] metricbeat_elasticsearch_1 is up-to-date
[2021-01-04T16:16:44.763Z] Creating metricbeat_metricbeat_1 ... 
[2021-01-04T16:16:44.763Z] 
Creating metricbeat_metricbeat_1 ... done
time="2021-01-04T16:16:44Z" level=debug msg="Docker compose executed." cmd="[up -d]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/metricbeat/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false MYSQL_PATH:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/mysql MYSQL_VARIANT:percona MYSQL_VERSION:8.0.13-4 indexName:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:16:44.763Z] time="2021-01-04T16:16:44Z" level=info msg="Metricbeat is running configured for the service" metricbeatVersion=7.x-SNAPSHOT service=mysql serviceVersion=8.0.13-4 variant=Percona
[2021-01-04T16:16:45.024Z] Attaching to metricbeat_metricbeat_1
[2021-01-04T16:16:45.024Z] metricbeat_1     | 2021-01-04T16:16:44.160Z	INFO	instance/beat.go:660	Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs]
[2021-01-04T16:16:45.024Z] metricbeat_1     | 2021-01-04T16:16:44.160Z	DEBUG	[beat]	instance/beat.go:712	Beat metadata path: /usr/share/metricbeat/data/meta.json
[2021-01-04T16:16:45.024Z] metricbeat_1     | 2021-01-04T16:16:44.166Z	INFO	instance/beat.go:668	Beat ID: befa8b36-c07f-43b7-8912-7e419e10d45b
[2021-01-04T16:16:45.024Z] metricbeat_1     | 2021-01-04T16:16:44.167Z	DEBUG	[seccomp]	seccomp/seccomp.go:117	Loading syscall filter	{"seccomp_filter": {"no_new_privs":true,"flag":"tsync","policy":{"default_action":"errno","syscalls":[{"names":["accept","accept4","access","arch_prctl","bind","brk","chmod","chown","clock_gettime","clone","close","connect","dup","dup2","epoll_create","epoll_create1","epoll_ctl","epoll_pwait","epoll_wait","exit","exit_group","fchdir","fchmod","fchmodat","fchown","fchownat","fcntl","fdatasync","flock","fstat","fstatfs","fsync","ftruncate","futex","getcwd","getdents","getdents64","geteuid","getgid","getpeername","getpid","getppid","getrandom","getrlimit","getrusage","getsockname","getsockopt","gettid","gettimeofday","getuid","inotify_add_watch","inotify_init1","inotify_rm_watch","ioctl","kill","listen","lseek","lstat","madvise","mincore","mkdirat","mmap","mprotect","munmap","nanosleep","newfstatat","open","openat","pipe","pipe2","poll","ppoll","pread64","pselect6","pwrite64","read","readlink","readlinkat","recvfrom","recvmmsg","recvmsg","rename","renameat","rt_sigaction","rt_sigprocmask","rt_sigreturn","sched_getaffinity","sched_yield","sendfile","sendmmsg","sendmsg","sendto","set_robust_list","setitimer","setsockopt","shutdown","sigaltstack","socket","splice","stat","statfs","sysinfo","tgkill","time","tkill","uname","unlink","unlinkat","wait4","waitid","write","writev"],"action":"allow"}]}}}
[2021-01-04T16:16:45.024Z] metricbeat_1     | 2021-01-04T16:16:44.167Z	INFO	[seccomp]	seccomp/seccomp.go:124	Syscall filter successfully installed
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.167Z	INFO	[beat]	instance/beat.go:996	Beat info	{"system_info": {"beat": {"path": {"config": "/usr/share/metricbeat", "data": "/usr/share/metricbeat/data", "home": "/usr/share/metricbeat", "logs": "/usr/share/metricbeat/logs"}, "type": "metricbeat", "uuid": "befa8b36-c07f-43b7-8912-7e419e10d45b"}}}
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.167Z	INFO	[beat]	instance/beat.go:1005	Build info	{"system_info": {"build": {"commit": "86bc1a06d02bbc8e4384be56cd85fd55f47e810b", "libbeat": "7.12.0", "time": "2020-12-21T13:02:19.000Z", "version": "7.12.0"}}}
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.167Z	INFO	[beat]	instance/beat.go:1008	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":8,"version":"go1.14.12"}}}
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.168Z	INFO	[beat]	instance/beat.go:1012	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-01-04T16:03:45Z","containerized":true,"name":"4126c9d0ce9c","ip":["127.0.0.1/8","172.18.0.4/16"],"kernel_version":"5.4.0-1032-gcp","mac":["02:42:ac:12:00:04"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0,"id":"1d22c1d280174b8c894a1f75bdd1669a"}}}
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.169Z	INFO	[beat]	instance/beat.go:1041	Process info	{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/metricbeat", "exe": "/usr/share/metricbeat/metricbeat", "name": "metricbeat", "pid": 8, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2021-01-04T16:16:43.160Z"}}}
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.169Z	INFO	instance/beat.go:304	Setup Beat: metricbeat; Version: 7.12.0
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.169Z	DEBUG	[beat]	instance/beat.go:330	Initializing output plugins
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.169Z	INFO	[index-management]	idxmgmt/std.go:184	Set output.elasticsearch.index to 'metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir' as ILM is enabled.
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.169Z	INFO	eslegclient/connection.go:99	elasticsearch url: http://elasticsearch:9200
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.170Z	DEBUG	[publisher]	pipeline/consumer.go:148	start pipeline event consumer
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.170Z	INFO	[publisher]	pipeline/module.go:113	Beat name: 4126c9d0ce9c
[2021-01-04T16:16:45.025Z] metricbeat_1     | 2021-01-04T16:16:44.195Z	DEBUG	[modules]	beater/metricbeat.go:151	Available modules and metricsets: Register [ModuleFactory:[aws, awsfargate, azure, beat, cloudfoundry, docker, elasticsearch, kibana, linux, logstash, mongodb, mssql, mysql, oracle, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, appsearch/stats, aws/billing, aws/cloudwatch, aws/ec2, aws/rds, aws/sqs, awsfargate/task_stats, azure/app_insights, azure/billing, azure/monitor, azure/storage, beat/state, beat/stats, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/mgr_cluster_disk, ceph/mgr_cluster_health, ceph/mgr_osd_perf, ceph/mgr_osd_pool_stats, ceph/mgr_osd_tree, ceph/mgr_pool_disk, ceph/monitor_health, ceph/osd_df, ceph/osd_tree, ceph/pool_disk, cloudfoundry/container, cloudfoundry/counter, cloudfoundry/value, consul/agent, coredns/stats, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/enrich, elasticsearch/index, elasticsearch/index_recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, gcp/billing, gcp/metrics, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, istio/citadel, istio/galley, istio/mesh, istio/mixer, istio/pilot, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state_container, kubernetes/state_cronjob, kubernetes/state_daemonset, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_persistentvolume, kubernetes/state_persistentvolumeclaim, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_resourcequota, kubernetes/state_service, kubernetes/state_statefulset, kubernetes/state_storageclass, kubernetes/system, kubernetes/volume, kvm/dommemstat, kvm/status, linux/conntrack, linux/iostat, linux/ksm, linux/memory, linux/pageinfo, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, mssql/performance, mssql/transaction_log, munin/node, mysql/galera_status, mysql/query, mysql/status, nats/connection, nats/connections, nats/route, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, openmetrics/collector, oracle/performance, oracle/tablespace, php_fpm/pool, php_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, prometheus/query, prometheus/remote_write, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, sql/query, stan/channels, stan/stats, stan/subscriptions, statsd/server, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/network_summary, system/process, system/process_summary, system/raid, system/service, system/socket, system/socket_summary, system/uptime, system/users, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[LightModules:[ibmmq/qmgr, azure/container_registry, azure/container_instance, azure/container_service, azure/database_account, azure/app_state, azure/compute_vm, azure/compute_vm_scaleset, activemq/broker, activemq/queue, activemq/topic, redisenterprise/node, redisenterprise/proxy, mysql/performance, kafka/broker, kafka/producer, kafka/consumer, iis/webserver, iis/website, istio/proxy, istio/istiod, cockroachdb/status, aws/s3_daily_storage, aws/dynamodb, aws/ebs, aws/usage, aws/s3_request, aws/lambda, aws/vpn, aws/transitgateway, aws/natgateway, aws/elb, aws/sns, gcp/compute, gcp/pubsub, gcp/loadbalancing, gcp/storage, tomcat/threading, tomcat/memory, tomcat/requests, tomcat/cache]]]
[2021-01-04T16:16:45.026Z] metricbeat_1     | 2021-01-04T16:16:44.196Z	INFO	[monitoring]	log/log.go:117	Starting metrics logging every 30s
[2021-01-04T16:16:45.026Z] metricbeat_1     | 2021-01-04T16:16:44.196Z	INFO	instance/beat.go:468	metricbeat start running.
[2021-01-04T16:16:45.026Z] metricbeat_1     | 2021-01-04T16:16:44.196Z	DEBUG	[module]	module/wrapper.go:127	Starting Wrapper[name=mysql, len(metricSetWrappers)=1]
[2021-01-04T16:16:45.026Z] metricbeat_1     | 2021-01-04T16:16:44.196Z	DEBUG	[module]	module/wrapper.go:181	mysql/status will start after 3.536963809s
[2021-01-04T16:16:45.026Z] time="2021-01-04T16:16:45Z" level=debug msg="Docker compose executed." cmd="[logs metricbeat]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/metricbeat/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:16:45.026Z] time="2021-01-04T16:16:45Z" level=warning msg="There was an error executing the query" desiredHits=5 elapsedTime=6.605902ms error="Error getting response from Elasticsearch. Status: 404 Not Found, ResponseError: map[error:map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias root_cause:[map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias type:index_not_found_exception]] type:index_not_found_exception] status:404]" index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=1
[2021-01-04T16:16:45.597Z] time="2021-01-04T16:16:45Z" level=warning msg="There was an error executing the query" desiredHits=5 elapsedTime=542.904272ms error="Error getting response from Elasticsearch. Status: 404 Not Found, ResponseError: map[error:map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias root_cause:[map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias type:index_not_found_exception]] type:index_not_found_exception] status:404]" index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=2
[2021-01-04T16:16:46.977Z] time="2021-01-04T16:16:46Z" level=warning msg="There was an error executing the query" desiredHits=5 elapsedTime=1.697100918s error="Error getting response from Elasticsearch. Status: 404 Not Found, ResponseError: map[error:map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias root_cause:[map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias type:index_not_found_exception]] type:index_not_found_exception] status:404]" index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=3
[2021-01-04T16:16:48.883Z] time="2021-01-04T16:16:48Z" level=warning msg="There was an error executing the query" desiredHits=5 elapsedTime=3.80504653s error="Error getting response from Elasticsearch. Status: 404 Not Found, ResponseError: map[error:map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias root_cause:[map[index:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir index_uuid:_na_ reason:no such index [metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir] resource.id:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir resource.type:index_or_alias type:index_not_found_exception]] type:index_not_found_exception] status:404]" index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=4
[2021-01-04T16:16:54.164Z] time="2021-01-04T16:16:53Z" level=debug msg="Response information" hits=0 status="200 OK" took=1
[2021-01-04T16:16:54.164Z] time="2021-01-04T16:16:53Z" level=warning msg="Waiting for more hits in the index" currentHits=0 desiredHits=5 elapsedTime=8.834417592s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=5
[2021-01-04T16:16:59.442Z] time="2021-01-04T16:16:58Z" level=debug msg="Response information" hits=1 status="200 OK" took=2
[2021-01-04T16:16:59.442Z] time="2021-01-04T16:16:58Z" level=warning msg="Waiting for more hits in the index" currentHits=1 desiredHits=5 elapsedTime=13.360357044s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=6
[2021-01-04T16:17:01.981Z] time="2021-01-04T16:17:01Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2021-01-04T16:17:01.981Z] time="2021-01-04T16:17:01Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=16.520005257s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=7
[2021-01-04T16:17:10.105Z] time="2021-01-04T16:17:08Z" level=debug msg="Response information" hits=2 status="200 OK" took=2
[2021-01-04T16:17:10.105Z] time="2021-01-04T16:17:08Z" level=warning msg="Waiting for more hits in the index" currentHits=2 desiredHits=5 elapsedTime=23.956457695s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=8
[2021-01-04T16:17:16.676Z] time="2021-01-04T16:17:15Z" level=debug msg="Response information" hits=3 status="200 OK" took=3
[2021-01-04T16:17:16.676Z] time="2021-01-04T16:17:15Z" level=warning msg="Waiting for more hits in the index" currentHits=3 desiredHits=5 elapsedTime=30.946091942s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=9
[2021-01-04T16:17:20.869Z] time="2021-01-04T16:17:20Z" level=debug msg="Response information" hits=4 status="200 OK" took=3
[2021-01-04T16:17:20.869Z] time="2021-01-04T16:17:20Z" level=warning msg="Waiting for more hits in the index" currentHits=4 desiredHits=5 elapsedTime=35.064939439s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=10
[2021-01-04T16:17:27.442Z] time="2021-01-04T16:17:26Z" level=debug msg="Response information" hits=4 status="200 OK" took=2
[2021-01-04T16:17:27.442Z] time="2021-01-04T16:17:26Z" level=warning msg="Waiting for more hits in the index" currentHits=4 desiredHits=5 elapsedTime=41.177753604s index=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir retry=11
[2021-01-04T16:17:32.720Z] time="2021-01-04T16:17:31Z" level=debug msg="Response information" hits=5 status="200 OK" took=3
[2021-01-04T16:17:32.720Z] time="2021-01-04T16:17:31Z" level=info msg="Hits number satisfied" currentHits=5 desiredHits=5 elapsedTime=46.908818635s retries=12
[2021-01-04T16:17:32.720Z] time="2021-01-04T16:17:31Z" level=debug msg="Response information" hits=5 status="200 OK" took=2
[2021-01-04T16:17:32.720Z] time="2021-01-04T16:17:31Z" level=info msg="Hits number satisfied" currentHits=5 desiredHits=5 elapsedTime=6.95851ms retries=1
[2021-01-04T16:17:32.720Z] Stopping metricbeat_metricbeat_1 ... 
[2021-01-04T16:17:32.980Z] 
Stopping metricbeat_metricbeat_1 ... done
Removing metricbeat_metricbeat_1 ... 
[2021-01-04T16:17:32.980Z] 
Removing metricbeat_metricbeat_1 ... done
Going to remove metricbeat_metricbeat_1
[2021-01-04T16:17:33.240Z] time="2021-01-04T16:17:33Z" level=debug msg="Docker compose executed." cmd="[rm -fvs metricbeat]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/mysql/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:17:33.240Z] time="2021-01-04T16:17:33Z" level=debug msg="Service removed from compose" profile=metricbeat service=metricbeat
[2021-01-04T16:17:33.810Z] Stopping metricbeat_mysql_1 ... 
[2021-01-04T16:17:36.351Z] 
Stopping metricbeat_mysql_1 ... done
Removing metricbeat_mysql_1 ... 
[2021-01-04T16:17:36.351Z] 
Removing metricbeat_mysql_1 ... done
Going to remove metricbeat_mysql_1
[2021-01-04T16:17:36.351Z] time="2021-01-04T16:17:36Z" level=debug msg="Docker compose executed." cmd="[rm -fvs mysql]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/metricbeat/docker-compose.yml /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/services/mysql/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:17:36.351Z] time="2021-01-04T16:17:36Z" level=debug msg="Service removed from compose" profile=metricbeat service=mysql
[2021-01-04T16:17:36.351Z] time="2021-01-04T16:17:36Z" level=debug msg="Index deleted using Elasticsearch Go client" indexName=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir status="400 Bad Request"
[2021-01-04T16:17:36.351Z] time="2021-01-04T16:17:36Z" level=debug msg="Index Alias deleted using Elasticsearch Go client" indexAlias=metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir status="400 Bad Request"
[2021-01-04T16:17:36.920Z] Stopping metricbeat_elasticsearch_1 ... 
[2021-01-04T16:17:38.302Z] 
Stopping metricbeat_elasticsearch_1 ... done
Removing metricbeat_elasticsearch_1 ... 
[2021-01-04T16:17:38.303Z] 
Removing metricbeat_elasticsearch_1 ... done
Removing network metricbeat_default
[2021-01-04T16:17:38.303Z] time="2021-01-04T16:17:38Z" level=debug msg="Docker compose executed." cmd="[down --remove-orphans]" composeFilePaths="[/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/.op/compose/profiles/metricbeat/docker-compose.yml]" env="map[BEAT_STRICT_PERMS:false indexName:metricbeat-7.x-mysql-percona-8.0.13-4-1eh0jlir logLevel:debug metricbeatConfigFile:/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551/src/github.com/elastic/e2e-testing/e2e/_suites/metricbeat/configurations/mysql.yml metricbeatTag:7.x-SNAPSHOT mysqlTag:8.0.13-4 serviceName:mysql stackVersion:7.x-SNAPSHOT]" profile=metricbeat
[2021-01-04T16:17:38.303Z] <?xml version="1.0" encoding="UTF-8"?>
[2021-01-04T16:17:38.303Z] <testsuites name="main" tests="7" skipped="0" failures="0" errors="0" time="699.125428207">
[2021-01-04T16:17:38.303Z]   <testsuite name="Integrations" tests="7" skipped="0" failures="0" errors="0" time="657.1527552">
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-MariaDB-10.2.23 sends metrics to Elasticsearch without errors" status="passed" time="100.888255118"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-MariaDB-10.3.14 sends metrics to Elasticsearch without errors" status="passed" time="91.512319375"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-MariaDB-10.4.4 sends metrics to Elasticsearch without errors" status="passed" time="88.846009514"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-MySQL-5.7.12 sends metrics to Elasticsearch without errors" status="passed" time="86.037981186"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-MySQL-8.0.13 sends metrics to Elasticsearch without errors" status="passed" time="90.273181155"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-Percona-5.7.24 sends metrics to Elasticsearch without errors" status="passed" time="83.623765632"></testcase>
[2021-01-04T16:17:38.303Z]     <testcase name="mysql-Percona-8.0.13-4 sends metrics to Elasticsearch without errors" status="passed" time="93.304832834"></testcase>
[2021-01-04T16:17:38.303Z]   </testsuite>
[2021-01-04T16:17:38.303Z] </testsuites>+ sed -e 's/^[ \t]*//; s#>.*failed$#>#g' outputs/TEST-metricbeat
[2021-01-04T16:17:38.303Z] + grep -E '^<.*>$'
[2021-01-04T16:17:38.303Z] + exit 0
[2021-01-04T16:17:38.360Z] Recording test results
[2021-01-04T16:17:38.870Z] [Checks API] No suitable checks publisher found.
[2021-01-04T16:17:38.885Z] Archiving artifacts
[2021-01-04T16:17:40.216Z] Stage "Release" skipped due to earlier failure(s)
[2021-01-04T16:17:40.515Z] Running on worker-1244230 in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-551
[2021-01-04T16:17:40.561Z] [INFO] getVaultSecret: Getting secrets
[2021-01-04T16:17:40.622Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-01-04T16:17:42.593Z] + chmod 755 generate-build-data.sh
[2021-01-04T16:17:42.593Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-551/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-551/runs/2 FAILURE 965531
[2021-01-04T16:17:42.594Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-551/runs/2/steps/?limit=10000 -o steps-info.json
[2021-01-04T16:17:49.021Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-551/runs/2/tests/?status=FAILED -o tests-errors.json
[2021-01-04T16:17:49.021Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-551/runs/2/log/ -o pipeline-log.txt

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 3
Passed 18
Skipped 0
Total 21

Genuine test errors 3

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Initializing / End-To-End Tests / fleet_agent_endpoint_integration / [empty] – TEST-fleet.xml
  • Name: Initializing / End-To-End Tests / fleet_fleet_mode_agent / [empty] – TEST-fleet.xml
  • Name: Initializing / End-To-End Tests / fleet_stand_alone_agent / [empty] – TEST-fleet.xml

@mdelapenya
Copy link
Contributor Author

mdelapenya commented Jan 4, 2021

The errors I see in this PR against 7.x branch, which are the same errors as in any 7.x CI build (see https://beats-ci.elastic.co/job/e2e-tests/job/e2e-testing-mbp/job/7.x/35/), are related to Kibana not being able to start:

[2021-01-04T05:37:33.496Z] time="2021-01-04T05:37:33Z" level=warning msg="The Kibana instance is not healthy yet" elapsedTime=2m56.706823576s error="Get "http://localhost:5601/status\": dial tcp 127.0.0.1:5601: connect: connection refused" retry=42 statusEndpoint="http://localhost:5601/status"
[2021-01-04T05:37:33.496Z] time="2021-01-04T05:37:33Z" level=fatal msg="The Kibana instance could not get the healthy status" error="Get "http://localhost:5601/status\": dial tcp 127.0.0.1:5601: connect: connection refused" minutes=3m0s

@EricDavisX @ph do we know if docker.elastic.co/kibana/kibana:7.x-SNAPSHOT image suffers from any issue?

Did not mention it, but same code in 7.10.x passes the tests

@mdelapenya
Copy link
Contributor Author

Digging even more into the Kibana error, the container logs are not clear to me:

{"type":"log","@timestamp":"2021-01-04T16:45:00+00:00","tags":["fatal","root"],"pid":8,"message":"Error: Invalid server options {\n "port": 5601,\n "routes": {\n "cache": {\n "privacy": "private",\n "otherwise": "private, no-cache, no-store, must-revalidate"\n },\n "cors": false,\n "payload": {\n "maxBytes": 1048576\n },\n "validate": {\n "failAction": function defaultValidationErrorHandler(request, h, err) {\n // Newer versions of Joi don't format the key for missing params the same way. This shim\n // provides backwards compatibility. Unfortunately, Joi doesn't export it's own Error class\n // in JS so we have to rely on the name key before we can cast it.\n //\n // The Hapi code we're 'overwriting' can be found here:\n // https://github.com/hapijs/hapi/blob/master/lib/validation.js#L102\\n if (err && err.name === 'ValidationError' && err.hasOwnProperty('output')) {\n const validationError = err;\n const validationKeys = [];\n validationError.details.forEach(detail => {\n if (detail.path.length > 0) {\n validationKeys.push(_hoek.default.escapeHtml(detail.path.join('.')));\n } else {\n // If no path, use the value sigil to signal the entire value had an issue.\n validationKeys.push('value');\n }\n });\n validationError.output.payload.validation.keys = validationKeys;\n }\n\n throw err;\n},\n "options": {\n "abortEarly": false\n },\n "payload": "[value => Promise.resolve((0, _prototype_pollution.validateObject)(value))]"\n }\n },\n "state": {\n "strictHeader": false,\n "isHttpOnly": true,\n "isSameSite": false\n },\n "host" [1]: "0"\n}\n\n[1] "host" must be a valid hostname\n at Object.exports.apply (/usr/share/kibana/node_modules/@hapi/hapi/lib/config.js:19:15)\n at Object.internals.setup (/usr/share/kibana/node_modules/@hapi/hapi/lib/core.js:649:23)\n at new module.exports.internals.Core (/usr/share/kibana/node_modules/@hapi/hapi/lib/core.js:116:46)\n at new module.exports (/usr/share/kibana/node_modules/@hapi/hapi/lib/server.js:23:18)\n at createServer (/usr/share/kibana/src/core/server/http/http_tools.js:119:18)\n at HttpServer.setup (/usr/share/kibana/src/core/server/http/http_server.js:86:48)\n at HttpService.runNotReadyServer (/usr/share/kibana/src/core/server/http/http_service.js:165:26)\n at HttpService.setup (/usr/share/kibana/src/core/server/http/http_service.js:80:18)\n at Server.setup (/usr/share/kibana/src/core/server/server.js:192:23)\n at Root.setup (/usr/share/kibana/src/core/server/root/index.js:47:14)\n at bootstrap (/usr/share/kibana/src/core/server/bootstrap.js:110:5)\n at Command. (/usr/share/kibana/src/cli/serve/serve.js:180:5)"}

FATAL Error: Invalid server options {
"port": 5601,
"routes": {
"cache": {
"privacy": "private",
"otherwise": "private, no-cache, no-store, must-revalidate"
},
"cors": false,
"payload": {
"maxBytes": 1048576
},
"validate": {
"failAction": function defaultValidationErrorHandler(request, h, err) {\n // Newer versions of Joi don't format the key for missing params the same way. This shim\n // provides backwards compatibility. Unfortunately, Joi doesn't export it's own Error class\n // in JS so we have to rely on the name key before we can cast it.\n //\n // The Hapi code we're 'overwriting' can be found here:\n // https://github.com/hapijs/hapi/blob/master/lib/validation.js#L102\n if (err && err.name === 'ValidationError' && err.hasOwnProperty('output')) {\n const validationError = err;\n const validationKeys = [];\n validationError.details.forEach(detail => {\n if (detail.path.length > 0) {\n validationKeys.push(_hoek.default.escapeHtml(detail.path.join('.')));\n } else {\n // If no path, use the value sigil to signal the entire value had an issue.\n validationKeys.push('value');\n }\n });\n validationError.output.payload.validation.keys = validationKeys;\n }\n\n throw err;\n},
"options": {
"abortEarly": false
},
"payload": "[value => Promise.resolve((0, _prototype_pollution.validateObject)(value))]"
}
},
"state": {
"strictHeader": false,
"isHttpOnly": true,
"isSameSite": false
},
"host" [1]: "0"
}

[1] "host" must be a valid hostname

@mdelapenya mdelapenya marked this pull request as ready for review January 5, 2021 14:39
@mdelapenya mdelapenya mentioned this pull request Jan 5, 2021
8 tasks
@mdelapenya
Copy link
Contributor Author

Closing in favour of #558

@mdelapenya mdelapenya closed this Jan 5, 2021
@mdelapenya mdelapenya deleted the -b branch January 5, 2021 14:46
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants