Multiple upstream values not supported

[Nereis]

Describe the bug
As explain here, nginx can log multiple responses in the upstream variables

request_time=0.016 upstream_response_time=0.000, 0.016 upstream_connect_time=0.000, 0.000 upstream_header_time=0.000, 0.016

which is not supportted by the nginx-log-exporter

To Reproduce
Using a proxy or I think having multiple upstream servers with one failing to force the double responses should do it

Expected behavior
At least parse the line with no error and consider the last value

Configuration file

Nginx conf.d

map $remote_addr $remote_addr_subnet {
    ~(?P<ip>\d+\.\d+\.\d+)\.    $ip.0;
    ~(?P<ip>[^:]+:[^:]+):       $ip::;
    default                     unknown;
}

log_format custom '[$time_local] client=$remote_addr remote_user=\"$remote_user\" request=\"$request\" request_length=$request_length status=$status body_bytes_sent=$body_bytes_sent referer=\"$http_referer\" user_agent=\"$http_user_agent\" x_forwarded=\"$http_x_forwarded_for\" request_time=$request_time upstream_response_time=$upstream_response_time upstream_connect_time=$upstream_connect_time upstream_header_time=$upstream_header_time subnet=$remote_addr_subnet';

nginx-log-exporter config

listen:
  port: 4040
  address: "127.0.0.1"
  metrics_endpoint: "/metrics"

consul:
  enable: false

namespaces:
  - name: nginx
    format: "[$time_local] client=$remote_addr remote_user=\"$remote_user\" request=\"$request\" request_length=$request_length status=$status body_bytes_sent=$body_bytes_sent referer=\"$http_referer\" user_agent=\"$http_user_agent\" x_forwarded=\"$http_x_forwarded_for\" request_time=$request_time upstream_response_time=$upstream_response_time upstream_connect_time=$upstream_connect_time upstream_header_time=$upstream_header_time"

    source:
      files:
        - /var/log/nginx/projectportal.elca.ch.access.log
    histogram_buckets: [.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10]

Example log file

Aug 18 07:13:14 xxx prometheus-nginxlog-exporter[1153]: 
    error while parsing line
        '[18/Aug/2022:07:13:14 +0200] client=192.168.3.26 remote_user="-" request="GET /jira/rest/gadget/1.0/favfilters/count?filter=47524&_=1660784281372 HTTP/1.1" request_length=769 status=200 body_bytes_sent=75 referer="xxx" user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0" x_forwarded="xxx, xxx" request_time=0.016 upstream_response_time=0.000, 0.016 upstream_connect_time=0.000, 0.000 upstream_header_time=0.000, 0.016 subnet=192.168.3.0'
    text log parsing err: access log line
        '[18/Aug/2022:07:13:14 +0200] client=192.168.3.26 remote_user="-" request="GET /jira/rest/gadget/1.0/favfilters/count?filter=47524&_=1660784281372 HTTP/1.1" request_length=769 status=200 body_bytes_sent=75 referer="xxx" user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0" x_forwarded="xxx, xxx" request_time=0.016 upstream_response_time=0.000, 0.016 upstream_connect_time=0.000, 0.000 upstream_header_time=0.000, 0.016 subnet=192.168.3.0'
    does not match given format
        '^\[(?P<time_local>[^]]*)\] client=(?P<remote_addr>[^ ]*) remote_user="(?P<remote_user>[^"]*)" request="(?P<request>[^"]*)" request_length=(?P<request_length>[^ ]*) status=(?P<status>[^ ]*) body_bytes_sent=(?P<body_bytes_sent>[^ ]*) referer="(?P<http_referer>[^"]*)" user_agent="(?P<http_user_agent>[^"]*)" x_forwarded="(?P<http_x_forwarded_for>[^"]*)" request_time=(?P<request_time>[^ ]*) upstream_response_time=(?P<upstream_response_time>[^ ]*) upstream_connect_time=(?P<upstream_connect_time>[^ ]*) upstream_header_time=(?P<upstream_header_time>[^ ]*)'

Metrics output

nginx_parse_errors_total{job="nginx-log-exporter"} 28913

Environment:

• Exporter version: 1.9.2
• OS: Debian 10
• Deployment method: ansible using the released tar.gz

[kralewitz]

I've sent a PR to fix this (#267 ), but it was autoclosed today.

[helmich-bot]

There has not been any activity to this issue in the last 30 days. It will automatically be closed after 7 more days. Remove the stale label to prevent this.

[martin-helmich]

Apologies, this should not have gotten stale. I'll have a look at the provided PR.

[martin-helmich]

Fixed with #267 -- thanks for the contribution!