Skip to content

Latest commit

 

History

History
3085 lines (3073 loc) · 117 KB

STATUS.md

File metadata and controls

3085 lines (3073 loc) · 117 KB

Data pipeline status

This file shows the latest status of the data pipeline.

Vaccinations

Get (scraping)

4/111 scripts failed, 21/111 were skipped. Latest update was 2022-09-27T07:04:16.

module status timestamp execution_time (sec) error
cowidev.vax.incremental.moldova ❌ 2022-09-27T07:04:16 40.05
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connection.py", line 179, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (, 'Connection to vaccinare.gov.md timed out. (connect timeout=20)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='vaccinare.gov.md', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(, 'Connection to vaccinare.gov.md timed out. (connect timeout=20)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/moldova.py", line 59, in main
    Moldova().export()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/moldova.py", line 54, in export
    df = self.read().pipe(self.pipeline)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/moldova.py", line 17, in read
    soup = get_soup(self.source_url)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 83, in get_soup
    response = get_response(source, request_method, use_proxy, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 49, in get_response
    raise err
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 43, in get_response
    response = requests.get(source, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/requests/adapters.py", line 507, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='vaccinare.gov.md', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(, 'Connection to vaccinare.gov.md timed out. (connect timeout=20)'))
cowidev.vax.incremental.russia ❌ 2022-09-27T07:04:16 0.45
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/russia.py", line 80, in main
    Russia().export()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/russia.py", line 66, in export
    data = read(source).pipe(pipeline)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/russia.py", line 11, in read
    soup = get_soup(source)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 83, in get_soup
    response = get_response(source, request_method, use_proxy, **kwargs)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 52, in get_response
    raise ValueError(
ValueError: Source https://gogov.ru/articles/covid-v-stats not reached! Error code 502 Bad Gateway
cowidev.vax.incremental.macao ❌ 2022-09-27T07:03:35 19.89
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/macao.py", line 86, in main
    Macao().export()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/macao.py", line 72, in export
    data = self.read()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/incremental/macao.py", line 20, in read
    elem = driver.find_element_by_partial_link_text("Weekly Bulletin on COVID-19")
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 462, in find_element_by_partial_link_text
    return self.find_element(by=By.PARTIAL_LINK_TEXT, value=link_text)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"partial link text","selector":"Weekly Bulletin on COVID-19"}
  (Session info: headless chrome=105.0.5195.102)
cowidev.vax.batch.japan ❌ 2022-09-27T07:03:16 1.51
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/batch/japan.py", line 236, in main
    Japan().export()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/batch/japan.py", line 220, in export
    df = self.read().pipe(self.pipeline_base)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/batch/japan.py", line 156, in read
    df_latest = self.read_latest().pipe(self.pipe_latest)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/batch/japan.py", line 88, in read_latest
    dfs.append(self._read_xlsx(self.source_url, self.sheets, self.metrics))
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/batch/japan.py", line 97, in _read_xlsx
    xlsx = pd.ExcelFile(url)
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/pandas/io/excel/_base.py", line 1191, in __init__
    ext = inspect_excel_format(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/pandas/io/excel/_base.py", line 1070, in inspect_excel_format
    with get_handle(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/pandas/io/common.py", line 609, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/pandas/io/common.py", line 312, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/mnt/owid_live_covid/covid-19-data/scripts/venv/lib/python3.9/site-packages/pandas/io/common.py", line 212, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/home/owid/.pyenv/versions/3.9.6/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
cowidev.vax.incremental.united_arab_emirates ⚠️ 2022-09-27T07:03:02 NaN
cowidev.vax.incremental.vietnam ⚠️ 2022-09-27T07:03:02 NaN
cowidev.vax.incremental.sri_lanka ⚠️ 2022-09-27T07:02:56 NaN
cowidev.vax.incremental.saint_lucia ⚠️ 2022-09-27T07:02:39 NaN
cowidev.vax.incremental.philippines ⚠️ 2022-09-27T07:02:38 NaN
cowidev.vax.incremental.qatar ⚠️ 2022-09-27T07:02:38 NaN
cowidev.vax.incremental.morocco ⚠️ 2022-09-27T07:02:23 NaN
cowidev.vax.incremental.myanmar ⚠️ 2022-09-27T07:02:23 NaN
cowidev.vax.incremental.nepal ⚠️ 2022-09-27T07:02:23 NaN
cowidev.vax.incremental.north_macedonia ⚠️ 2022-09-27T07:02:23 NaN
cowidev.vax.incremental.iran ⚠️ 2022-09-27T07:02:09 NaN
cowidev.vax.incremental.guatemala ⚠️ 2022-09-27T07:02:07 NaN
cowidev.vax.incremental.gabon ⚠️ 2022-09-27T07:02:05 NaN
cowidev.vax.incremental.costa_rica ⚠️ 2022-09-27T07:01:48 NaN
cowidev.vax.incremental.azerbaijan ⚠️ 2022-09-27T07:01:26 NaN
cowidev.vax.incremental.africacdc ⚠️ 2022-09-27T07:01:25 NaN
cowidev.vax.incremental.antigua_barbuda ⚠️ 2022-09-27T07:01:25 NaN
cowidev.vax.batch.ukraine ⚠️ 2022-09-27T07:01:03 NaN
cowidev.vax.batch.netherlands ⚠️ 2022-09-27T07:00:30 NaN
cowidev.vax.batch.portugal ⚠️ 2022-09-27T07:00:30 NaN
cowidev.vax.batch.indonesia ⚠️ 2022-09-27T07:00:17 NaN
cowidev.vax.incremental.paho ✅ 2022-09-27T07:03:14 50.05
cowidev.vax.incremental.suriname ✅ 2022-09-27T07:03:10 14.70
cowidev.vax.incremental.who ✅ 2022-09-27T07:03:07 4.83
cowidev.vax.incremental.china ✅ 2022-09-27T07:03:06 78.07
cowidev.vax.incremental.zambia ✅ 2022-09-27T07:03:05 0.82
cowidev.vax.incremental.turkey ✅ 2022-09-27T07:03:04 1.52
cowidev.vax.incremental.taiwan ✅ 2022-09-27T07:03:02 5.33
cowidev.vax.incremental.thailand ✅ 2022-09-27T07:03:02 4.15
cowidev.vax.incremental.spain ✅ 2022-09-27T07:02:57 3.96
cowidev.vax.incremental.monaco ✅ 2022-09-27T07:02:56 37.10
cowidev.vax.incremental.south_africa ✅ 2022-09-27T07:02:53 8.20
cowidev.vax.incremental.serbia ✅ 2022-09-27T07:02:45 5.79
cowidev.vax.incremental.pakistan ✅ 2022-09-27T07:02:38 1.45
cowidev.vax.incremental.poland ✅ 2022-09-27T07:02:38 0.28
cowidev.vax.incremental.northern_cyprus ✅ 2022-09-27T07:02:24 0.85
cowidev.vax.incremental.mongolia ✅ 2022-09-27T07:02:23 0.93
cowidev.vax.incremental.montenegro ✅ 2022-09-27T07:02:23 0.10
cowidev.vax.incremental.kazakhstan ✅ 2022-09-27T07:02:22 11.82
cowidev.vax.batch.united_states ✅ 2022-09-27T07:02:19 66.10
cowidev.vax.incremental.laos ✅ 2022-09-27T07:02:18 2.36
cowidev.vax.batch.ecdc ✅ 2022-09-27T07:02:16 121.92
cowidev.vax.incremental.kyrgyzstan ✅ 2022-09-27T07:02:16 1.64
cowidev.vax.incremental.kosovo ✅ 2022-09-27T07:02:15 2.65
cowidev.vax.incremental.jamaica ✅ 2022-09-27T07:02:12 2.46
cowidev.vax.incremental.hungary ✅ 2022-09-27T07:02:10 1.86
cowidev.vax.incremental.isle_of_man ✅ 2022-09-27T07:02:10 0.89
cowidev.vax.incremental.india ✅ 2022-09-27T07:02:09 0.96
cowidev.vax.incremental.georgia ✅ 2022-09-27T07:02:08 2.87
cowidev.vax.incremental.guernsey ✅ 2022-09-27T07:02:08 1.08
cowidev.vax.incremental.iceland ✅ 2022-09-27T07:02:08 0.48
cowidev.vax.incremental.greenland ✅ 2022-09-27T07:02:07 1.81
cowidev.vax.incremental.dominican_republic ✅ 2022-09-27T07:02:05 7.43
cowidev.vax.incremental.faeroe_islands ✅ 2022-09-27T07:02:05 0.80
cowidev.vax.incremental.finland ✅ 2022-09-27T07:02:05 0.34
cowidev.vax.incremental.fiji ✅ 2022-09-27T07:02:05 0.00
cowidev.vax.incremental.el_salvador ✅ 2022-09-27T07:02:04 4.60
cowidev.vax.incremental.cyprus ✅ 2022-09-27T07:01:59 2.51
cowidev.vax.incremental.cuba ✅ 2022-09-27T07:01:58 8.75
cowidev.vax.incremental.curacao ✅ 2022-09-27T07:01:57 0.81
cowidev.vax.incremental.bangladesh ✅ 2022-09-27T07:01:56 25.32
cowidev.vax.incremental.croatia ✅ 2022-09-27T07:01:49 0.41
cowidev.vax.incremental.barbados ✅ 2022-09-27T07:01:48 12.19
cowidev.vax.incremental.bulgaria ✅ 2022-09-27T07:01:48 7.47
cowidev.vax.incremental.brazil ✅ 2022-09-27T07:01:41 0.62
cowidev.vax.batch.switzerland ✅ 2022-09-27T07:01:40 41.69
cowidev.vax.batch.south_korea ✅ 2022-09-27T07:01:36 54.48
cowidev.vax.incremental.bahrain ✅ 2022-09-27T07:01:31 4.57
cowidev.vax.incremental.aruba ✅ 2022-09-27T07:01:26 1.13
cowidev.vax.batch.zimbabwe ✅ 2022-09-27T07:01:25 1.01
cowidev.vax.batch.uruguay ✅ 2022-09-27T07:01:24 1.01
cowidev.vax.batch.slovakia ✅ 2022-09-27T07:01:23 45.97
cowidev.vax.batch.united_kingdom ✅ 2022-09-27T07:01:12 9.55
cowidev.vax.batch.trinidad_and_tobago ✅ 2022-09-27T07:01:03 2.20
cowidev.vax.batch.sweden ✅ 2022-09-27T07:01:01 13.76
cowidev.vax.batch.spc ✅ 2022-09-27T07:00:58 14.73
cowidev.vax.batch.latvia ✅ 2022-09-27T07:00:47 18.47
cowidev.vax.batch.new_zealand ✅ 2022-09-27T07:00:44 13.64
cowidev.vax.batch.slovenia ✅ 2022-09-27T07:00:42 3.33
cowidev.vax.batch.singapore ✅ 2022-09-27T07:00:38 1.36
cowidev.vax.batch.romania ✅ 2022-09-27T07:00:37 6.52
cowidev.vax.batch.saudi_arabia ✅ 2022-09-27T07:00:37 3.12
cowidev.vax.batch.peru ✅ 2022-09-27T07:00:34 3.55
cowidev.vax.batch.luxembourg ✅ 2022-09-27T07:00:30 1.70
cowidev.vax.batch.malta ✅ 2022-09-27T07:00:30 0.76
cowidev.vax.batch.malaysia ✅ 2022-09-27T07:00:30 0.64
cowidev.vax.batch.norway ✅ 2022-09-27T07:00:30 0.34
cowidev.vax.batch.hong_kong ✅ 2022-09-27T07:00:29 11.89
cowidev.vax.batch.lithuania ✅ 2022-09-27T07:00:29 0.85
cowidev.vax.batch.italy ✅ 2022-09-27T07:00:28 9.94
cowidev.vax.batch.jersey ✅ 2022-09-27T07:00:28 0.98
cowidev.vax.batch.israel ✅ 2022-09-27T07:00:27 9.77
cowidev.vax.batch.greece ✅ 2022-09-27T07:00:19 2.74
cowidev.vax.batch.germany ✅ 2022-09-27T07:00:19 2.04
cowidev.vax.batch.ireland ✅ 2022-09-27T07:00:18 0.21
cowidev.vax.batch.canada ✅ 2022-09-27T07:00:17 5.16
cowidev.vax.batch.denmark ✅ 2022-09-27T07:00:17 3.28
cowidev.vax.batch.france ✅ 2022-09-27T07:00:17 0.47
cowidev.vax.batch.czechia ✅ 2022-09-27T07:00:16 3.42
cowidev.vax.batch.ecuador ✅ 2022-09-27T07:00:16 1.82
cowidev.vax.batch.estonia ✅ 2022-09-27T07:00:16 0.60
cowidev.vax.batch.austria ✅ 2022-09-27T07:00:14 3.91
cowidev.vax.batch.argentina ✅ 2022-09-27T07:00:14 3.52
cowidev.vax.batch.belgium ✅ 2022-09-27T07:00:13 2.56
cowidev.vax.batch.chile ✅ 2022-09-27T07:00:13 1.10
cowidev.vax.batch.australia ✅ 2022-09-27T07:00:12 2.18
cowidev.vax.batch.bolivia ✅ 2022-09-27T07:00:12 2.13

Process (Checks)

4/223 processes failed, 0/223 were skipped. Latest update was 2022-09-27T07:04:43.

location status timestamp error
Tunisia ❌ 2022-09-27T07:04:41
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/process.py", line 95, in _process_location_and_move_file
    df = process_location(df, monotonic_check_skip, anomaly_check_skip)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/utils.py", line 57, in process_location
    country_df_sanity_checks(
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 72, in country_df_sanity_checks
    checker.run()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 252, in run
    self.check_metrics()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 169, in check_metrics
    self._check_metrics_monotonic(df)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 185, in _check_metrics_monotonic
    raise ValueError(
ValueError: Tunisia -- Column total_vaccinations must be monotonically increasing! Check:
    location       date  ... people_fully_vaccinated total_boosters
328  Tunisia 2022-09-13  ...                 6382822        1242934

[1 rows x 8 columns]
Senegal ❌ 2022-09-27T07:04:39
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/process.py", line 95, in _process_location_and_move_file
    df = process_location(df, monotonic_check_skip, anomaly_check_skip)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/utils.py", line 57, in process_location
    country_df_sanity_checks(
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 72, in country_df_sanity_checks
    checker.run()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 252, in run
    self.check_metrics()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 169, in check_metrics
    self._check_metrics_monotonic(df)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 185, in _check_metrics_monotonic
    raise ValueError(
ValueError: Senegal -- Column total_vaccinations must be monotonically increasing! Check:
    location       date  ... people_fully_vaccinated total_boosters
117  Senegal 2022-09-11  ...                 1096047          17060

[1 rows x 8 columns]
Sint Maarten (Dutch part) ❌ 2022-09-27T07:04:39
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/process.py", line 95, in _process_location_and_move_file
    df = process_location(df, monotonic_check_skip, anomaly_check_skip)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/utils.py", line 57, in process_location
    country_df_sanity_checks(
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 72, in country_df_sanity_checks
    checker.run()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 252, in run
    self.check_metrics()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 169, in check_metrics
    self._check_metrics_monotonic(df)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 185, in _check_metrics_monotonic
    raise ValueError(
ValueError: Sint Maarten (Dutch part) -- Column people_fully_vaccinated must be monotonically increasing! Check:
                     location  ... total_boosters
51  Sint Maarten (Dutch part)  ...           9021

[1 rows x 8 columns]
Mozambique ❌ 2022-09-27T07:04:35
show
Traceback (most recent call last):
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/process.py", line 95, in _process_location_and_move_file
    df = process_location(df, monotonic_check_skip, anomaly_check_skip)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/cmd/vax/process/utils.py", line 57, in process_location
    country_df_sanity_checks(
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 72, in country_df_sanity_checks
    checker.run()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 252, in run
    self.check_metrics()
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 169, in check_metrics
    self._check_metrics_monotonic(df)
  File "/mnt/owid_live_covid/covid-19-data/scripts/src/cowidev/vax/utils/checks.py", line 185, in _check_metrics_monotonic
    raise ValueError(
ValueError: Mozambique -- Column total_vaccinations must be monotonically increasing! Check:
      location       date  ... people_fully_vaccinated total_boosters
64  Mozambique 2022-09-11  ...                13272757         606946

[1 rows x 8 columns]
Venezuela ✅ 2022-09-27T07:04:43
Vietnam ✅ 2022-09-27T07:04:43
Wales ✅ 2022-09-27T07:04:43
Wallis and Futuna ✅ 2022-09-27T07:04:43
Yemen ✅ 2022-09-27T07:04:43
Zambia ✅ 2022-09-27T07:04:43
Zimbabwe ✅ 2022-09-27T07:04:43
Turkmenistan ✅ 2022-09-27T07:04:42
Turks and Caicos Islands ✅ 2022-09-27T07:04:42
Tuvalu ✅ 2022-09-27T07:04:42
Uganda ✅ 2022-09-27T07:04:42
Ukraine ✅ 2022-09-27T07:04:42
United Arab Emirates ✅ 2022-09-27T07:04:42
United Kingdom ✅ 2022-09-27T07:04:42
United States ✅ 2022-09-27T07:04:42
Uruguay ✅ 2022-09-27T07:04:42
Uzbekistan ✅ 2022-09-27T07:04:42
Vanuatu ✅ 2022-09-27T07:04:42
Syria ✅ 2022-09-27T07:04:41
Taiwan ✅ 2022-09-27T07:04:41
Tajikistan ✅ 2022-09-27T07:04:41
Tanzania ✅ 2022-09-27T07:04:41
Thailand ✅ 2022-09-27T07:04:41
Timor ✅ 2022-09-27T07:04:41
Togo ✅ 2022-09-27T07:04:41
Tokelau ✅ 2022-09-27T07:04:41
Tonga ✅ 2022-09-27T07:04:41
Trinidad and Tobago ✅ 2022-09-27T07:04:41
Turkey ✅ 2022-09-27T07:04:41
Solomon Islands ✅ 2022-09-27T07:04:40
Somalia ✅ 2022-09-27T07:04:40
South Africa ✅ 2022-09-27T07:04:40
South Korea ✅ 2022-09-27T07:04:40
South Sudan ✅ 2022-09-27T07:04:40
Spain ✅ 2022-09-27T07:04:40
Sri Lanka ✅ 2022-09-27T07:04:40
Sudan ✅ 2022-09-27T07:04:40
Suriname ✅ 2022-09-27T07:04:40
Sweden ✅ 2022-09-27T07:04:40
Switzerland ✅ 2022-09-27T07:04:40
Samoa ✅ 2022-09-27T07:04:39
San Marino ✅ 2022-09-27T07:04:39
Sao Tome and Principe ✅ 2022-09-27T07:04:39
Saudi Arabia ✅ 2022-09-27T07:04:39
Scotland ✅ 2022-09-27T07:04:39
Serbia ✅ 2022-09-27T07:04:39
Seychelles ✅ 2022-09-27T07:04:39
Sierra Leone ✅ 2022-09-27T07:04:39
Singapore ✅ 2022-09-27T07:04:39
Slovakia ✅ 2022-09-27T07:04:39
Slovenia ✅ 2022-09-27T07:04:39
Pitcairn ✅ 2022-09-27T07:04:38
Poland ✅ 2022-09-27T07:04:38
Portugal ✅ 2022-09-27T07:04:38
Qatar ✅ 2022-09-27T07:04:38
Romania ✅ 2022-09-27T07:04:38
Russia ✅ 2022-09-27T07:04:38
Rwanda ✅ 2022-09-27T07:04:38
Saint Helena ✅ 2022-09-27T07:04:38
Saint Kitts and Nevis ✅ 2022-09-27T07:04:38
Saint Lucia ✅ 2022-09-27T07:04:38
Saint Vincent and the Grenadines ✅ 2022-09-27T07:04:38
Northern Cyprus ✅ 2022-09-27T07:04:37
Northern Ireland ✅ 2022-09-27T07:04:37
Norway ✅ 2022-09-27T07:04:37
Oman ✅ 2022-09-27T07:04:37
Pakistan ✅ 2022-09-27T07:04:37
Palestine ✅ 2022-09-27T07:04:37
Panama ✅ 2022-09-27T07:04:37
Papua New Guinea ✅ 2022-09-27T07:04:37
Paraguay ✅ 2022-09-27T07:04:37
Peru ✅ 2022-09-27T07:04:37
Philippines ✅ 2022-09-27T07:04:37
Namibia ✅ 2022-09-27T07:04:36
Nauru ✅ 2022-09-27T07:04:36
Nepal ✅ 2022-09-27T07:04:36
Netherlands ✅ 2022-09-27T07:04:36
New Caledonia ✅ 2022-09-27T07:04:36
New Zealand ✅ 2022-09-27T07:04:36
Nicaragua ✅ 2022-09-27T07:04:36
Niger ✅ 2022-09-27T07:04:36
Nigeria ✅ 2022-09-27T07:04:36
Niue ✅ 2022-09-27T07:04:36
North Macedonia ✅ 2022-09-27T07:04:36
Malta ✅ 2022-09-27T07:04:35
Mauritania ✅ 2022-09-27T07:04:35
Mauritius ✅ 2022-09-27T07:04:35
Mexico ✅ 2022-09-27T07:04:35
Moldova ✅ 2022-09-27T07:04:35
Monaco ✅ 2022-09-27T07:04:35
Mongolia ✅ 2022-09-27T07:04:35
Montenegro ✅ 2022-09-27T07:04:35
Montserrat ✅ 2022-09-27T07:04:35
Morocco ✅ 2022-09-27T07:04:35
Myanmar ✅ 2022-09-27T07:04:35
Liberia ✅ 2022-09-27T07:04:34
Libya ✅ 2022-09-27T07:04:34
Liechtenstein ✅ 2022-09-27T07:04:34
Lithuania ✅ 2022-09-27T07:04:34
Luxembourg ✅ 2022-09-27T07:04:34
Macao ✅ 2022-09-27T07:04:34
Madagascar ✅ 2022-09-27T07:04:34
Malawi ✅ 2022-09-27T07:04:34
Malaysia ✅ 2022-09-27T07:04:34
Maldives ✅ 2022-09-27T07:04:34
Mali ✅ 2022-09-27T07:04:34
Jordan ✅ 2022-09-27T07:04:33
Kazakhstan ✅ 2022-09-27T07:04:33
Kenya ✅ 2022-09-27T07:04:33
Kiribati ✅ 2022-09-27T07:04:33
Kosovo ✅ 2022-09-27T07:04:33
Kuwait ✅ 2022-09-27T07:04:33
Kyrgyzstan ✅ 2022-09-27T07:04:33
Laos ✅ 2022-09-27T07:04:33
Latvia ✅ 2022-09-27T07:04:33
Lebanon ✅ 2022-09-27T07:04:33
Lesotho ✅ 2022-09-27T07:04:33
Indonesia ✅ 2022-09-27T07:04:32
Iran ✅ 2022-09-27T07:04:32
Iraq ✅ 2022-09-27T07:04:32
Ireland ✅ 2022-09-27T07:04:32
Isle of Man ✅ 2022-09-27T07:04:32
Israel ✅ 2022-09-27T07:04:32
Italy ✅ 2022-09-27T07:04:32
Jamaica ✅ 2022-09-27T07:04:32
Japan ✅ 2022-09-27T07:04:32
Jersey ✅ 2022-09-27T07:04:32
Guatemala ✅ 2022-09-27T07:04:31
Guernsey ✅ 2022-09-27T07:04:31
Guinea ✅ 2022-09-27T07:04:31
Guinea-Bissau ✅ 2022-09-27T07:04:31
Guyana ✅ 2022-09-27T07:04:31
Haiti ✅ 2022-09-27T07:04:31
Honduras ✅ 2022-09-27T07:04:31
Hong Kong ✅ 2022-09-27T07:04:31
Hungary ✅ 2022-09-27T07:04:31
Iceland ✅ 2022-09-27T07:04:31
India ✅ 2022-09-27T07:04:31
France ✅ 2022-09-27T07:04:30
French Polynesia ✅ 2022-09-27T07:04:30
Gabon ✅ 2022-09-27T07:04:30
Gambia ✅ 2022-09-27T07:04:30
Georgia ✅ 2022-09-27T07:04:30
Germany ✅ 2022-09-27T07:04:30
Ghana ✅ 2022-09-27T07:04:30
Gibraltar ✅ 2022-09-27T07:04:30
Greece ✅ 2022-09-27T07:04:30
Greenland ✅ 2022-09-27T07:04:30
Grenada ✅ 2022-09-27T07:04:30
El Salvador ✅ 2022-09-27T07:04:29
England ✅ 2022-09-27T07:04:29
Equatorial Guinea ✅ 2022-09-27T07:04:29
Estonia ✅ 2022-09-27T07:04:29
Eswatini ✅ 2022-09-27T07:04:29
Ethiopia ✅ 2022-09-27T07:04:29
Faeroe Islands ✅ 2022-09-27T07:04:29
Falkland Islands ✅ 2022-09-27T07:04:29
Fiji ✅ 2022-09-27T07:04:29
Finland ✅ 2022-09-27T07:04:29
Curacao ✅ 2022-09-27T07:04:28
Cyprus ✅ 2022-09-27T07:04:28
Czechia ✅ 2022-09-27T07:04:28
Democratic Republic of Congo ✅ 2022-09-27T07:04:28
Denmark ✅ 2022-09-27T07:04:28
Djibouti ✅ 2022-09-27T07:04:28
Dominica ✅ 2022-09-27T07:04:28
Dominican Republic ✅ 2022-09-27T07:04:28
Ecuador ✅ 2022-09-27T07:04:28
Egypt ✅ 2022-09-27T07:04:28
Chad ✅ 2022-09-27T07:04:27
Chile ✅ 2022-09-27T07:04:27
China ✅ 2022-09-27T07:04:27
Colombia ✅ 2022-09-27T07:04:27
Comoros ✅ 2022-09-27T07:04:27
Congo ✅ 2022-09-27T07:04:27
Cook Islands ✅ 2022-09-27T07:04:27
Costa Rica ✅ 2022-09-27T07:04:27
Cote d'Ivoire ✅ 2022-09-27T07:04:27
Croatia ✅ 2022-09-27T07:04:27
Cuba ✅ 2022-09-27T07:04:27
British Virgin Islands ✅ 2022-09-27T07:04:26
Brunei ✅ 2022-09-27T07:04:26
Bulgaria ✅ 2022-09-27T07:04:26
Burkina Faso ✅ 2022-09-27T07:04:26
Burundi ✅ 2022-09-27T07:04:26
Cambodia ✅ 2022-09-27T07:04:26
Cameroon ✅ 2022-09-27T07:04:26
Canada ✅ 2022-09-27T07:04:26
Cape Verde ✅ 2022-09-27T07:04:26
Cayman Islands ✅ 2022-09-27T07:04:26
Central African Republic ✅ 2022-09-27T07:04:26
Belarus ✅ 2022-09-27T07:04:25
Belgium ✅ 2022-09-27T07:04:25
Belize ✅ 2022-09-27T07:04:25
Benin ✅ 2022-09-27T07:04:25
Bermuda ✅ 2022-09-27T07:04:25
Bhutan ✅ 2022-09-27T07:04:25
Bolivia ✅ 2022-09-27T07:04:25
Bonaire Sint Eustatius and Saba ✅ 2022-09-27T07:04:25
Bosnia and Herzegovina ✅ 2022-09-27T07:04:25
Botswana ✅ 2022-09-27T07:04:25
Brazil ✅ 2022-09-27T07:04:25
Antigua and Barbuda ✅ 2022-09-27T07:04:24
Argentina ✅ 2022-09-27T07:04:24
Armenia ✅ 2022-09-27T07:04:24
Aruba ✅ 2022-09-27T07:04:24
Australia ✅ 2022-09-27T07:04:24
Austria ✅ 2022-09-27T07:04:24
Azerbaijan ✅ 2022-09-27T07:04:24
Bahamas ✅ 2022-09-27T07:04:24
Bahrain ✅ 2022-09-27T07:04:24
Bangladesh ✅ 2022-09-27T07:04:24
Barbados ✅ 2022-09-27T07:04:24
Afghanistan ✅ 2022-09-27T07:04:23
Albania ✅ 2022-09-27T07:04:23
Algeria ✅ 2022-09-27T07:04:23
Andorra ✅ 2022-09-27T07:04:23
Angola ✅ 2022-09-27T07:04:23
Anguilla ✅ 2022-09-27T07:04:23

Testing

Get (scraping)

18/130 scripts failed, 9/130 were skipped. Latest update was 2022-08-10T14:33:15.

module status timestamp execution_time (sec) error
cowidev.testing.batch.hong_kong ❌ 2022-06-23T13:00:51 0.23
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/hong_kong.py", line 84, in main
    HongKong().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/hong_kong.py", line 79, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/hong_kong.py", line 24, in read
    return pd.read_csv(self.source_url)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 609, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 312, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 212, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
cowidev.testing.batch.canada ❌ 2022-06-23T12:59:16 0.64
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/canada.py", line 67, in main
    Canada().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/canada.py", line 57, in export
    df = self.read()  # .pipe(self.pipeline_base)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/canada.py", line 31, in read
    df = pd.read_csv(data, usecols=["prname", "date", "numtested", "numtests"])
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 135, in __init__
    self._validate_usecols_names(usecols, self.orig_names)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py", line 866, in _validate_usecols_names
    raise ValueError(
ValueError: Usecols do not match columns, columns expected but not found: ['numtests', 'numtested']
cowidev.testing.batch.costa_rica ❌ 2022-06-23T12:58:49 0.74
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/costa_rica.py", line 60, in main
    CostaRica().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/costa_rica.py", line 55, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/costa_rica.py", line 17, in read
    df = pd.read_csv(self.source_url, delimiter=";", usecols=["nue_posi", "conf_nexo", "nue_descar", "FECHA"])
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 609, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 312, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 212, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404:
cowidev.testing.batch.spain ❌ 2022-06-23T12:58:30 1.19
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/spain.py", line 51, in main
    Spain().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/spain.py", line 46, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/spain.py", line 19, in read
    df = pd.read_csv(self.test_url, encoding="cp1252", delimiter=";")
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 609, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 312, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 212, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not found
cowidev.testing.incremental.syria ❌ 2022-06-23T12:39:51 0.85
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/syria.py", line 104, in main
    Syria().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/syria.py", line 91, in export
    data = self.read()[0]
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/syria.py", line 29, in read
    data = self._parse_data(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/syria.py", line 36, in _parse_data
    elem = self._get_relevant_element(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/syria.py", line 54, in _get_relevant_element
    elem = soup.find_all("h4", "title")[0].find("a")
IndexError: list index out of range
cowidev.testing.incremental.russia ❌ 2022-06-23T12:39:50 40.02
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 179, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (, 'Connection to rospotrebnadzor.ru timed out. (connect timeout=20)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='rospotrebnadzor.ru', port=443): Max retries exceeded with url: /about/info/news/?PAGEN_1=1 (Caused by ConnectTimeoutError(, 'Connection to rospotrebnadzor.ru timed out. (connect timeout=20)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/russia.py", line 104, in main
    Russia().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/russia.py", line 91, in export
    data = self.read()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/russia.py", line 30, in read
    soup = get_soup(url)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 83, in get_soup
    response = get_response(source, request_method, use_proxy, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 49, in get_response
    raise err
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 43, in get_response
    response = requests.get(source, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/adapters.py", line 507, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='rospotrebnadzor.ru', port=443): Max retries exceeded with url: /about/info/news/?PAGEN_1=1 (Caused by ConnectTimeoutError(, 'Connection to rospotrebnadzor.ru timed out. (connect timeout=20)'))
cowidev.testing.incremental.mongolia ❌ 2022-06-23T12:39:10 2.26
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/mongolia.py", line 31, in main
    Mongolia().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/mongolia.py", line 19, in export
    increment(
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/utils/incremental.py", line 50, in increment
    _check_fields(df_current, country, source_url, source_label, units, date, count, daily_change)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/utils/incremental.py", line 111, in _check_fields
    raise ValueError(f"`cumulative_total` can't be lower than currently highers 'Cumulative total' value.")
ValueError: `cumulative_total` can't be lower than currently highers 'Cumulative total' value.
cowidev.testing.incremental.papua_new_guinea ❌ 2022-06-23T12:39:10 0.01
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='covid19.info.gov.pg', port=443): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/papua_new_guinea.py", line 54, in main
    PapuaNewGuinea().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/papua_new_guinea.py", line 41, in export
    data = self.read()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/papua_new_guinea.py", line 21, in read
    soup = get_soup(self.source_url)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 83, in get_soup
    response = get_response(source, request_method, use_proxy, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 49, in get_response
    raise err
  File "/Users/cam/covid-19-data/scripts/src/cowidev/utils/web/scraping.py", line 43, in get_response
    response = requests.get(source, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='covid19.info.gov.pg', port=443): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
cowidev.testing.incremental.moldova ❌ 2022-06-23T12:39:08 6.31
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/moldova.py", line 100, in main
    Moldova().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/moldova.py", line 86, in export
    data = self.read()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/moldova.py", line 27, in read
    data = self._parse_data(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/moldova.py", line 33, in _parse_data
    url = self._get_relevant_element(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/moldova.py", line 55, in _get_relevant_element
    if not link:
UnboundLocalError: local variable 'link' referenced before assignment
cowidev.testing.incremental.jordan ❌ 2022-06-23T12:39:02 1.26
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/jordan.py", line 25, in read
    count = self._request()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/jordan.py", line 110, in _request
    response = json.loads(
KeyError: 'results'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/jordan.py", line 147, in main
    Jordan().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/jordan.py", line 141, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/jordan.py", line 28, in read
    raise KeyError("No value found. Please modify the payload and headers.")
KeyError: 'No value found. Please modify the payload and headers.'
cowidev.testing.incremental.georgia ❌ 2022-06-23T12:39:00 6.88
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'date'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/georgia.py", line 115, in main
    Georgia().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/georgia.py", line 107, in export
    date=data["date"],
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/series.py", line 942, in __getitem__
    return self._get_value(key)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/series.py", line 1051, in _get_value
    loc = self.index.get_loc(label)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'date'
cowidev.testing.incremental.emro ❌ 2022-06-23T12:38:54 11.65
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/emro.py", line 155, in main
    EMRO().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/emro.py", line 150, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/emro.py", line 54, in read
    df = self._parse_data(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/emro.py", line 66, in _parse_data
    df = self._parse_metrics(df_list)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/emro.py", line 92, in _parse_metrics
    df = [table for table in df_list if self.columns_to_check["tests"] in table.columns][0]
IndexError: list index out of range
cowidev.testing.incremental.cambodia ❌ 2022-06-23T12:38:42 6.87
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/cambodia.py", line 41, in main
    Cambodia().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/cambodia.py", line 19, in export
    count = clean_count(soup.select("p+ div strong:nth-child(1)")[0].text)
IndexError: list index out of range
cowidev.testing.incremental.belarus ❌ 2022-06-23T12:38:35 2.77
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/belarus.py", line 70, in main
    Belarus().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/belarus.py", line 65, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/belarus.py", line 24, in read
    df = self._parse_data(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/belarus.py", line 30, in _parse_data
    elem = soup.find(text=self.regex["element"]).parent
AttributeError: 'NoneType' object has no attribute 'parent'
cowidev.testing.incremental.barbados ❌ 2022-06-23T12:38:32 1.54
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/barbados.py", line 72, in main
    Barbados().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/barbados.py", line 67, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/barbados.py", line 24, in read
    df = self._parse_data(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/barbados.py", line 36, in _parse_data
    count = self._parse_metrics(soup)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/incremental/barbados.py", line 51, in _parse_metrics
    count = re.search(self.regex["count"], text).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
cowidev.testing.batch.turkey ❌ 2022-06-23T12:38:31 17.08
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/turkey.py", line 64, in main
    Turkey().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/turkey.py", line 59, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/generic.py", line 5430, in pipe
    return com.pipe(self, func, *args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/common.py", line 471, in pipe
    return func(obj, *args, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/turkey.py", line 55, in pipeline
    df = df.pipe(self.pipe_filter).pipe(self.pipe_rename_columns).pipe(self.pipe_date).pipe(self.pipe_metadata)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/generic.py", line 5430, in pipe
    return com.pipe(self, func, *args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/common.py", line 471, in pipe
    return func(obj, *args, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/turkey.py", line 33, in pipe_filter
    return df[["Tarih", "Bugünkü Test Sayısı"]].dropna()
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/frame.py", line 3464, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1)[1]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexing.py", line 1314, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexing.py", line 1377, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: "['Bugünkü Test Sayısı'] not in index"
cowidev.testing.batch.north_korea ❌ 2022-06-23T12:38:12 45.68
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Daily change in cumulative total'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/north_korea.py", line 49, in main
    NorthKorea().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/north_korea.py", line 44, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/generic.py", line 5430, in pipe
    return com.pipe(self, func, *args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/common.py", line 471, in pipe
    return func(obj, *args, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/north_korea.py", line 40, in pipeline
    df = df.pipe(self.pipe_rename_columns).pipe(self.pipe_filter).pipe(self.pipe_metadata)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/generic.py", line 5430, in pipe
    return com.pipe(self, func, *args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/common.py", line 471, in pipe
    return func(obj, *args, **kwargs)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/north_korea.py", line 34, in pipe_filter
    df["Daily change in cumulative total"] = pd.to_numeric(df["Daily change in cumulative total"])
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'Daily change in cumulative total'
cowidev.testing.batch.mexico ❌ 2022-06-23T12:37:27 0.67
show
Traceback (most recent call last):
  File "/Users/cam/covid-19-data/scripts/src/cowidev/cmd/commons/get.py", line 47, in run
    module.main()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/mexico.py", line 74, in main
    Mexico().export()
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/mexico.py", line 69, in export
    df = self.read().pipe(self.pipeline)
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/mexico.py", line 26, in read
    self.url_melt(f"{base_url}Confirmados_{yesterday}.csv", "positive"),
  File "/Users/cam/covid-19-data/scripts/src/cowidev/testing/batch/mexico.py", line 15, in url_melt
    df_melt = pd.read_csv(url).melt(id_vars=["cve_ent", "poblacion", "nombre"], var_name="Date", value_name=name)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 609, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 312, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/Users/cam/covid-19-data/venv/lib/python3.10/site-packages/pandas/io/common.py", line 212, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
cowidev.testing.incremental.ukraine ⚠️ 2022-06-23T12:35:50 NaN
cowidev.testing.incremental.vanuatu ⚠️ 2022-06-23T12:35:50 NaN
cowidev.testing.incremental.gibraltar ⚠️ 2022-06-23T12:35:28 NaN
cowidev.testing.incremental.benin ⚠️ 2022-06-23T12:35:20 NaN
cowidev.testing.batch.philippines ⚠️ 2022-06-23T12:35:04 NaN
cowidev.testing.batch.kazakhstan ⚠️ 2022-06-23T12:35:00 NaN
cowidev.testing.batch.indonesia ⚠️ 2022-06-23T12:34:59 NaN
cowidev.testing.batch.guatemala ⚠️ 2022-06-23T12:34:58 NaN
cowidev.testing.batch.andorra ⚠️ 2022-06-23T12:34:54 NaN
cowidev.testing.batch.australia ✅ 2022-08-10T14:33:15 0.30
cowidev.testing.batch.liechtenstein ✅ 2022-06-23T12:37:24 144.54
cowidev.testing.incremental.kosovo ✅ 2022-06-23T12:36:00 26.83
cowidev.testing.incremental.vietnam ✅ 2022-06-23T12:35:53 3.34
cowidev.testing.incremental.timor ✅ 2022-06-23T12:35:51 2.53
cowidev.testing.incremental.tunisia ✅ 2022-06-23T12:35:51 2.46
cowidev.testing.incremental.myanmar ✅ 2022-06-23T12:35:50 12.59
cowidev.testing.incremental.togo ✅ 2022-06-23T12:35:50 1.75
cowidev.testing.incremental.saint_vincent_and_the_grenadines ✅ 2022-06-23T12:35:49 2.58
cowidev.testing.incremental.singapore ✅ 2022-06-23T12:35:49 1.69
cowidev.testing.incremental.suriname ✅ 2022-06-23T12:35:48 1.24
cowidev.testing.incremental.sweden ✅ 2022-06-23T12:35:48 1.21
cowidev.testing.incremental.nepal ✅ 2022-06-23T12:35:47 8.91
cowidev.testing.incremental.pakistan ✅ 2022-06-23T12:35:47 5.08
cowidev.testing.incremental.saint_lucia ✅ 2022-06-23T12:35:47 1.09
cowidev.testing.incremental.paraguay ✅ 2022-06-23T12:35:46 2.10
cowidev.testing.incremental.saint_kitts_nevis ✅ 2022-06-23T12:35:46 0.70
cowidev.testing.incremental.nicaragua ✅ 2022-06-23T12:35:45 6.82
cowidev.testing.incremental.palau ✅ 2022-06-23T12:35:45 2.97
cowidev.testing.incremental.new_zealand ✅ 2022-06-23T12:35:42 3.55
cowidev.testing.incremental.north_macedonia ✅ 2022-06-23T12:35:42 3.48
cowidev.testing.incremental.morocco ✅ 2022-06-23T12:35:39 1.19
cowidev.testing.incremental.laos ✅ 2022-06-23T12:35:38 5.38
cowidev.testing.incremental.lebanon ✅ 2022-06-23T12:35:38 4.88
cowidev.testing.incremental.libya ✅ 2022-06-23T12:35:37 3.95
cowidev.testing.incremental.maldives ✅ 2022-06-23T12:35:36 1.43
cowidev.testing.incremental.fiji ✅ 2022-06-23T12:35:34 6.73
cowidev.testing.incremental.cape_verde ✅ 2022-06-23T12:35:33 11.84
cowidev.testing.incremental.haiti ✅ 2022-06-23T12:35:33 4.42
cowidev.testing.incremental.iran ✅ 2022-06-23T12:35:33 4.26
cowidev.testing.incremental.greece ✅ 2022-06-23T12:35:32 3.58
cowidev.testing.incremental.bahamas ✅ 2022-06-23T12:35:28 9.52
cowidev.testing.incremental.croatia ✅ 2022-06-23T12:35:28 6.66
cowidev.testing.incremental.faeroe_islands ✅ 2022-06-23T12:35:28 1.64
cowidev.testing.incremental.equatorial_guinea ✅ 2022-06-23T12:35:27 3.76
cowidev.testing.incremental.el_salvador ✅ 2022-06-23T12:35:23 1.22
cowidev.testing.incremental.belize ✅ 2022-06-23T12:35:22 2.47
cowidev.testing.batch.ecdc ✅ 2022-06-23T12:35:21 25.16
cowidev.testing.incremental.bulgaria ✅ 2022-06-23T12:35:21 1.19
cowidev.testing.incremental.antigua_barbuda ✅ 2022-06-23T12:35:20 3.64
cowidev.testing.incremental.bahrain ✅ 2022-06-23T12:35:20 1.59
cowidev.testing.incremental.bangladesh ✅ 2022-06-23T12:35:20 1.47
cowidev.testing.batch.united_states_virgin_islands ✅ 2022-06-23T12:35:18 4.09
cowidev.testing.batch.united_states ✅ 2022-06-23T12:35:18 4.06
cowidev.testing.incremental.albania ✅ 2022-06-23T12:35:18 2.72
cowidev.testing.incremental.azerbaijan ✅ 2022-06-23T12:35:18 1.93
cowidev.testing.batch.united_arab_emirates ✅ 2022-06-23T12:35:16 2.85
cowidev.testing.batch.united_kingdom ✅ 2022-06-23T12:35:16 2.22
cowidev.testing.incremental.africacdc ✅ 2022-06-23T12:35:16 0.90
cowidev.testing.batch.switzerland ✅ 2022-06-23T12:35:15 4.97
cowidev.testing.batch.zambia ✅ 2022-06-23T12:35:15 0.49
cowidev.testing.batch.uruguay ✅ 2022-06-23T12:35:15 0.32
cowidev.testing.batch.northern_mariana_islands ✅ 2022-06-23T12:35:14 12.22
cowidev.testing.batch.puerto_rico ✅ 2022-06-23T12:35:14 10.44
cowidev.testing.batch.thailand ✅ 2022-06-23T12:35:14 2.82
cowidev.testing.batch.trinidad_and_tobago ✅ 2022-06-23T12:35:13 0.57
cowidev.testing.batch.sri_lanka ✅ 2022-06-23T12:35:12 3.10
cowidev.testing.batch.taiwan ✅ 2022-06-23T12:35:12 1.76
cowidev.testing.batch.netherlands ✅ 2022-06-23T12:35:11 10.22
cowidev.testing.batch.south_korea ✅ 2022-06-23T12:35:10 1.97
cowidev.testing.batch.slovenia ✅ 2022-06-23T12:35:09 1.25
cowidev.testing.batch.senegal ✅ 2022-06-23T12:35:08 0.63
cowidev.testing.batch.serbia ✅ 2022-06-23T12:35:08 0.55
cowidev.testing.batch.slovakia ✅ 2022-06-23T12:35:08 0.49
cowidev.testing.batch.saudi_arabia ✅ 2022-06-23T12:35:08 0.46
cowidev.testing.batch.south_africa ✅ 2022-06-23T12:35:08 0.31
cowidev.testing.batch.marshall_islands ✅ 2022-06-23T12:35:07 6.63
cowidev.testing.batch.qatar ✅ 2022-06-23T12:35:07 3.31
cowidev.testing.batch.rwanda ✅ 2022-06-23T12:35:07 3.00
cowidev.testing.batch.guam ✅ 2022-06-23T12:35:04 6.33
cowidev.testing.batch.luxembourg ✅ 2022-06-23T12:35:04 3.73
cowidev.testing.batch.peru ✅ 2022-06-23T12:35:04 0.65
cowidev.testing.batch.portugal ✅ 2022-06-23T12:35:04 0.40
cowidev.testing.batch.norway ✅ 2022-06-23T12:35:03 1.14
cowidev.testing.batch.israel ✅ 2022-06-23T12:35:02 2.82
cowidev.testing.batch.colombia ✅ 2022-06-23T12:35:01 6.74
cowidev.testing.batch.hungary ✅ 2022-06-23T12:35:01 2.11
cowidev.testing.batch.latvia ✅ 2022-06-23T12:35:01 1.17
cowidev.testing.batch.lithuania ✅ 2022-06-23T12:35:01 0.84
cowidev.testing.batch.malaysia ✅ 2022-06-23T12:35:01 0.25
cowidev.testing.batch.malta ✅ 2022-06-23T12:35:01 0.24
cowidev.testing.batch.iceland ✅ 2022-06-23T12:35:00 1.07
cowidev.testing.batch.ireland ✅ 2022-06-23T12:35:00 0.76
cowidev.testing.batch.japan ✅ 2022-06-23T12:35:00 0.49
cowidev.testing.batch.italy ✅ 2022-06-23T12:35:00 0.29
cowidev.testing.batch.bosnia_herzegovina ✅ 2022-06-23T12:34:59 5.02
cowidev.testing.batch.estonia ✅ 2022-06-23T12:34:59 2.25
cowidev.testing.batch.france ✅ 2022-06-23T12:34:59 1.94
cowidev.testing.batch.finland ✅ 2022-06-23T12:34:59 1.57
cowidev.testing.batch.germany ✅ 2022-06-23T12:34:59 0.99
cowidev.testing.batch.india ✅ 2022-06-23T12:34:59 0.42
cowidev.testing.batch.cyprus ✅ 2022-06-23T12:34:58 3.29
cowidev.testing.batch.cuba ✅ 2022-06-23T12:34:58 2.82
cowidev.testing.batch.denmark ✅ 2022-06-23T12:34:58 2.75
cowidev.testing.batch.ecuador ✅ 2022-06-23T12:34:58 1.02
cowidev.testing.batch.argentina ✅ 2022-06-23T12:34:57 3.24
cowidev.testing.batch.austria ✅ 2022-06-23T12:34:57 3.22
cowidev.testing.batch.czechia ✅ 2022-06-23T12:34:57 1.80
cowidev.testing.batch.armenia ✅ 2022-06-23T12:34:56 1.97
cowidev.testing.batch.belgium ✅ 2022-06-23T12:34:55 1.60
cowidev.testing.batch.brazil ✅ 2022-06-23T12:34:55 0.85
cowidev.testing.batch.chile ✅ 2022-06-23T12:34:55 0.63
cowidev.testing.batch.bolivia ✅ 2022-06-23T12:34:54 0.36