@kopf.timer continues to run after CR is deleted

[ghost]

Long story short

The timer function still runs the code even when the custom resource is deleted.

Description

The @kopf.timer decorator still triggered even when there's no more CR of that group/version exists. I expects that when the CR of that kind is deleted the timer would stop since no more CR of that group/version exists.

In the console output below where I put an extra empty line showing that I've deleted the CR from k8s but kopf continues to run the function.

The code snippet to reproduce the issue

import kopf

@kopf.timer('diligent.com', 'v1', 'restarts', interval=5)
def restart(spec, logger, **kwargs):
    logger.info(spec)

The exact command to reproduce the issue

kopf run restart.py --verbose

The full output of the command that failed

(restart) [releng|consul] ➜  src git:(dev) ✗ kopf run restart.py -v
[2020-08-14 11:37:11,033] kopf.reactor.activit [INFO    ] Initial authentication has been initiated.
[2020-08-14 11:37:11,033] kopf.activities.auth [DEBUG   ] Activity 'login_via_pykube' is invoked.
[2020-08-14 11:37:11,037] kopf.activities.auth [DEBUG   ] Pykube is configured via kubeconfig file.
[2020-08-14 11:37:11,038] kopf.activities.auth [INFO    ] Activity 'login_via_pykube' succeeded.
[2020-08-14 11:37:11,038] kopf.reactor.activit [INFO    ] Initial authentication has finished.
[2020-08-14 11:37:11,812] kopf.engines.peering [WARNING ] Default peering object not found, falling back to the standalone mode.
[2020-08-14 11:37:27,003] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Handling cycle is finished, waiting for new changes since now.
[2020-08-14 11:37:27,003] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:27,003] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:27,005] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:37:32,008] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:32,009] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:32,010] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.

[2020-08-14 11:37:37,016] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:37,016] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:37,017] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:37:42,024] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:42,024] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:42,026] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:37:47,029] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:47,029] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:47,031] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:37:52,033] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:52,034] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:52,035] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:37:57,041] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:37:57,041] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:37:57,042] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
[2020-08-14 11:38:02,049] kopf.objects         [DEBUG   ] [consul/restart-on-consul-updates] Timer 'restart' is invoked.
[2020-08-14 11:38:02,049] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] {'annotation': 'consul.diligent.com/restart'}
[2020-08-14 11:38:02,051] kopf.objects         [INFO    ] [consul/restart-on-consul-updates] Timer 'restart' succeeded.
^C[2020-08-14 11:38:03,514] kopf.reactor.running [INFO    ] Signal SIGINT is received. Operator is stopping.
[2020-08-14 11:38:03,514] kopf.reactor.running [DEBUG   ] Root task 'credentials retriever' is cancelled.
[2020-08-14 11:38:03,514] kopf.reactor.running [DEBUG   ] Root task 'poster of events' is cancelled.
[2020-08-14 11:38:03,515] kopf.reactor.running [WARNING ] Root task 'daemon killer' is finished unexpectedly.
[2020-08-14 11:38:03,515] kopf.reactor.running [DEBUG   ] Root task 'watcher of restarts.diligent.com' is cancelled.
[2020-08-14 11:38:03,515] kopf.reactor.running [DEBUG   ] Root tasks are stopped: finishing normally; tasks left: set()

Environment

• Kopf version: kopf, version 0.27
• Kubernetes version: v1.17.5
• Python version: Python 3.8.5
• OS/platform: Linux x86_64 GNU/Linux

Python packages installed

aiohttp==3.6.2
aiojobs==0.2.2
async-timeout==3.0.1
attrs==19.3.0
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
idna==2.10
iso8601==0.1.12
kopf==0.27
logzero==1.5.0
multidict==4.7.6
pip==20.2.2
pykube-ng==20.7.2
python-consul==1.1.0
PyYAML==5.3.1
requests==2.24.0
setuptools==49.2.0
six==1.15.0
typing-extensions==3.7.4.2
urllib3==1.25.10
wheel==0.34.2
yarl==1.5.1

[nolar]

Solved in nolar/kopf#548, available since kopf>=0.28rc3

[ghost]

Thanks @nolar. I'll give this a try.