AndTrigger does not work

[NikkoFox]

Hi
Here is part of the code, but the scheduler does not work
And the example from the documentation also does not work.

from apscheduler.schedulers.blocking import BlockingScheduler
from apscheduler.triggers.interval import IntervalTrigger
from apscheduler.triggers.cron import CronTrigger
from apscheduler.triggers.combining import AndTrigger
scheduler = BlockingScheduler()
trigger_day = AndTrigger([IntervalTrigger(minutes=2), CronTrigger(hour='7-22')])
scheduler.add_job(job, trigger_day, max_instances=3, id='test')
scheduler.replace_existing = True
scheduler.start()

What am I doing wrong?

[agronholm]

It seems that this AndTrigger can never find a next fire time. This is a bug.

[agronholm]

Disregard that. The actual problem is that because IntervalTrigger starts counting from current time, it never agrees with the CronTrigger because CronTrigger never produces any nonzero seconds or microseconds that IntervalTrigger likely will. This problem needs to be documented and tools provided to debug triggers that hang this way.

[alexdev27]

Any news about that problem?

[agronholm]

I've explained the problem and it's a design issue. If you have any suggestions, I'm open to ideas.

[kaos]

First thing that comes to my mind, is to have an optional parameter threshold that indicates how close the times has to be (in seconds, for instance) in order to be accepted.
Example:

AndTrigger([IntervalTrigger(minutes=2), CronTrigger(hour='7-22')], threshold=2)

So if the difference between the interval and cron next run times are less than 2 seconds, use one of them. (like, the closest to occur, or the one from the first trigger in the list, etc..)

[agronholm]

I am adding a threshold parameter to the trigger as suggested. It will be in the next 4.0 series release.

[agronholm]

@kaos Trying to implement this, I realized that there is a potential problem: triggers like the interval trigger may veer "off course" if threshold is not 0 because the previous fire time for the next iteration might be a "compromise" between several triggers. Do you feel like this will be a problem? The "real" solution would be storing the "previous fire times" generated by all triggers but I am not at the point where I would consider stateful triggers as a feature.

[kaos]

How's that?
The trigger time ought to even out to get synchronized. Ah, it all comes down to how the trigger time to use is selected.
I'd say this would be "by design". With stateless triggers (as we want them) we only need to be clear how the trigger time is picked, when using a threshold.

I see a few options here:

a. Use the trigger time from the first trigger in the list. Always. If this is the interval trigger, it may drag, as you described. But if you specify the cron trigger first, then the interval trigger, you should be fine, as the interval trigger will get a "clean" even last trigger time, that we got from the cron trigger, and as such, will not veer off.

b. An additional option specifying which trigger (index) to be "normative" (which might default to the first, for instance)

c. Give the triggers a sense of "priority", which dictates which triggers time to actually use. Possibly as input parameter to the trigger itself, with a default for cron trigger to have a higher prio than interval. This would be the most flexible I think, but might be overkill.

[agronholm]

A bit after I wrote my previous comment I ended up attempting to fix this just like your first suggestion but I haven't got it to work yet.

[thor-the-norseman]

@agronholm Speaking of priorities, I have a use case for something like that:

I'm working on a project named ZASD (ZFS Automatic Snapshot Daemon) in which I have a set of cron schedules, each of which trigger filesystem snapshots. In a typical configuration, a group of schedules A, B, C, ..., will trigger snapshots in a repeating lockstep pattern, say ABC, B, BC, B, ABC, B, BC, B, ...

Crucially, each schedule is associated with a user-configured tag (such as 'weekly', 'daily' or 'hourly') a list of filesystems and a priority. When triggers coincide on the same filesystem, the trigger with the highest priority is picked and the rest are discarded, since only one snapshot is required at a time. Having redundant snapshots pollutes the system-wide snapshot list.

I have faced great difficulties in applying APScheduler to this use case without resorting to monkey patching your code.

Could you please:

a) Promise that coinciding triggers on an OrTrigger are picked in the order they were listed in, and provide a means for a job to discover which trigger object was picked?

b) Provide a means for a job to discover the full chain of triggers that caused it to execute?

Option A is simpler to implement, but isn't useful if the full trigger traversal path is significant to your application. Option B is more general and doesn't require any guarantees.

Discovery of the concrete Python objects isn't important. I would rather have a means of attaching generic metadata to jobs, triggers and schedules, and a means of retrieving it from inside an executing job.

[agronholm]

I have faced great difficulties in applying APScheduler to this use case without resorting to monkey patching your code.

Sounds like it might be easier for you to write your own trigger class then?

b) Provide a means for a job to discover the full chain of triggers that caused it to execute?

I had not intended to do that, as I don't have any idea how to fit that into the planned API (the combining triggers aren't special in that sense). While I don't think this is be what you were thinking of, there are plans to add tracing capabilities to triggers so you can see how their internal decision making works.

By the way, the major refactoring of the triggers for APScheduler 4.0 is done. The rest of the library does not work yet, but the triggers pass all the tests with 100% line coverage. Feel free to experiment!

[agronholm]

There is a threshold parameter for AndTrigger in 4.0, and the use case should've used CronTrigger(hour='7-22', minute='*/2') anyway.

[rafalkrupinski]

Wouldn't it be easier to add a param to IntervalTrigger to make it round the next fire time to a full second?

[agronholm]

You mean add special casing for IntervalTrigger? Then what about all the custom triggers people could have on their own?

[agronholm]

What I mean to say is, do you have a problem with the solution I came up with?

[rafalkrupinski]

I don't like the threshold, because it's not obvious what happens when threshold kicks in - is the earliest time used, latest or average? I need to check the documentation. Should the next fire time be calculated from the last actual fire time or from the last result of Interval.next()? There are no options to control these behaviors. Threshold seems very complex to implement right.

On the other hand, we already have all the tools to make it work - setting first fire time to the full second and not using µs in the interval.
Optionally there could be round-to-second parameter, that would save user from calculating the first fire time.

You mean add special casing for IntervalTrigger?

Interval is special case anyway, as it's the only built-in trigger with µs resolution, isn't it?

Then what about all the custom triggers people could have on their own?

If you want to combine your triggers in 3.x branch they already have to round fire times.
4.x is alpha - anything goes.

---

edit:
Ah, it's actually about alignment than resolution. My bad.

Still, all other (built-in) triggers align to full seconds don't they? So to combine any other trigger with them, they need to be second-aligned too. In general it should be users responsibility to align their triggers, but there could be an option to auto-align IntervalTrigger to full second.

Am I missing something?

[agronholm]

The threshold option is a substantially more flexible solution than rounding to the nearest second, as you can set a higher or lower threshold than one second.

Are you unsatisfied with how the combining triggers are documented (here)?