ENH: Add a check whether scheduling is stuck

[Flying-Tom]

What do these changes do?

Use stagemonitor to trace and cancel those stale subtasks

Related issue number

Fixes #489

Check code requirements

• tests added / passed (if needed)
• Ensure all linting tests pass

[codecov]

Codecov Report

Merging #695 (d022c82) into main (fe7caba) will decrease coverage by 3.76%.
The diff coverage is 58.97%.

@@            Coverage Diff             @@
##             main     #695      +/-   ##
==========================================
- Coverage   93.50%   89.75%   -3.76%     
==========================================
  Files        1025     1025              
  Lines       79354    79377      +23     
  Branches    16441    16443       +2     
==========================================
- Hits        74203    71244    -2959     
- Misses       3466     6268    +2802     
- Partials     1685     1865     +180     

Flag	Coverage Δ
unittests	89.64% <58.97%> (-3.76%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...orbits/_mars/services/scheduling/worker/service.py	100.00% <ø> (ø)
...bits/_mars/services/scheduling/worker/execution.py	86.59% <58.97%> (-11.62%)	⬇️

... and 127 files with indirect coverage changes

[aresnow1]

test_mem_quota_allocation failed.

[aresnow1]

Great job! LGTM