[WB-7432] reduce pyagent heartbeat rate

[dannygoldstein]

https://wandb.atlassian.net/browse/WB-7432

Description

wandb.agent() currently heartbeats 50-100 times per second, when only 1 heartbeat per 5 seconds is needed. This is because the heartbeat is in the path of a while/continue loop with no sleep. This PR adds a 5 second sleep to this path to ensure that heartbeats are emitted no more than once per 5 seconds in this path

Testing

How was this PR tested?
Manually

Release Notes

Below, please enter user-facing release notes as one or more bullet points.
If your change is not user-visible, write NO RELEASE NOTES instead, with no bullet points.

------------- BEGIN RELEASE NOTES ------------------
Fixes an issue where wandb.agent() sent too many heartbeats to the server, sometimes leading to rate limits
------------- END RELEASE NOTES --------------------

[vanpelt]

May want to consider changing this to like 5 seconds, I think it's unlikely that would cause much degradation to the user experience.

[dannygoldstein]

May want to consider changing this to like 5 seconds, I think it's unlikely that would cause much degradation to the user experience.

done

[codecov]

Codecov Report

Merging #2923 (8869fd5) into master (8225a30) will increase coverage by 0.02%.
The diff coverage is 90.90%.

@@            Coverage Diff             @@
##           master    #2923      +/-   ##
==========================================
+ Coverage   78.67%   78.69%   +0.02%     
==========================================
  Files         194      194              
  Lines       26491    26490       -1     
==========================================
+ Hits        20841    20847       +6     
+ Misses       5650     5643       -7     

Flag	Coverage Δ
functest	56.04% <90.90%> (+0.03%)	⬆️
unittest	70.52% <54.54%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
wandb/agents/pyagent.py	71.14% <90.90%> (-0.12%)	⬇️
wandb/sdk/lib/git.py	75.86% <0.00%> (ø)
wandb/sdk/internal/sender.py	91.92% <0.00%> (+0.13%)	⬆️
wandb/filesync/step_prepare.py	94.28% <0.00%> (+1.42%)	⬆️
wandb/sdk/internal/meta.py	90.18% <0.00%> (+3.06%)	⬆️

[raubitsj]

Thanks, merging