You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If using sky logs .. | grep ... in a pipe, then ctrl-c would print out a long stacktrace:
$ sky logs sky-spot-controller-abb92b6d 34 | grep 'preempted from'
(zh-14-26 pid=97928) I 09-13 04:21:03 recovery_strategy.py:355] Cluster is preempted from zone us-west1-a.
(zh-14-26 pid=97928) I 09-13 05:31:15 recovery_strategy.py:355] Cluster is preempted from zone us-west1-b.
^C--- Logging error ---
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/logging/__init__.py", line 1087, in emit
self.flush()
BrokenPipeError: [Errno 32] Broken pipe
Call stack:
File "/home/ubuntu/.local/bin/sky", line 8, in <module>
sys.exit(cli())
File "/home/ubuntu/.local/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/utils/common_utils.py", line 147, in _record
return f(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/cli.py", line 776, in invoke
return super().invoke(ctx)
File "/home/ubuntu/.local/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/.local/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/.local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/utils/common_utils.py", line 168, in _record
return f(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/cli.py", line 1210, in logs
core.tail_logs(cluster, job_id)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/core.py", line 354, in tail_logs
backend.tail_logs(handle, job_id)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/backends/cloud_vm_ray_backend.py", line 2403, in tail_logs
returncode = self.run_on_head(
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/utils/common_utils.py", line 168, in _record
return f(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/backends/cloud_vm_ray_backend.py", line 2692, in run_on_head
return runner.run(
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/utils/command_runner.py", line 255, in run
return log_lib.run_with_log(' '.join(command),
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/skylet/log_lib.py", line 233, in run_with_log
proc.wait()
File "/opt/conda/lib/python3.9/subprocess.py", line 1189, in wait
return self._wait(timeout=timeout)
File "/opt/conda/lib/python3.9/subprocess.py", line 1917, in _wait
(pid, sts) = self._try_wait(0)
File "/opt/conda/lib/python3.9/subprocess.py", line 1875, in _try_wait
(pid, sts) = os.waitpid(self.pid, wait_flags)
File "/home/ubuntu/.local/lib/python3.9/site-packages/sky/backends/backend_utils.py", line 1909, in interrupt_handler
logger.warning(f'{colorama.Fore.LIGHTBLACK_EX}The job will keep '
Message: '\x1b[90mThe job will keep running after Ctrl-C.\x1b[0m'
Arguments: ()
Shared connection to 44.210.146.121 closed.
Aborted!
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
The text was updated successfully, but these errors were encountered:
If using
sky logs .. | grep ...
in a pipe, then ctrl-c would print out a long stacktrace:The text was updated successfully, but these errors were encountered: