Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline not starting on SciServer Compute #1849

Closed
brollb opened this issue Aug 6, 2020 · 2 comments
Closed

Pipeline not starting on SciServer Compute #1849

brollb opened this issue Aug 6, 2020 · 2 comments
Milestone

Comments

@brollb
Copy link
Contributor

brollb commented Aug 6, 2020

It appears that for some users (such as kmoore), running a pipeline on SciServer Compute hangs indefinitely. Looking at the server logs shows:

2020-08-06T16:59:31.823Z - error: [gme:server:worker:simpleworker:pid_514591:WorkerFunctions:PluginManagerBase:plugin:ExecutePipeline] Could not execute "GetCifar10Data": Error: SciServer Files request failed: Error in job definition

As this appears to be specific to this user, I suspect the error is happening on the SciServer side. However, this should be handled better by deepforge so we don't need to go to the server logs.

@brollb brollb added this to the v2.4.0 milestone Aug 6, 2020
@umesh-timalsina
Copy link
Contributor

The issue seems to be that the sciserver docker image DeepForge is accessible to some users while not for other users. So, I would recharacterize this issue not to be a code related bug in deepforge but rather an issue with sciserver's docker image for deepforge access to every account.

@brollb
Copy link
Contributor Author

brollb commented Aug 6, 2020

Thanks, @umesh-timalsina for finding the source of the issue. I am going to leave this issue since the error is not handled properly in deepforge. (The pipeline should show the error when an error occurs during job creation.)

brollb added a commit that referenced this issue Aug 6, 2020

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@brollb brollb closed this as completed in a8db90f Aug 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants