-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[UI] Show execution details in Run Details Page ML Metadata tab of steps #3457
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
import { | ||
Context, | ||
Api, | ||
Execution, | ||
getResourceProperty, | ||
ExecutionProperties, | ||
ExecutionCustomProperties, | ||
} from '@kubeflow/frontend'; | ||
import { | ||
GetContextByTypeAndNameRequest, | ||
GetExecutionsByContextRequest, | ||
} from '@kubeflow/frontend/src/mlmd/generated/ml_metadata/proto/metadata_store_service_pb'; | ||
|
||
async function getContext({ type, name }: { type: string; name: string }): Promise<Context> { | ||
const request = new GetContextByTypeAndNameRequest(); | ||
request.setTypeName(type); | ||
request.setContextName(name); | ||
try { | ||
const res = await Api.getInstance().metadataStoreService.getContextByTypeAndName(request); | ||
const context = res.getContext(); | ||
if (context == null) { | ||
throw new Error('Cannot find specified context'); | ||
} | ||
return context; | ||
} catch (err) { | ||
err.message = `Cannot find context with ${JSON.stringify(request.toObject())}: ` + err.message; | ||
throw err; | ||
} | ||
} | ||
|
||
/** | ||
* @throws error when network error, or not found | ||
*/ | ||
export async function getTfxRunContext(argoWorkflowName: string): Promise<Context> { | ||
// argoPodName has the general form "pipelineName-workflowId-executionId". | ||
// All components of a pipeline within a single run will have the same | ||
// "pipelineName-workflowId" prefix. | ||
const pipelineName = argoWorkflowName | ||
.split('-') | ||
.slice(0, -1) | ||
.join('_'); | ||
const runID = argoWorkflowName; | ||
// An example run context name is parameterized_tfx_oss.parameterized-tfx-oss-4rq5v. | ||
const tfxRunContextName = `${pipelineName}.${runID}`; | ||
return await getContext({ name: tfxRunContextName, type: 'run' }); | ||
} | ||
|
||
/** | ||
* @throws error when network error, or not found | ||
*/ | ||
export async function getKfpRunContext(argoWorkflowName: string): Promise<Context> { | ||
return await getContext({ name: argoWorkflowName, type: 'KfpRun' }); | ||
} | ||
|
||
/** | ||
* @throws error when network error | ||
*/ | ||
export async function getExecutionsFromContext(context: Context): Promise<Execution[]> { | ||
const request = new GetExecutionsByContextRequest(); | ||
request.setContextId(context.getId()); | ||
try { | ||
const res = await Api.getInstance().metadataStoreService.getExecutionsByContext(request); | ||
const list = res.getExecutionsList(); | ||
if (list == null) { | ||
throw new Error('response.getExecutionsList() is empty'); | ||
} | ||
return list; | ||
} catch (err) { | ||
err.message = | ||
`Cannot find executions by context ${context.getId()} with name ${context.getName()}: ` + | ||
err.message; | ||
throw err; | ||
} | ||
} | ||
|
||
export enum KfpExecutionProperties { | ||
KFP_POD_NAME = 'kfp_pod_name', | ||
} | ||
|
||
export const ExecutionHelpers = { | ||
getPipeline(execution: Execution): string | number | undefined { | ||
return ( | ||
getResourceProperty(execution, ExecutionProperties.PIPELINE_NAME) || | ||
getResourceProperty(execution, ExecutionCustomProperties.WORKSPACE, true) || | ||
getResourceProperty(execution, ExecutionCustomProperties.RUN_ID, true) || | ||
undefined | ||
); | ||
}, | ||
getName(execution: Execution): string | number | undefined { | ||
return ( | ||
getResourceProperty(execution, ExecutionProperties.NAME) || | ||
getResourceProperty(execution, ExecutionProperties.COMPONENT_ID) || | ||
getResourceProperty(execution, ExecutionCustomProperties.TASK_ID, true) || | ||
undefined | ||
); | ||
}, | ||
getState(execution: Execution): string | number | undefined { | ||
return getResourceProperty(execution, ExecutionProperties.STATE) || undefined; | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually this remind me of one thing. See https://github.com/tensorflow/tfx/blob/5ee3de9c77a92e904010c7656d6269d3b28744bf/tfx/orchestration/kubeflow/base_component.py#L136
Users can actually override the TFX run id to be something different than Argo workflow ID, which might cause problem to this piece of logic.
@neuromage Not sure how useful that customization is currently. Shall we consider closing the door of overriding run_id?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! that's a good catch.
From the comment
looks like it was for testing.
+1 for closing the door, or intentionally keep it only for testing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add something explicit like
workflow_name
that's not overridable?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean here or on the TFX side?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the TFX side.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I think I'll make a CL closing the door.