Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add buildAsync method for async graph compilation #266

Merged
merged 2 commits into from
May 20, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -480,10 +480,9 @@ that shares the same buffer as the input tensor. (In the case of reshape or sque
the entire data is shared, while in the case of slice, a part of the input data is shared.)
The implementation may use views, as above, for intermediate values.

The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method of the {{MLGraphBuilder}} interface is used to compile and optimize
the computation graph used to compute one or more specified outputs. The key
purpose of the compilation step is to enable optimizations that span two or
more operations, such as operation or loop fusion.
Before the execution, the computation graph that is used to compute one or more specified outputs needs to be compiled and optimized. The key purpose of the compilation step is to enable optimizations that span two or more operations, such as operation or loop fusion.

There are multiple ways by which the graph may be compiled. The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method compiles the graph immediately on the calling thread, which must be a worker thread running on CPU or GPU device, and returns an {{MLGraph}}. The {{MLGraphBuilder}}.{{MLGraphBuilder/buildAsync()}} method compiles the graph in background without blocking the calling thread, and returns a {{Promise}} that resolves to an {{MLGraph}}. Both compilation methods produce an {{MLGraph}} that represents a compiled graph for optimal execution.

Once the {{MLGraph}} is constructed, there are multiple ways by which the graph may be executed. The
{{MLContext}}.{{MLContext/compute()}} method represents a way the execution of the graph is carried out immediately
Expand Down Expand Up @@ -986,14 +985,17 @@ interface MLGraphBuilder {
// Create a single-value operand from the specified number of the specified type.
MLOperand constant(double value, optional MLOperandType type = "float32");

// Compile the graph up to the specified output operands
// Compile the graph up to the specified output operands synchronously.
[Exposed=(DedicatedWorker)]
MLGraph build(MLNamedOperands outputs);

// Compile the graph up to the specified output operands asynchronously.
Promise<MLGraph> buildAsync(MLNamedOperands outputs);
};
</script>

<div class="note">
The {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method compiles the graph builder state up to the specified output operands into a compiled graph according to the type of {{MLContext}} that creates it. Since this operation can be costly in some machine configurations, the calling thread must only be a worker thread to avoid potential disruption of the user experience. When the {{[[contextType]]}} of the {{MLContext}} is set to [=default-context|default=], the compiled graph is initialized right before the {{MLGraphBuilder/build()}} method call returns. This graph initialization stage is important for optimal performance of the subsequent graph executions. See [[#api-mlcommandencoder-graph-initialization]] for more detail.
Both {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} and {{MLGraphBuilder}}.{{MLGraphBuilder/buildAsync()}} methods compile the graph builder state up to the specified output operands into a compiled graph according to the type of {{MLContext}} that creates it. Since this operation can be costly in some machine configurations, the calling thread of the {{MLGraphBuilder}}.{{MLGraphBuilder/build()}} method must only be a worker thread to avoid potential disruption of the user experience. When the {{[[contextType]]}} of the {{MLContext}} is set to [=default-context|default=], the compiled graph is initialized right before the {{MLGraph}} is returned. This graph initialization stage is important for optimal performance of the subsequent graph executions. See [[#api-mlcommandencoder-graph-initialization]] for more detail.
</div>

### batchNormalization ### {#api-mlgraphbuilder-batchnorm}
Expand Down