Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add compute getResult #983

Merged
merged 10 commits into from
Nov 29, 2021
Merged

add compute getResult #983

merged 10 commits into from
Nov 29, 2021

Conversation

alexcos20
Copy link
Member

@alexcos20 alexcos20 commented Aug 23, 2021

Closes #982

  1. Removes signature parameter of ocean.compute.status (not used anymore)
  2. Add ocean.compute.getResult to get the file content of index X
  3. ocean.provider exposes a computeLimits object defined as:

export interface ComputeLimits {
algoTimeLimit?: string
storageExpiry?: string
}
(both expressed in seconds)

this allows the market to expose computeLimits to users

@alexcos20 alexcos20 added Status: InProgress Work in progress, don't merge Breaking Breaking changes labels Aug 23, 2021
@alexcos20 alexcos20 self-assigned this Aug 23, 2021
@alexcos20 alexcos20 marked this pull request as ready for review August 23, 2021 07:18
@alexcos20
Copy link
Member Author

Please DO NOT merge, needs updated versions of provider/C2D components.
After approval, we are going to deploy all new components at a specific time

@alexcos20 alexcos20 removed the Status: InProgress Work in progress, don't merge label Aug 27, 2021
@alexcos20 alexcos20 removed the Status: InProgress Work in progress, don't merge label Sep 17, 2021
@mihaisc
Copy link
Contributor

mihaisc commented Sep 17, 2021

what is algoTimeLimit ?

@alexcos20
Copy link
Member Author

what is algoTimeLimit ?

Maximum amount of time that an algorithm is allowed to run in a C2d env. If service timeout > algoTimeLimit, then algoTimeLimit is enforced

this.logger.error(e)
throw e
}
return destination
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the point of returning this? You have destination as a parameter and then it's never reassigned

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, it's overkill. any result !== null means that the download succeeded

* @param {number} index Compute Result index
* @param {string} destination: destination folder
* @param {string} did Decentralized identifier.
* @param {DDO} ddo If undefined then the ddo will be fetched by did, this is just to optimize network calls
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so sounds like a param like asset: DDO | DID | string and then check what you get passed would be nicer pattern here to limit parameters

* @param {string} destination: destination folder
* @param {string} did Decentralized identifier.
* @param {DDO} ddo If undefined then the ddo will be fetched by did, this is just to optimize network calls
* @param {ServiceCompute} service If undefined then we get the service from the ddo
Copy link
Contributor

@kremalicious kremalicious Sep 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where else would we get it from? Why can another compute service be passed to this method which seems to be then different from the DDO compute service? What's the use case for this? Something like "I want the results for a random compute service I just came up with, but for this existing DDO which does not have this compute service"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's for use cases where you already have the service, but not the ddo. This was actually from the market

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I see this right, we always have full DDOs in this getComputeJobs method:
https://github.com/oceanprotocol/market/blob/main/src/utils/compute.ts#L257

we just don't pass it as part of ComputeJobMetaData but we actually could:
https://github.com/oceanprotocol/market/blob/main/src/utils/compute.ts#L188

Cause then we could remove that passing only a service use case from here. By always requiring passing of an asset which then is either a DDO or just string/DID we have made 3 parameters into 1 too

@codeclimate
Copy link

codeclimate bot commented Sep 27, 2021

Code Climate has analyzed commit ee75240 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 84.6% (50% is the threshold).

This pull request will bring the total coverage in the repository to 81.1% (0.1% change).

View more on Code Climate.

index: number,
destination: string,
account: Account
): Promise<any> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on typings and underlying code this method returns a string

did?: string,
ddo?: DDO,
service?: ServiceCompute
): Promise<any> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on underlying typing and code of provider.computeResult() this returns a Promise<string>

@kremalicious
Copy link
Contributor

kremalicious commented Sep 27, 2021

So main use case to me would be: "I have a jobId and a ddo and I want to get all results for it." How do I know how many index I have to iterate through? What does user see if we iterate through 20 results? 20 download prompts in browser?

@mihaisc
Copy link
Contributor

mihaisc commented Sep 27, 2021

@kremalicious the flow is the following

  • call status get a list of ComputeJob
  • display list of jobs like we do now, but you don't need to sign to get this list ( you currently do)
    image
  • when you click on a result you call getResult with the jobId that you have and did (or ddo , or service) . You will need to sign and after that the download will begin.

The UI will not change. The difference is that we now call status with different params twice and in the new version it will be status + getResult

@kremalicious
Copy link
Contributor

kremalicious commented Sep 27, 2021

the flow is the following

perfect, thanks for explanation! So that's already good guidance on what needs to be done over in market for this

@mihaisc
Copy link
Contributor

mihaisc commented Sep 27, 2021

Already linked this in the issue

@alexcos20 alexcos20 merged commit 5666ded into main Nov 29, 2021
@alexcos20 alexcos20 deleted the feature/secure_c2d_output branch November 29, 2021 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Secure C2D Outputs
3 participants