Arweave microservice

[alexcos20]

See oceanprotocol/uploader_backend#1 for big picture

On every 10 minutes, Arweave microservice should register itself to DBS, using the register endpoint.
DBS_URL will be defined as env.

POST  DBS_URL/register

{
    "type": "arweave",
    "description":  "File storage on Arweave ",
    "url": "http://microservice.url",
    "payment":
            [ 
                {
                    "chainId": 1,
                    "acceptedTokens": 
                        [ 
                            "OCEAN": "0xOCEAN_on_MAINNET",
                            "DAI": "0xDAI_ON_MAINNET"
                        ]
                },
                {
                    "chainId": "polygon_chain_id",
                    "acceptedTokens": 
                        [ 
                            "OCEAN": "0xOCEAN_on_POLYGON",
                            "DAI": "0xDAI_ON_POLYGON"
                        ]
                }
            ]
}  

Microservice API Endpoints:

GetQuote

Description: Gets a quote in order to store some files

Path: POST /getQuote

Arguments:

{
    "type": "arweave",
    "files": [
                {"length":2343545},
                {"length":2343545},
            ],
    "duration": 4353545453
    "payment": {
        "chainId": 1,
        "tokenAddress": "0xOCEAN_on_MAINNET"
    },
    "userAddress": "0x456"
}

Where:

• type: type of storage desired
• files : array with files lenght
• duration: how long to store this files (in seconds)
• payment.chainId: chainId that will be used to make the payment
• payment.token: token that will be used to make the payment
• userAddress:

Returns:

{
    "tokenAmount": 500,
    "approveAddress": "0x123",
    "chainId": 1,
    "tokenAddress": "0xOCEAN_on_MAINNET",
    "quoteId": "xxxx"
}  

Where:

- tokenAmount - tokenAmount that needs to be approved for "approveAddress" on tokenAddress  (backend server will do a transferFrom to get the payment)
- chainId - chainId used for payment
- quoteId  - backend server will generate a quoteId

Upload

Description: Upload some files

Path: POST /upload

Input:

{
   "quoteId": "23",
   "nonce":"1",
   "signature":  "2222",
  "files":[
       "ipfs://xxxx","ipfs://yyyy"]
}

Microservice will upload files to Arweave and it will take the payment
Returns: 200 OK if succeded

GetStatus

Description: Gets status for a job

Path: POST /getStatus?quoteId=xxx

Returns:

{
    "status": 0,
}  

Where:

Status	Status Description
0	No such quote
1	Waiting for files to be uploaded by the user
2	Processing payment
3	Processing done
4	Uploading files to storage
5	Upload done

GetLink

Description: Gets DDO files object for a job

Path: POST /getLink?quoteId=xxx&nonce=1&signature=0xXXXXX

Input:

• quoteId
• nonce (timestamp) (has to be higher then previous stored nonce for this user)
• signature: user signed hash of SHA256(quoteID+nonce)
  Returns:

[
    {
       "type": "arweave",
       "transactionHash": "xxxx",
    }
]

[MantisClone]

@alexcos20 The Upload input says

  "files":[
       "ipfs://xxxx","ipfs://yyyy"]

Does this mean we receive IPFS URIs and we need to download them before uploading to Arweave?

[MantisClone]

@alexcos20 Do you want us to submit our PR to this repo? Or do you want to create a new arweave-upload repo? I'd recommend the latter.

[MantisClone]

@alexcos20 Do you want us to implement this service in Python and Flask, or in Node.js and Express?

Details:
If we use Python and Flask, we think that the file-to-be-uploaded will need to be stored on disk before being uploaded. The pybundlr interface developed by @trentmc uses the Bundlr CLI which doesn't seem to support piped files (we tried it).

If we use Node.js and Express, we think that the Bundlr JS APIs would allow us to keep the file-to-be-uploaded in memory.

[MantisClone]

@alexcos20 We (@corrie-sloot and I) believe that getLink and getStatus should be GET methods. Do you agree?

[kremalicious]

No REST API path should have "get" in their API path. Here were my suggestions for the API paths:
oceanprotocol/uploader_backend#1 (comment)

[MantisClone]

At today's Core Tech Working group meeting @alexcos20 answered some questions:

1. Upload files input is a list of URIs. We need to download the file before uploading to Arweave.
2. @alexcos20 will create an arweave-upload (or other similarly named) repo. @corrie-sloot and I will submit PRs to the new repo.
3. We can write the microservice in Node.js. We don't need to use Python the pybundlr interface written by @trentmc (it was created for a different purpose anyway)
4. @alexcos20 agrees that getLink and getStatus should be GET methods.
5. @alexcos20 agrees that the REST API paths should not include the word get. He said he'd update the spec.
6. @alexcos20 explained that the Arweave Upload service must pull funds out of the user's account with transferFrom and then upload the file on the user's behalf. The market should not interact with the Bundlr APIs at all.

[MantisClone]

@alexcos20 When you're finished updating the Arweave microservice endpoint paths (to remove get from getQuote) and updated the getLink and getStatus to be GET requests rather than POST, can you please let @corrie-sloot and I know?