Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: paychmgr: Support paych funding (a.k.a. fast paid retrieval) #7883

Merged
merged 30 commits into from
Mar 2, 2022
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
8e46b9e
paych: API to pre-fund channels
magik6k Jan 4, 2022
9715113
paych: Test pre-funding
magik6k Jan 4, 2022
2e76375
Fix paych itests
magik6k Jan 4, 2022
7a938b2
paych: Output FIL in cli
magik6k Jan 4, 2022
8f6f21c
paych: Print available amounts in paych status
magik6k Jan 4, 2022
ff8b95d
paych: Reserve flag for add-funds cli
magik6k Jan 4, 2022
1f2621b
Make retrieval work with reused channels
magik6k Jan 4, 2022
eab8fec
gen stuff
magik6k Jan 4, 2022
533349c
paych: Fix cli tests
magik6k Jan 5, 2022
b0e7bc1
paych: Cleanup available fund logic
magik6k Jan 5, 2022
5b585c0
paych: reset fundsReqQueue correctly
magik6k Jan 5, 2022
8b19b84
paych: option to force off-chain get
magik6k Jan 6, 2022
4235a97
retrieval: OffChainRetrieval config
magik6k Jan 6, 2022
8f9e730
paych: Better off-chain errors
magik6k Jan 6, 2022
550e274
paych: Don't return settling/collected chennals from OutboundActiveBy…
magik6k Jan 6, 2022
10af768
Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse
magik6k Feb 14, 2022
f61eb23
api: separate method for paych funding
magik6k Feb 14, 2022
df5bd14
paychmgr: Review comments
magik6k Feb 14, 2022
36a1934
paychmgr: Fix tests after api changes
magik6k Feb 14, 2022
f9485eb
Update paychmgr/store.go
magik6k Feb 14, 2022
da5ae1e
paychmgr: erlier checks in completeAmount
magik6k Feb 14, 2022
deb9882
Merge branch 'feat/paych-avail-reuse' of github.com:filecoin-project/…
magik6k Feb 14, 2022
60b9acc
gen
magik6k Feb 14, 2022
e1a3605
feat(markets): update markets to simplify client adapter
hannahhoward Feb 16, 2022
cdca1a4
Merge pull request #8102 from filecoin-project/feat/paych-avail-reuse…
magik6k Feb 16, 2022
681b907
Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse
magik6k Feb 16, 2022
e961766
fix lotus-soup build
magik6k Feb 16, 2022
3849995
paychmgr: AvailableAmt -> NonReservedAmt
magik6k Feb 16, 2022
a0cb609
Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse
magik6k Feb 22, 2022
e9a6f5f
Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse
magik6k Mar 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions api/api_full.go
Original file line number Diff line number Diff line change
Expand Up @@ -689,7 +689,17 @@ type FullNode interface {
// MethodGroup: Paych
// The Paych methods are for interacting with and managing payment channels

PaychGet(ctx context.Context, from, to address.Address, amt types.BigInt) (*ChannelInfo, error) //perm:sign
// PaychGet gets or creates a payment channel between address pair
magik6k marked this conversation as resolved.
Show resolved Hide resolved
// The specified amount will be reserved for use. If there aren't enough non-reserved funds
// available, funds will be added through an on-chain message.
// - When opts.OffChain is true, this call will not cause any messages to be sent to the chain (no automatic
// channel creation/funds adding). If the operation can't be performed without sending a message an error will be
// returned. Note that even when this option is specified, this call can be blocked by previous operations on the
// channel waiting for on-chain operations.
PaychGet(ctx context.Context, from, to address.Address, amt types.BigInt, opts PaychGetOpts) (*ChannelInfo, error) //perm:sign
// PaychFund gets or creates a payment channel between address pair.
// The specified amount will be added to the channel through on-chain send for future use
PaychFund(ctx context.Context, from, to address.Address, amt types.BigInt) (*ChannelInfo, error) //perm:sign
PaychGetWaitReady(context.Context, cid.Cid) (address.Address, error) //perm:sign
PaychAvailableFunds(ctx context.Context, ch address.Address) (*ChannelAvailableFunds, error) //perm:sign
PaychAvailableFundsByFromTo(ctx context.Context, from, to address.Address) (*ChannelAvailableFunds, error) //perm:sign
Expand Down Expand Up @@ -828,6 +838,10 @@ const (
PCHOutbound
)

type PaychGetOpts struct {
OffChain bool
}

type PaychStatus struct {
ControlAddr address.Address
Direction PCHDir
Expand All @@ -845,16 +859,23 @@ type ChannelAvailableFunds struct {
From address.Address
// To is the to address of the channel
To address.Address
// ConfirmedAmt is the amount of funds that have been confirmed on-chain
// for the channel

// ConfirmedAmt is the total amount of funds that have been confirmed on-chain for the channel
ConfirmedAmt types.BigInt
// PendingAmt is the amount of funds that are pending confirmation on-chain
PendingAmt types.BigInt

// NonReservedAmt is part of ConfirmedAmt that is available for use (e.g. when the payment channel was pre-funded)
NonReservedAmt types.BigInt
// PendingAvailableAmt is the amount of funds that are pending confirmation on-chain that will become available once confirmed
PendingAvailableAmt types.BigInt

// PendingWaitSentinel can be used with PaychGetWaitReady to wait for
// confirmation of pending funds
PendingWaitSentinel *cid.Cid
// QueuedAmt is the amount that is queued up behind a pending request
QueuedAmt types.BigInt

// VoucherRedeemedAmt is the amount that is redeemed by vouchers on-chain
// and in the local datastore
VoucherReedeemedAmt types.BigInt
Expand Down
23 changes: 19 additions & 4 deletions api/mocks/mock_full.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 17 additions & 4 deletions api/proxy_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions api/v0api/v1_wrapper.go
Original file line number Diff line number Diff line change
Expand Up @@ -337,4 +337,8 @@ func (w *WrapperV1Full) clientRetrieve(ctx context.Context, order RetrievalOrder
finish(w.ClientExport(ctx, eref, *ref))
}

func (w *WrapperV1Full) PaychGet(ctx context.Context, from, to address.Address, amt types.BigInt) (*api.ChannelInfo, error) {
return w.FullNode.PaychFund(ctx, from, to, amt)
}

var _ FullNode = &WrapperV1Full{}
Binary file modified build/openrpc/full.json.gz
Binary file not shown.
Binary file modified build/openrpc/miner.json.gz
Binary file not shown.
Binary file modified build/openrpc/worker.json.gz
Binary file not shown.
34 changes: 23 additions & 11 deletions cli/paych.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"sort"
"strings"

"github.com/filecoin-project/lotus/api"
lapi "github.com/filecoin-project/lotus/api"

"github.com/filecoin-project/lotus/paychmgr"

Expand Down Expand Up @@ -39,12 +39,15 @@ var paychAddFundsCmd = &cli.Command{
Usage: "Add funds to the payment channel between fromAddress and toAddress. Creates the payment channel if it doesn't already exist.",
ArgsUsage: "[fromAddress toAddress amount]",
Flags: []cli.Flag{

&cli.BoolFlag{
Name: "restart-retrievals",
Usage: "restart stalled retrieval deals on this payment channel",
Value: true,
},
&cli.BoolFlag{
Name: "reserve",
Usage: "mark funds as reserved",
},
},
Action: func(cctx *cli.Context) error {
if cctx.Args().Len() != 3 {
Expand All @@ -66,7 +69,7 @@ var paychAddFundsCmd = &cli.Command{
return ShowHelp(cctx, fmt.Errorf("parsing amount failed: %s", err))
}

api, closer, err := GetFullNodeAPI(cctx)
api, closer, err := GetFullNodeAPIV1(cctx)
if err != nil {
return err
}
Expand All @@ -76,7 +79,14 @@ var paychAddFundsCmd = &cli.Command{

// Send a message to chain to create channel / add funds to existing
// channel
info, err := api.PaychGet(ctx, from, to, types.BigInt(amt))
var info *lapi.ChannelInfo
if cctx.Bool("reserve") {
info, err = api.PaychGet(ctx, from, to, types.BigInt(amt), lapi.PaychGetOpts{
OffChain: false,
})
} else {
info, err = api.PaychFund(ctx, from, to, types.BigInt(amt))
}
if err != nil {
return err
}
Expand Down Expand Up @@ -163,13 +173,13 @@ var paychStatusCmd = &cli.Command{
},
}

func paychStatus(writer io.Writer, avail *api.ChannelAvailableFunds) {
func paychStatus(writer io.Writer, avail *lapi.ChannelAvailableFunds) {
if avail.Channel == nil {
if avail.PendingWaitSentinel != nil {
fmt.Fprint(writer, "Creating channel\n")
fmt.Fprintf(writer, " From: %s\n", avail.From)
fmt.Fprintf(writer, " To: %s\n", avail.To)
fmt.Fprintf(writer, " Pending Amt: %d\n", avail.PendingAmt)
fmt.Fprintf(writer, " Pending Amt: %s\n", types.FIL(avail.PendingAmt))
fmt.Fprintf(writer, " Wait Sentinel: %s\n", avail.PendingWaitSentinel)
return
}
Expand All @@ -189,10 +199,12 @@ func paychStatus(writer io.Writer, avail *api.ChannelAvailableFunds) {
{"Channel", avail.Channel.String()},
{"From", avail.From.String()},
{"To", avail.To.String()},
{"Confirmed Amt", fmt.Sprintf("%d", avail.ConfirmedAmt)},
{"Pending Amt", fmt.Sprintf("%d", avail.PendingAmt)},
{"Queued Amt", fmt.Sprintf("%d", avail.QueuedAmt)},
{"Voucher Redeemed Amt", fmt.Sprintf("%d", avail.VoucherReedeemedAmt)},
{"Confirmed Amt", fmt.Sprintf("%s", types.FIL(avail.ConfirmedAmt))},
{"Available Amt", fmt.Sprintf("%s", types.FIL(avail.NonReservedAmt))},
{"Voucher Redeemed Amt", fmt.Sprintf("%s", types.FIL(avail.VoucherReedeemedAmt))},
{"Pending Amt", fmt.Sprintf("%s", types.FIL(avail.PendingAmt))},
{"Pending Available Amt", fmt.Sprintf("%s", types.FIL(avail.PendingAvailableAmt))},
{"Queued Amt", fmt.Sprintf("%s", types.FIL(avail.QueuedAmt))},
}
if avail.PendingWaitSentinel != nil {
nameValues = append(nameValues, []string{
Expand Down Expand Up @@ -576,7 +588,7 @@ func outputVoucher(w io.Writer, v *paych.SignedVoucher, export bool) error {
}
}

fmt.Fprintf(w, "Lane %d, Nonce %d: %s", v.Lane, v.Nonce, v.Amount.String())
fmt.Fprintf(w, "Lane %d, Nonce %d: %s", v.Lane, v.Nonce, types.FIL(v.Amount))
if export {
fmt.Fprintf(w, "; %s", enc)
}
Expand Down
4 changes: 4 additions & 0 deletions documentation/en/api-v0-methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -4061,6 +4061,8 @@ Response:
"To": "f01234",
"ConfirmedAmt": "0",
"PendingAmt": "0",
"NonReservedAmt": "0",
"PendingAvailableAmt": "0",
"PendingWaitSentinel": null,
"QueuedAmt": "0",
"VoucherReedeemedAmt": "0"
Expand Down Expand Up @@ -4088,6 +4090,8 @@ Response:
"To": "f01234",
"ConfirmedAmt": "0",
"PendingAmt": "0",
"NonReservedAmt": "0",
"PendingAvailableAmt": "0",
"PendingWaitSentinel": null,
"QueuedAmt": "0",
"VoucherReedeemedAmt": "0"
Expand Down
45 changes: 43 additions & 2 deletions documentation/en/api-v1-unstable-methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@
* [PaychAvailableFunds](#PaychAvailableFunds)
* [PaychAvailableFundsByFromTo](#PaychAvailableFundsByFromTo)
* [PaychCollect](#PaychCollect)
* [PaychFund](#PaychFund)
* [PaychGet](#PaychGet)
* [PaychGetWaitReady](#PaychGetWaitReady)
* [PaychList](#PaychList)
Expand Down Expand Up @@ -4456,6 +4457,8 @@ Response:
"To": "f01234",
"ConfirmedAmt": "0",
"PendingAmt": "0",
"NonReservedAmt": "0",
"PendingAvailableAmt": "0",
"PendingWaitSentinel": null,
"QueuedAmt": "0",
"VoucherReedeemedAmt": "0"
Expand Down Expand Up @@ -4483,6 +4486,8 @@ Response:
"To": "f01234",
"ConfirmedAmt": "0",
"PendingAmt": "0",
"NonReservedAmt": "0",
"PendingAvailableAmt": "0",
"PendingWaitSentinel": null,
"QueuedAmt": "0",
"VoucherReedeemedAmt": "0"
Expand All @@ -4508,8 +4513,10 @@ Response:
}
```

### PaychGet
There are not yet any comments for this method.
### PaychFund
PaychFund gets or creates a payment channel between address pair.
The specified amount will be added to the channel through on-chain send for future use


Perms: sign

Expand All @@ -4532,6 +4539,40 @@ Response:
}
```

### PaychGet
PaychGet gets or creates a payment channel between address pair
The specified amount will be reserved for use. If there aren't enough non-reserved funds
available, funds will be added through an on-chain message.
- When opts.OffChain is true, this call will not cause any messages to be sent to the chain (no automatic
channel creation/funds adding). If the operation can't be performed without sending a message an error will be
returned. Note that even when this option is specified, this call can be blocked by previous operations on the
channel waiting for on-chain operations.


Perms: sign

Inputs:
```json
[
"f01234",
"f01234",
"0",
{
"OffChain": true
}
]
```

Response:
```json
{
"Channel": "f01234",
"WaitSentinel": {
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
}
}
```

### PaychGetWaitReady


Expand Down
1 change: 1 addition & 0 deletions documentation/en/cli-lotus.md
Original file line number Diff line number Diff line change
Expand Up @@ -1347,6 +1347,7 @@ USAGE:

OPTIONS:
--restart-retrievals restart stalled retrieval deals on this payment channel (default: true)
--reserve mark funds as reserved (default: false)
--help, -h show help (default: false)

```
Expand Down
8 changes: 8 additions & 0 deletions documentation/en/default-lotus-config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,14 @@
# env var: LOTUS_CLIENT_SIMULTANEOUSTRANSFERSFORRETRIEVAL
#SimultaneousTransfersForRetrieval = 20

# Require that retrievals perform no on-chain operations. Paid retrievals
# without existing payment channels with available funds will fail instead
# of automatically performing on-chain operations.
#
# type: bool
# env var: LOTUS_CLIENT_OFFCHAINRETRIEVAL
#OffChainRetrieval = false


[Wallet]
# type: string
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ require (
github.com/filecoin-project/go-data-transfer v1.14.0
github.com/filecoin-project/go-fil-commcid v0.1.0
github.com/filecoin-project/go-fil-commp-hashhash v0.1.0
github.com/filecoin-project/go-fil-markets v1.19.0
github.com/filecoin-project/go-fil-markets v1.19.1
github.com/filecoin-project/go-jsonrpc v0.1.5
github.com/filecoin-project/go-padreader v0.0.1
github.com/filecoin-project/go-paramfetch v0.0.4
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -327,8 +327,8 @@ github.com/filecoin-project/go-fil-commcid v0.1.0 h1:3R4ds1A9r6cr8mvZBfMYxTS88Oq
github.com/filecoin-project/go-fil-commcid v0.1.0/go.mod h1:Eaox7Hvus1JgPrL5+M3+h7aSPHc0cVqpSxA+TxIEpZQ=
github.com/filecoin-project/go-fil-commp-hashhash v0.1.0 h1:imrrpZWEHRnNqqv0tN7LXep5bFEVOVmQWHJvl2mgsGo=
github.com/filecoin-project/go-fil-commp-hashhash v0.1.0/go.mod h1:73S8WSEWh9vr0fDJVnKADhfIv/d6dCbAGaAGWbdJEI8=
github.com/filecoin-project/go-fil-markets v1.19.0 h1:kap2q2wTM6tfkVO5gMA5DD9GUeTvkDhMfhjCtEwMDM8=
github.com/filecoin-project/go-fil-markets v1.19.0/go.mod h1:qsb3apmo4RSJYCEq40QxVdU7UZospN6nFJLOBHuaIbc=
github.com/filecoin-project/go-fil-markets v1.19.1 h1:o5sziAp8zCsvIg3KYMgIpwm8gyOl4MDzEKEf0Qq5L3U=
github.com/filecoin-project/go-fil-markets v1.19.1/go.mod h1:qsb3apmo4RSJYCEq40QxVdU7UZospN6nFJLOBHuaIbc=
github.com/filecoin-project/go-hamt-ipld v0.1.5 h1:uoXrKbCQZ49OHpsTCkrThPNelC4W3LPEk0OrS/ytIBM=
github.com/filecoin-project/go-hamt-ipld v0.1.5/go.mod h1:6Is+ONR5Cd5R6XZoCse1CWaXZc0Hdb/JeX+EQCQzX24=
github.com/filecoin-project/go-hamt-ipld/v2 v2.0.0 h1:b3UDemBYN2HNfk3KOXNuxgTTxlWi3xVvbQP0IT38fvM=
Expand Down
4 changes: 3 additions & 1 deletion itests/paych_api_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,9 @@ func TestPaymentChannelsAPI(t *testing.T) {
require.NoError(t, err)

channelAmt := int64(7000)
channelInfo, err := paymentCreator.PaychGet(ctx, createrAddr, receiverAddr, abi.NewTokenAmount(channelAmt))
channelInfo, err := paymentCreator.PaychGet(ctx, createrAddr, receiverAddr, abi.NewTokenAmount(channelAmt), api.PaychGetOpts{
OffChain: false,
})
require.NoError(t, err)

channel, err := paymentCreator.PaychGetWaitReady(ctx, channelInfo.WaitSentinel)
Expand Down
Loading