Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gnorkle): improve README, add proof of ownership system #2822

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 79 additions & 15 deletions examples/gno.land/p/demo/gnorkle/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,39 @@
# gnorkle

gnorkle is an attempt at a generalized oracle implementation. The gnorkle `Instance` definitions lives in the `gnorkle` directory. It can be effectively initialized by using the `gnorkle.NewInstance` constructor. The owner of the instance is then able to add feeds with or without whitelists, manage the instance's own whitelist, remove feeds, get the latest feed values, and handle incoming message requests that result in some type of action taking place.
gnorkle is an attempt at a generalized oracle implementation. The gnorkle
`Instance` definitions live in the `gnorkle` directory. It can be effectively
initialized by using the `gnorkle.NewInstance()` constructor.

An example of gnorkle in action can be found in the `examples/gno.land/r/gnoland/ghverify` realm. Any realm that wishes to integrate the minimum oracle functionality should include functions that relay incoming messages to the oracle instance's message handler and that can produce the latest feed values.
The gnorkle `Instance` can contain one or more data [Feeds](#feeds).

// when you make an instance, what do you do next?

give a general short overview of the process of creating


The owner of the
instance is then able to add feeds with or without whitelisted agents, manage
the instance's own whitelist, remove feeds, get the latest feed values, and handle
incoming message requests that result in some type of action taking place.

An example of gnorkle in action can be found in the
`examples/gno.land/r/gnoland/ghverify` realm. Any realm that wishes to integrate
the minimum oracle functionality should include functions that relay incoming
messages to the oracle instance's message handler and that can produce the
latest feed values.

## Feeds

Feeds that are added to an oracle must implement the `gnorkle.Feed` interface. A feed can be thought of as a mechanism that takes in off-chain data and produces data to be consumed on-chain. The three important components of a feed are:
- Tasks: a feed is composed of one or more `feed.Task` instances. These tasks ultimately instruct an agent what it needs to do to provide data to a feed for ingestion.
- Ingester: a feed should have a `gnorkle.Ingester` instance that handles incoming data from agents. The ingester's role is to perform the correct action on the incoming data, like adding it to an aggregate or storing it for later evaluation.
- Storage: a feed should have a `gnorkle.Storage` instance that manages the state of the data a feed publishes -- data to be consumed on-chain.
Feeds that are added to an oracle must implement the `gnorkle.Feed` interface.
A feed can be thought of as a mechanism that takes in off-chain data and produces
data to be consumed on-chain. The three important components of a feed are:
- Tasks: a feed is composed of one or more `feed.Task` instances. These tasks
ultimately instruct an agent what it needs to do to provide data to a feed for ingestion.
- Ingester: a feed should have a `gnorkle.Ingester` instance that handles
incoming data from agents. The ingester's role is to perform the correct
action on the incoming data, like adding it to an aggregate or storing it for later evaluation.
- Storage: a feed should have a `gnorkle.Storage` instance that manages the
state of the data a feed publishes -- data to be consumed on-chain.

A single oracle instance may be composed of many feeds, each of which has a unique ID.

Expand All @@ -25,10 +49,17 @@ The only feed currently implemented is the `feeds/static.Feed` type.

## Tasks

It's not hard to be a task -- just implement the one method `feed.Task` interface. On-chain task definitions should not do anything other than store data and be able to marshal that data to JSON. Of course, it is also useful if the task marshal's a `type` field as part of the JSON object so an agent is able to know what type of task it is dealing with and what data to expect in the payload.
It's not hard to be a task -- just implement the one method `feed.Task` interface.
On-chain task definitions should not do anything other than store data and be
able to marshal that data to JSON. Of course, it is also useful if the task
marshal's a `type` field as part of the JSON object so an agent is able to know
what type of task it is dealing with and what data to expect in the payload.

### Example use case
Imagine there is a public API out there. The oracle defines a task with a type of `HTTPGET`. The agent interacting with the oracle knows how to extract the task's type and extract the rest of the data to complete the task

Imagine there is a public API out there. The oracle defines a task with a
type of `HTTPGET`. The agent interacting with the oracle knows how to extract
the task's type and extract the rest of the data to complete the task
```go
type apiGet struct {
taskType string
Expand All @@ -42,24 +73,36 @@ apiGet {
"url": "http://example.com/api/latest"
}
```
The agent can use this data to make the request and publish the results to the oracle. Tasks can have structures as complex or simple as is required.
The agent can use this data to make the request and publish the results to the
oracle. Tasks can have structures as complex or simple as is required.

## Ingesters

An ingester's primary role is to receive data provided by agents and figure out what to do with it. Ingesters must implement the `gnorkle.Ingester` interface. There are currently two message function types that an ingester may want to handle, `message.FuncTypeIngest` and `message.FuncTypeCommit`. The former message type should result in the ingester accumulating data in its own data store, while the latter should use what it has in its data store to publish a feed value to the `gnorkle.Storage` instance provided to it.
An ingester's primary role is to receive data provided by agents and figure out
what to do with it. Ingesters must implement the `gnorkle.Ingester` interface.
There are currently two message function types that an ingester may want to
handle, `message.FuncTypeIngest` and `message.FuncTypeCommit`. The former
message type should result in the ingester accumulating data in its own data
store, while the latter should use what it has in its data store to publish a
feed value to the `gnorkle.Storage` instance provided to it.

The only ingester currently implemented is the `ingesters/single.ValueIngester` type.

### Example use case
The `Ingester` interface has two main methods that must be implemented, `Ingest` and `CommitValue`, as defined here:
The `Ingester` interface has two main methods that must be implemented, `Ingest`
and `CommitValue`, as defined here:
```go
type Ingester interface {
Type() ingester.Type
Ingest(value, providerAddress string) (canAutoCommit bool)
CommitValue(storage Storage, providerAddress string)
}
```
Consider an oracle that provides a price feed. This price feed would need an ingester capable of ingesting incoming values and producing a final result at the end of a given amount of time. So we may have something that looks like:

Consider an oracle that provides a price feed. This price feed would need an
ingester capable of ingesting incoming values and producing a final result at
the end of a given amount of time. So we may have something that looks like:

```go
type MultiValueIngester struct {
agentAddresses []string
Expand Down Expand Up @@ -93,12 +136,22 @@ An ingester is highly customizable and should be used to do any necessary aggreg

## Storage

Storage types are responsible for storing values produced by a feed's ingester. A storage type must implement `gnorkle.Storage`. This type should be able add values to the storage, retrieve the latest value, and retrieve a set of historical values. It is probably a good idea to make the storage bounded.
Storage types are responsible for storing values produced by a feed's ingester.
A storage type must implement `gnorkle.Storage`. This type should be able to add
values to the storage, retrieve the latest value, and retrieve a set of historical
values. It is probably a good idea to make the storage bounded.

The only storage currently implemented is the `storage/simple.Storage` type.

### Example use case
In most cases the storage implementation will be a key value store, so its use case is fairly generic. The `Put` method is used to store finalized data and the `GetLatest` method can be used by consumers of the data. `Storage` is an interface in order to allow for memory management throughout the lifetime of the oracle writing data to it. The `GetHistory` method can be used, if desired, to keep a fixed historical window of published data points.

In most cases the storage implementation will be a key value store, so its use
case is fairly generic. The `Put` method is used to store finalized data and
the `GetLatest` method can be used by consumers of the data. `Storage` is an
interface in order to allow for memory management throughout the lifetime of
the oracle writing data to it. The `GetHistory` method can be used, if desired,
to keep a fixed historical window of published data points.

```go
type Storage interface {
Put(value string)
Expand All @@ -109,4 +162,15 @@ type Storage interface {

## Whitelists

Whitelists are optional but they can be set on both the oracle instance and the feed levels. The absence of a whitelist definition indicates that ALL addresses should be considered to be whitelisted. Otherwise, the presence of a defined whitelist indicates the callers address MUST be in the whitelist in order for the request to succeed. A feed whitelist has precedence over the oracle instance's whitelist. If a feed has a whitelist and the caller is not on it, the call fails. If a feed doesn't have a whitelist but the instance does and the caller is not on it, the call fails. If neither have a whitelist, the call succeeds. The whitlist logic mostly lives in `gnorkle/whitelist.gno` while the only current `gnorkle.Whitelist` implementation is the `agent.Whitelist` type. The whitelist is not owned by the feeds they are associated with in order to not further pollute the `gnorkle.Feed` interface.
Whitelists are optional but they can be set on both the oracle instance and the
feed levels. The absence of a whitelist definition indicates that ALL addresses
should be considered to be whitelisted. Otherwise, the presence of a defined
whitelist indicates the callers address MUST be in the whitelist in order for
the request to succeed. A feed whitelist has precedence over the oracle instance's
whitelist. If a feed has a whitelist and the caller is not on it, the call fails.
If a feed doesn't have a whitelist but the instance does and the caller is not on
it, the call fails. If neither have a whitelist, the call succeeds. The whitelist
logic mostly lives in `gnorkle/whitelist.gno` while the only current
`gnorkle.Whitelist` implementation is the `agent.Whitelist` type. The
whitelist is not owned by the feeds they are associated with in order to not
further pollute the `gnorkle.Feed` interface.
2 changes: 1 addition & 1 deletion examples/gno.land/p/demo/gnorkle/agent/whitelist.gno
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func (m *Whitelist) RemoveAddress(address string) {
m.store.Remove(address)
}

// HasDefinition returns true if the whitelist has a definition. It retuns false if
// HasDefinition returns true if the whitelist has a definition. It returns false if
// `ClearAddresses` has been called without any subsequent `AddAddresses` calls, or
// if `AddAddresses` has never been called.
func (m Whitelist) HasDefinition() bool {
Expand Down
2 changes: 1 addition & 1 deletion examples/gno.land/p/demo/gnorkle/message/type.gno
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ type FuncType string
const (
// FuncTypeIngest means the agent is sending data for ingestion.
FuncTypeIngest FuncType = "ingest"
// FuncTypeCommit means the agent is requesting a feed commit the transitive data
// FuncTypeCommit means the agent is requesting a feed commit for the transitive data
// being held by its ingester.
FuncTypeCommit FuncType = "commit"
// FuncTypeRequest means the agent is requesting feed definitions for all those
Expand Down
1 change: 1 addition & 0 deletions examples/gno.land/p/demo/ownable/ownable.gno
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ func NewWithAddress(addr std.Address) *Ownable {
}

// TransferOwnership transfers ownership of the Ownable struct to a new address
// It can be called only by the current owner, and it checks if the newOwner address is valid.
func (o *Ownable) TransferOwnership(newOwner std.Address) error {
err := o.CallerIsOwner()
if err != nil {
Expand Down
41 changes: 27 additions & 14 deletions examples/gno.land/r/gnoland/ghverify/contract.gno
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,43 @@ import (
"std"

"gno.land/p/demo/avl"
"gno.land/p/demo/entropy"
"gno.land/p/demo/gnorkle/feeds/static"
"gno.land/p/demo/gnorkle/gnorkle"
"gno.land/p/demo/gnorkle/message"
"gno.land/p/demo/ufmt"
)

// add an example agent > hackerspace PR
// add example messages which the agent can send
// update rendering to show possible verification requests :)

const (
// The agent should send this value if it has verified the github handle.
verifiedResult = "OK"
verifyMsg = `Verification request submitted!
Using the GitHub account you want to verify, please create a
public repo with the following name:`
)

var (
ownerAddress = std.GetOrigCaller()
oracle *gnorkle.Instance
postHandler postGnorkleMessageHandler
ownable *ownable.Ownable
oracle *gnorkle.Instance
postHandler postGnorkleMessageHandler

handleToAddressMap = avl.NewTree()
addressToHandleMap = avl.NewTree()
)

func init() {
oracle = gnorkle.NewInstance()
oracle.AddToWhitelist("", []string{string(ownerAddress)})
ownable = ownable.NewWithAddress("g125em6arxsnj49vx35f0n0z34putv5ty3376fg5") // @leohhhn
oracle.AddToWhitelist("", []string{ownable.Owner().String()})
}

type postGnorkleMessageHandler struct{}

// Handle does post processing after a message is ingested by the oracle feed. It extracts the value to realm
// Handle does post-processing after a message is ingested by the oracle feed. It extracts the value to realm
// storage and removes the feed from the oracle.
func (h postGnorkleMessageHandler) Handle(i *gnorkle.Instance, funcType message.FuncType, feed gnorkle.Feed) error {
if funcType != message.FuncTypeIngest {
Expand Down Expand Up @@ -69,7 +79,7 @@ func (h postGnorkleMessageHandler) Handle(i *gnorkle.Instance, funcType message.

// RequestVerification creates a new static feed with a single task that will
// instruct an agent to verify the github handle / gno address pair.
func RequestVerification(githubHandle string) {
func RequestVerification(githubHandle string) string {
gnoAddress := string(std.GetOrigCaller())
if err := oracle.AddFeeds(
static.NewSingleValueFeed(
Expand All @@ -83,6 +93,9 @@ func RequestVerification(githubHandle string) {
); err != nil {
panic(err)
}

return ufmt.Sprintf(`%s\n%d`, verifyMsg, entropy.New().Value()+uint32(std.GetHeight()))

}

// GnorkleEntrypoint is the entrypoint to the gnorkle oracle handler.
Expand All @@ -95,17 +108,13 @@ func GnorkleEntrypoint(message string) string {
return result
}

// SetOwner transfers ownership of the contract to the given address.
func SetOwner(owner std.Address) {
if ownerAddress != std.GetOrigCaller() {
panic("only the owner can set a new owner")
}

ownerAddress = owner
// TransferOwnership transfers ownership of the contract to the given address.
func TransferOwnership(newOwner std.Address) {
ownable.TransferOwnership(newOwner)

// In the context of this contract, the owner is the only one that can
// add new feeds to the oracle.
oracle.ClearWhitelist("")
oracle.ClearWhitelist("") // why does this take an argument? not obvious
oracle.AddToWhitelist("", []string{string(ownerAddress)})
}

Expand Down Expand Up @@ -144,3 +153,7 @@ func Render(_ string) string {

return result + "}"
}

func genVerifyRepoName(owner, addr string) string {

}
Loading