Skip to content

Commit

Permalink
feat: Keeper (#95)
Browse files Browse the repository at this point in the history
  • Loading branch information
Ben authored Mar 9, 2022
1 parent 4762289 commit d37059a
Show file tree
Hide file tree
Showing 28 changed files with 434 additions and 194 deletions.
40 changes: 29 additions & 11 deletions bot-twitter/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,15 @@
# Auctions twitter bot
# Unified Auctions bot

## What is "Unified Auctions Bot"?

Unified Auctions Bot" is a server-side application with two distinct functionalities:

- `Twitter bot` – a script that periodically fetches collateral auctions and publish a tweet if it finds a new auction
- `Keeper bot` – a script that periodically fetches collateral auctions and their market values, and if it finds that any
of them are profitable, executes respective transactions (authorizations and bidding)

The bots can be hosted by any user, provided, they configure related environment variables. For more details, please
refer to [`Environment variables`](#environment-variables) section below.

## Setup

Expand All @@ -16,26 +27,33 @@ $ npm run start

## Environment variables

- `INFURA_PROJECT_ID`: (required) [infura](https://infura.io/) project id (can be found in: dashboard -> ethereum -> create new project -> settings -> keys). Note: this project can not be restricted by the origin.
- `ETHEREUM_NETWORK`: (optional, default `kovan`) – internal network name on which the bot poll for auctions. Available options can be found in [constants/NETWORKS](../core/src/constants/NETWORKS.ts)
- `INFURA_PROJECT_ID`: (required) [infura](https://infura.io/) project id (can be found in: dashboard -> ethereum ->
create new project -> settings -> keys). Note: this project can not be restricted by the origin.
- `ETHEREUM_NETWORK`: (optional, default `kovan`) – internal network name on which the bot poll for auctions. Available
options can be found in [constants/NETWORKS](../core/src/constants/NETWORKS.ts)
- `REFETCH_INTERVAL`: (optional, default 60 seconds) – interval between auction fetching requests
- `FRONTEND_ORIGIN`: (required) An origin to which the bot will redirect users (valid example: `https://auctions.makerdao.network`)
- `WALLET_PRIVATE_KEY`: (required for server side execution) The secret key of the wallet (https://metamask.zendesk.com/hc/en-us/articles/360015289632-How-to-Export-an-Account-Private-Key)
- `TWITTER_*`: variables (optional for dev environment) – a set of secrets from twitter developer account with `OAuth 1.0a` `Elevated` access and `Read and Write` permissions:
- `KEEPER_*`: (optional) set of env variables to enable keeper bot:
- `KEEPER_WALLET_PRIVATE_KEY`: (required) The wallet private key (https://metamask.zendesk.com/hc/en-us/articles/360015289632-How-to-Export-an-Account-Private-Key)
- `KEEPER_MINIMUM_NET_PROFIT_DAI`: (required) The minimum net profit an auction must yield before the keeper automatically bids on it. Can be negative if one is willing to spend ETH on transaction fees to keep DAI stable
- `TWITTER_*`: (optional) set of env variables to enable twitter bot. Created via twitter developer account
with `OAuth 1.0a` `Elevated` access and `Read and Write` permissions:
- `TWITTER_API_KEY`: (required)
- `TWITTER_API_SECRET`: (required)
- `TWITTER_ACCESS_TOKEN`: (required)
- `TWITTER_ACCESS_SECRET`: (required)
- `FRONTEND_ORIGIN`: (required) The auction UI origin for the tweet (valid
example: `https://auctions.makerdao.network`)

Note: env variables are accessible via the `secret` command under `auction-ui/bot/${environment}`.

## Development Setup

Please see [this centralized guide](https://github.com/sidestream-tech/guides/blob/main/frontend-development/README.md) to get started with your development setup. Namely, you should:
Please see [this centralized guide](https://github.com/sidestream-tech/guides/blob/main/frontend-development/README.md)
to get started with your development setup. Namely, you should:

- have a minimum `node` and `npm` version (see `package.json` `engine` section)
- have a certain formatting and linting setup
- don't forget to create `./bot-twitter/.env` file
- have a minimum `node` and `npm` version (see `package.json` `engine` section)
- have a certain formatting and linting setup
- don't forget to create `./bot-twitter/.env` file

Help on both things is given in the linked resources above.

Expand All @@ -51,8 +69,8 @@ Testing scenario:
Using the `Dockerfile`:

Run the following commands:
```sh

```sh
# 1. Build the docker image
> docker build -t auction-ui-bot -f Dockerfile ..

Expand Down
4 changes: 4 additions & 0 deletions bot-twitter/nuxt.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ export default {
// Target: https://go.nuxtjs.dev/config-target
target: 'server',

server: {
port: 3001,
},

// Modules for dev and build (recommended): https://go.nuxtjs.dev/config-modules
buildModules: [
// https://github.com/nuxt-community/dotenv-module
Expand Down
6 changes: 6 additions & 0 deletions bot-twitter/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 9 additions & 8 deletions bot-twitter/src/auctions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,20 @@ const markAuctionAsKnown = function (auction: AuctionInitialInfo): void {
knownAuctionIds.add(auction.id);
};

export const getNewAuctions = async function (network: string): Promise<AuctionInitialInfo[]> {
const auctions = await fetchAllInitialAuctions(network);
console.info(`auctions: found "${auctions.length}" on "${network}" network`);

const activeActions = auctions.filter(auction => auction.isActive);
console.info(`auctions: "${activeActions.length}" of "${auctions.length}" are active`);

export const getNewAuctionsFromActiveAuctions = function (activeActions: AuctionInitialInfo[]): AuctionInitialInfo[] {
const newAuctions = activeActions.filter(activeAction => {
const isNew = activeAction.startDate > new Date(Date.now() - THRESHOLD_FOR_NEW_AUCTIONS);
return isNew && checkIfAuctionIsAlreadyKnown(activeAction);
});
console.info(`auctions: "${newAuctions.length}" of "${activeActions.length}" are new`);
console.info(`auctions: "${newAuctions.length}" of "${activeActions.length}" auctions are new`);

newAuctions.map(markAuctionAsKnown);
return newAuctions;
};

export const getAllAuctions = async function (network: string): Promise<AuctionInitialInfo[]> {
const auctions = await fetchAllInitialAuctions(network);
const auctionIds = auctions.map(auction => `"${auction.id}"`).join(', ');
console.info(`auctions: found "${auctions.length}" auctions (${auctionIds}) on "${network}" network`);
return auctions;
};
15 changes: 0 additions & 15 deletions bot-twitter/src/authorizations.ts

This file was deleted.

41 changes: 22 additions & 19 deletions bot-twitter/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,37 +1,40 @@
import { setTimeout as delay } from 'timers/promises';
import { getNetworkConfigByType } from 'auctions-core/src/constants/NETWORKS';
import { createSigner, setSigner } from 'auctions-core/src/signer';
import { getNewAuctions } from './auctions';
import { getAllAuctions, getNewAuctionsFromActiveAuctions } from './auctions';
import notify from './notify';
import { setupWallet } from './authorizations';
import participate, { setupKeeper } from './keeper';
import { ETHEREUM_NETWORK } from './variables';
import { setupTwitter } from './twitter';

const ETHEREUM_NETWORK = process.env.ETHEREUM_NETWORK || 'kovan';
const WALLET_PRIVATE_KEY = process.env.WALLET_PRIVATE_KEY;
const DEFAULT_REFETCH_INTERVAL = 60 * 1000;
const SETUP_DELAY = 3 * 1000;
const REFETCH_INTERVAL = parseInt(process.env.REFETCH_INTERVAL ?? '') || DEFAULT_REFETCH_INTERVAL;
let refetchIntervalId: ReturnType<typeof setTimeout> | undefined;

const loop = async function (): Promise<void> {
if (refetchIntervalId) {
clearInterval(refetchIntervalId);
}
try {
(await getNewAuctions(ETHEREUM_NETWORK)).map(notify);
const activeAuctions = await getAllAuctions(ETHEREUM_NETWORK);
if (activeAuctions.length === 0) {
return;
}
const newAuctions = getNewAuctionsFromActiveAuctions(activeAuctions);
newAuctions.map(notify);
activeAuctions.map(participate);
} catch (error) {
console.error('loop error:', error);
} finally {
refetchIntervalId = setTimeout(loop, REFETCH_INTERVAL);
}
};

const setup = async function (): Promise<void> {
const start = async function (): Promise<void> {
await delay(SETUP_DELAY);
getNetworkConfigByType(ETHEREUM_NETWORK);
if (WALLET_PRIVATE_KEY) {
setSigner(ETHEREUM_NETWORK, createSigner(ETHEREUM_NETWORK, WALLET_PRIVATE_KEY));
await setupWallet(ETHEREUM_NETWORK);
}
await loop();
await setupTwitter();
await setupKeeper();
loop();
setInterval(loop, REFETCH_INTERVAL);
};

setup().catch(error => {
start().catch(error => {
throw error;
});

export default function () {} // required by nuxt
151 changes: 151 additions & 0 deletions bot-twitter/src/keeper.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
import { AuctionInitialInfo } from 'auctions-core/src/types';
import getSigner, { createSigner, setSigner } from 'auctions-core/src/signer';
import { bidOnTheAuction, enrichAuction } from 'auctions-core/src/auctions';
import {
authorizeCollateral,
authorizeWallet,
getCollateralAuthorizationStatus,
getWalletAuthorizationStatus,
} from 'auctions-core/src/authorizations';
import { ETHEREUM_NETWORK, KEEPER_MINIMUM_NET_PROFIT_DAI, KEEPER_WALLET_PRIVATE_KEY } from './variables';

let isSetupCompleted = false;
const currentlyExecutedAuctions = new Set();

export const setupKeeper = async function () {
if (!KEEPER_WALLET_PRIVATE_KEY) {
console.warn('keeper: KEEPER_WALLET_PRIVATE_KEY variable is not set, keeper will not run');
return;
}
if (Number.isNaN(KEEPER_MINIMUM_NET_PROFIT_DAI)) {
console.warn('keeper: KEEPER_MINIMUM_NET_PROFIT_DAI is not set or not a number, keeper will not run');
return;
}
try {
setSigner(ETHEREUM_NETWORK, createSigner(ETHEREUM_NETWORK, KEEPER_WALLET_PRIVATE_KEY));
const signer = await getSigner(ETHEREUM_NETWORK);
const address = await signer.getAddress();
isSetupCompleted = true;
console.info(
`keeper: setup complete: using wallet "${address}", looking for minimum clear profit of "${KEEPER_MINIMUM_NET_PROFIT_DAI}" DAI`
);
} catch (error) {
console.warn('keeper: setup error, keeper will not run, please check that KEEPER_WALLET_PRIVATE_KEY is valid');
}
};

const checkAndParticipateIfPossible = async function (auction: AuctionInitialInfo) {
// check if setupKeeper hasn't run
if (!isSetupCompleted) {
return;
}

const signer = await getSigner(ETHEREUM_NETWORK);

// enrich the auction with more numbers
const auctionTransaction = await enrichAuction(ETHEREUM_NETWORK, auction);

// check if auction became inactive or finished
if (auctionTransaction.isFinished) {
console.info(`keeper: auction "${auction.id}" has already finished`);
return;
}
if (!auctionTransaction.isActive) {
console.info(`keeper: auction "${auction.id}" is inactive`);
return;
}

// check auction's profit
if (!auctionTransaction.transactionGrossProfit || auctionTransaction.transactionGrossProfit.isLessThan(0)) {
if (auctionTransaction.transactionGrossProfit) {
const profit = `${auctionTransaction.transactionGrossProfit.toFixed(0)} DAI`;
console.info(`keeper: auction "${auction.id}" is not yet profitable (current profit: ${profit})`);
} else {
console.info(`keeper: auction "${auction.id}" is not tradable`);
}
return;
}

// check auction's clear profit – profit without transaction fees
if (
auctionTransaction.transactionNetProfit &&
auctionTransaction.transactionNetProfit.toNumber() < KEEPER_MINIMUM_NET_PROFIT_DAI
) {
console.info(
`keeper: auction "${
auction.id
}" clear profit is smaller than min profit (${auctionTransaction.transactionNetProfit.toFixed(
0
)} < ${KEEPER_MINIMUM_NET_PROFIT_DAI})`
);
return;
} else {
console.info(
`keeper: auction "${auction.id}" clear profit is ${auctionTransaction.transactionNetProfit.toFixed(
0
)} DAI after transaction fees, moving on to the execution`
);
}

// get wallet authorization status
const walletAddress = await signer.getAddress();
const isWalletAuth = await getWalletAuthorizationStatus(ETHEREUM_NETWORK, walletAddress);

// try to authorize the wallet then return
if (!isWalletAuth) {
console.info(`keeper: wallet "${walletAddress}" has not been authorized yet. Attempting authorization now...`);
const transactionHash = await authorizeWallet(ETHEREUM_NETWORK, false);
console.info(`keeper: wallet "${walletAddress}" successfully authorized via "${transactionHash}" transaction`);
await checkAndParticipateIfPossible(auction);
return;
}

// get collateral authorization status
const isCollateralAuth = await getCollateralAuthorizationStatus(
ETHEREUM_NETWORK,
auctionTransaction.collateralType,
walletAddress
);

// try to authorize the collateral then return
if (!isCollateralAuth) {
console.info(
`keeper: collateral "${auctionTransaction.collateralType}" has not been authorized on wallet "${walletAddress}" yet. Attempting authorization now...`
);
const collateralTransactionHash = await authorizeCollateral(
ETHEREUM_NETWORK,
auctionTransaction.collateralType,
false
);
console.info(
`keeper: collateral "${auctionTransaction.collateralType}" successfully authorized on wallet "${walletAddress}" via "${collateralTransactionHash}" transaction`
);
await checkAndParticipateIfPossible(auction);
return;
}

// bid on the Auction
console.info(`keeper: auction "${auctionTransaction.id}": attempting swap execution`);
const bidHash = await bidOnTheAuction(ETHEREUM_NETWORK, auctionTransaction, walletAddress);
console.info(`keeper: auction "${auctionTransaction.id}" was succesfully executed via "${bidHash}" transaction`);
};

const participate = async function (auction: AuctionInitialInfo) {
// check if this auction is currently executed to avoid double execution
if (currentlyExecutedAuctions.has(auction.id)) {
return;
}
currentlyExecutedAuctions.add(auction.id);

// execute
try {
await checkAndParticipateIfPossible(auction);
} catch (error) {
console.error(`keeper: unexpected error: ${(error instanceof Error && error.message) || 'unknown'}`);
}

// clear pool of currently executed auctions
currentlyExecutedAuctions.delete(auction.id);
};

export default participate;
7 changes: 3 additions & 4 deletions bot-twitter/src/twitter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ import { isDev, enforceEnvVariables } from './utils';

let twitterClient: TwitterApi | undefined;

const setup = function (): void {
export const setupTwitter = function (): void {
if (isDev) {
console.warn('twitter: skipping twitter setup due to development mode');
console.warn('twitter: skipping twitter setup due to the dev mode');
return;
}
try {
Expand All @@ -27,11 +27,10 @@ const setup = function (): void {
accessSecret: process.env.TWITTER_ACCESS_SECRET,
} as TwitterApiTokens);
};
setup();

export const sendNotification = async function (message: string) {
if (isDev) {
console.warn('twitter: tweet is skipped due to development mode', message);
console.warn(`twitter: tweet is skipped due to the dev mode: "${message}"`);
return;
}
if (!twitterClient) {
Expand Down
3 changes: 3 additions & 0 deletions bot-twitter/src/variables.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export const ETHEREUM_NETWORK = process.env.ETHEREUM_NETWORK || 'kovan';
export const KEEPER_MINIMUM_NET_PROFIT_DAI = parseInt(process.env.KEEPER_MINIMUM_NET_PROFIT_DAI || '');
export const KEEPER_WALLET_PRIVATE_KEY = process.env.KEEPER_WALLET_PRIVATE_KEY;
Loading

0 comments on commit d37059a

Please sign in to comment.