Implement vault liquidation service

[valiafetisov]

Goal

PM is able to start a service, understand when its update is complete and fetch list of vaults at risk

Context

After the conclusion of the related investigation issue, where the implementation proposal was outlined, and implementation of the core logic we need to implement the backend service that keeps track of the vaults and determines which of them are at risk. Please use this issue to discuss implementation details.

Assets

• Implementation proposal Investigate functionality to start collateral auctions #363 (comment)
• Core functionality Vault liquidation core functionality #457
• Discussion with the DICU https://github.com/sidestream-tech/auction-ui/issues/433#issuecomment-1235232969
• Related TS types https://github.com/sidestream-tech/unified-auctions-ui/blob/main/core/src/types.ts#L270-L333

Tasks

• Use DICU endpoint to fetch list of active vaults
• Enrich this data with latest vault data
• Keep our own cache in a database
• Provide endpoint to fetch vaults at risk
• Provide endpoint to fetch DB dump
• Provide docker-compose or other ways to start both frontend and the service
• Provide store functionality to fetch and refetch vaults at risk
• Provide service readme with the list of env vars, start commands, etc
• Update main readme to outline new service
• Ensure that this feature doesn't affect the ability to use frontend independently (when no service is running)
• ? Update CI/CD to build and deploy new service (can be done in another PR)

[KirillDogadin-std]

I would suggest (since this was not mentioned before) to use express api in this case over more "advanced"/newer options such as Ts.ED since:

• it won't (likely) require a lot of time to set up and figure out to how to get to it since afaik at least 50% of the team members have seen it already
• the api is not of an advanced kind and the currenly required features are very scoped.
  • There're 2 endpoints, so this is small replacement effort if we will want to scale api with a more modern framework.

? Update CI/CD to build and deploy new service (can be done in another PR)

I would suggest extracting this into another PR

Ensure that this feature doesn't affect the ability to use frontend independently (when no service is running)

the frontend should nevertheless have missing functionality - right? (the vaults table will not be populated)

---

what about the functionality to continiously keep the database up-to-date? Is it going to be a different issue's scope?

[valiafetisov]

I would suggest to use h3 instead of express since it is used under the hood in nuxt3 (which we might migrate this project as well in the future). It is very similar to express, so you should have no problem to get started.

I would also suggest to use zod the same way we use it in https://github.com/sidestream-tech/sidebase-libs/tree/main/packages/nuxt-sidebase-parse

What would you suggest to use to build/package and develop it with? tsup? Something better that uses vite under the hood?

what about the functionality to continiously keep the database up-to-date? Is it going to be a different issue's scope?

It's expected to be covered by this issue.

the frontend should nevertheless have missing functionality

I guess frontend integration can be moved to yet another issue. General idea: if endpoints are down, the frontend should have missing functionality, but not broken state. For example, if started locally (via isDev kind-of check) frontend can signal that the service is not running. In production it should say that endpoints are down and that specific vault ids can be used to liquidate them + point user to some other UI/way to get vaults at risk.

[KirillDogadin-std]

• have nothing against using h3 except for narrow documentation.
• zod approach seems solid
• we can use tsup since it does not require a config and also has proven to be useful in the other company's projects
  • i did not discover a package that is powered vite within ~15 mins

No further questions about the rest of the topics.

Should i propose the service structure in more detail compared with the linked in Assets issue? (#363)

[valiafetisov]

Should i propose the service structure in more detail

If you have any questions / unsure about structure or already have it in the notes and want to document it. For me the general structure doesn't raise any questions, I'm only unsure about enrichment and when/how it is triggered. Compared to the events that we could've subscribe to, now we would now need to refetch it periodically or parse every block (depending on what takes less resources). What's the approach here?

[KirillDogadin-std]

What's the approach here?

Important terminology: in the answer for this q:

• events - general term
• events - technical blockchain term

General scope:

1. Determine the frequency we want to update our db with ( i would say each 12+3 seconds or in a "block-by-block" nature )
2. Define the events we want to react to.
3. Investigate wether or not all of the interesting events can be tracked in form of events (answer is no: jug contract / stability fee collection does not have event emitters
4. For those events that can't be tracked as events, we reside to "block-by-block" parsing: look at the transaction logs, take the topic of the log record and compare with the pre-defined list of the log entries we're looking for.
   https://etherscan.io/tx/0xe71d267d7ef5f3314970b2b8cc25709007881f1341c553ebb82b46ba282d2939#eventlog for example contains the log with topic[0] 0xb65337df which corresponds to the call of vat.fold ( update stability fee rates ). If you open the JUG contract and see its transactions that call drip - the same log topic will appear all the time. The arguments can be extracted from the topics and datafields

Proposal

• So I would split the logic into two steps:
  1. parse events: parseLatestEvents(network, provider, eventConfigs)
  2. parse transactions: parseLatestLogs(network, provider, topics)

Minmalistic types then are:

interface SearchableEvent {
  contractName: string;
  eventName: string;
  loggedValues: string[];
}

interface SearchableFunctionCall {
  topicSignatrue: string[];
  topicIndexToArgumentName: Record<number, string>;
  dataIndexToArgumentName?: Record<number, string>;
}

We want to react to:
Events:

• Bark in dog.sol for contract liquidation
• LogValue in osm oracle for minUnitPrice update
• Value in univ oracle for minUnitPrice update
• NewCdp for new vaults opened.
  Function calls:
• Jug.drip for update of the rates: topic 0xb65337df
• CDPManager.frob for changes in the vault's contents: topic 0x76088703