diff --git a/docs/developing/information/custom-indexers.md b/docs/developing/information/custom-indexers.md deleted file mode 100644 index bc8b707e5..000000000 --- a/docs/developing/information/custom-indexers.md +++ /dev/null @@ -1,274 +0,0 @@ ---- -title: Custom indexers -authors: Tezos Ukraine -last_update: - date: 10 July 2023 ---- - -## Introduction - -[DipDup](https://dipdup.io/) and [Dappetizer](https://dappetizer.dev/) are frameworks for creating custom selective indexers for working with complex contracts and decentralized applications, connecting analytics, getting data from IPFS, and much more. The main difference is that when using DipDup, you will be writing your indexing code in Python, and when using Dappetizer, you will be writing TypeScript or JavaScript. - -Selective indexers are software fetching and storing the requested on-chain data in a local database. They are much more challenging than public indexers because the developer must write the indexing logic by themselves: which data to fetch, how to process it, how to store it, etc. - -But selective indexers have two advantages. First, they use hardly any resources because they only store what you tell them. Second, they allow you to work with databases more flexibly. For instance, write down token amounts with correct decimal points. We shall show how to use it through a real-life example below. - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#what-its-all-about)What it’s all about - -As DipDup and Dappetizer are harder to set up, we will index a simple contract and perform a simple query. We chose tzBTC—wrapped Bitcoin on Tezos as a FA1.2 token. - -We will install each indexer and create an empty project. Then we will set it up to index what we need, for example, tzBTC holders' balances from storage or tzBTC transactions from the “transfer” entrypoint. And in the end, we will process the data and display it on a simple webpage. - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#installing-and-configuring-dipdup)Installing and configuring DipDup - -DipDup only works on Unix systems: macOS and Linux distributions. If you have Windows, use WSL or Linux in dual boot. - -First, you need to install [the Python environment](https://www.python.org/downloads/) in any convenient way. Then open a terminal and install DipDup using the command: - -```bash -python -c "$(curl -sSL https://dipdup.io/install.py)" -``` - -If you have macOS 12 or higher, replace `python` with `python3`. - -[![1](/img/developing/dipdup1.png)](https://indexers.tezos.org.ua/static/733a2c452ea5390046a06f83f0dcc6f0/5ca22/1.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#creating-a-project-in-dipdup)Creating a project in DipDup - -Now let's start configuring DipDup. First, let's create a folder in which we will create a project and go to it in the terminal. Then run the dipdup new command to create a new project. - -For example, let's index the contract of the tzBTC token since it is simple, and the creation of the indexer will not take much time. To do this, when choosing a configuration template, we will take the tzBTC option. - -[![2](/img/developing/dipdup2.png)](https://indexers.tezos.org.ua/static/a4ddf29abc78418f42b0aadbe536d42b/9bae8/2.png) - -Next, you need to fill in information about the project: name, version, owner, select a database, and all that. In the end, DipDup will independently generate the structure and necessary files for the project. - -[![3](/img/developing/dipdup3.png)](https://indexers.tezos.org.ua/static/ce921a279cebd9fecacd4e86b0391c7f/91425/3.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#indexer-configuration)Indexer configuration - -The dipdup.yml file will appear in the project folder. Its structure is as follows: - -* Spec and package are data about DipDup itself. -* Database: settings of the database. By default, it’s SQLite, but you can connect PostgreSQL -* Contracts: the name of the contract, its address, and typename for further use in the code. -* Datasources: data sources for indexing. By default, it’s TzKT, but you can add APIs from other websites for data, crypto exchanges for price quotes, or IPFS for NFTs; -* Indexes: a list of indexes that DipDup will create and use. In our case, we will collect information about tzBTC holders. -* Templates: schemes of operations to be indexed. Inside there are handlers, which are instructions on how tzBTC should process entrypoint calls: which data to fetch and where to store it. - -The most important thing here is templates. In these modules, we specify the entrypoints that need to be indexed: for on\_transfer data, we need to check operations with the transfer entrypoint, and for on\_mint it’s operations with mint. This is important because there will be no direct references to entry points or contracts in the handler code, and this feature can raise questions like “Where does the data come from?”. - -In this example, we will be checking tzBTC holders' balances. They change when the transfer and mint entry points are called, so we specify two handlers by entry point names — on\_mint and on\_transfer. - -[![4](/img/developing/dipdup4.png)](https://indexers.tezos.org.ua/static/94fa88718212cf7f75282cce5fe2e141/4c530/4.png) - -Check [DipDup Config Documentation](https://docs.dipdup.io/getting-started/creating-config) to learn more about all available options. - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#models-and-data-handlers)Models and data handlers - -Next, you need to create indexes, schemas, models, and handlers. To do this, go to the tzbtc folder in the terminal and execute the command: - -```bash -dipdup init -``` - -[![5](/img/developing/dipdup5.png)](https://indexers.tezos.org.ua/static/7c0f6b39910885a54a171086ff53ead6/5ca22/5.png) - -DipDup will create several files — models and handlers — that describe working with the database and indexing on-chain data, and we need to modify them to set up the indexing process. - -The first file is models.py. It stores data models, roughly speaking, descriptions of tables and columns in the database. In the Holder model, we will declare several types of columns: - -* an address with a length of 36 characters -* balance and turnover in a numeric format while specifying eight decimal places (decimal\_places) -* the number of transactions -* last activity time - -[![6](/img/developing/dipdup6.png)](https://indexers.tezos.org.ua/static/96c7b5f909c56975fdb566de6566e5b0/5ca22/6.png) - -The handlers are instructions on how the DipDup will process on-chain data when a tzBTC contract entry point call occurs. The handlers folder stores handlers for storage and calls to the tzBTC contract entry points. - -The first helper is on\_balance\_update. It will be called after each operation that will change the balances of the tzBTC holders, minting and transferring tokens in our case, and will update corresponding balances in our database. - -In it, we declared the on\_balance\_update function, to which we will pass three parameters: - -* address: the address of the holder. -* balance\_update: the amount of the holder's balance change. -* timestamp: operation time. - -The function will first check if there is such a holder address in the database and then either create a new row and write the new holder or update the old record. - -[![7](/img/developing/dipdup7.png)](https://indexers.tezos.org.ua/static/5940c23853d35d2d52025ba3228538bb/5ca22/7.png) - -The second handler, on\_mint, will be called when the mint entry point of the tzBTC contract is called. It will read the mint entry point call parameters and then call the on\_balance\_update function to update the records in the database. - -[![8](/img/developing/dipdup8.png)](https://indexers.tezos.org.ua/static/1112282e7bde378195b94423911160ee/cce16/8.png) - -IMPORTANT: in the “amount” variable, we save the "mint.parameter.value" value, which we take from the mint entry point’s call parameters. - -[![9](/img/developing/dipdup9.png)](https://indexers.tezos.org.ua/static/7f6845fabfc7073a10e28eaad8343dfa/8643c/9.png) - -Similarly, we take the address of the recipient of fresh tokens: "address = mint.parameter.to"; - -The third handler – on\_transfer – deals with transactions, that is, operations with the transfer entry point. It is more complex. First, you need to compare the sender and recipient addresses. If they match, the holders' balances have not changed, and the database should not be updated. Secondly, the transfer of tokens means a change in the balances of the sender and recipient. Therefore, the on\_balance\_update function must be called twice for each address while subtracting the transaction amount from the sender's balance and adding it to the recipient's balance. - -[![10](/img/developing/dipdup10.png)](https://indexers.tezos.org.ua/static/6229151572035395d3becb0cd770c489/5ca22/10.png) - -As in the on_mint handler, we get on-chain data from the call parameters of the transfer entry point: sender address from_, recipient address to, and transfer amount value. - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#running-the-indexer)Running the indexer - -In the terminal, run the following command: - -```bash -dipdup run -``` - -DipDup will start downloading the required data from the TzKT public API. Upon completion, it will begin processing new blocks. - -[![11](/img/developing/dipdup11.png)](https://indexers.tezos.org.ua/static/bd30fc184f1b8dc3183fee5f7b52c3b3/5ca22/11.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#connecting-a-database-to-a-project)Connecting a database to a project - -Let's make a project similar to the one in the previous lesson. Back then, we displayed NFT collateralized loan offers and generated links to the NFTs. - -Let's create a tzbtc\_dipdup.php file in the tzbtc\_dipdup folder and then connect it to the database first. By default, DipDup suggests you use [SQLite](https://www.sqlite.org/index.html). Connecting to it is slightly different from Postgres: you need to specify the full path to the database file, but you do not need to set the username, password, and port. - -In the $db\_dir variable, we specify the path to the database on the hard disk, and in $db\_handle - the new PDO($db\_dir) database connection method. - -[![12](/img/developing/dipdup12.png)](https://indexers.tezos.org.ua/static/2a16c6dd88a5963c6d228c28915506bc/a7a60/12.png) - -Then we will write a SQL query in which we select all addresses and balances of tzBTC holders with a non-zero balance and sorting in ascending order. To know exactly what tables you have in the database and how to reach them in SQL query, you can install [SQLite Browser](https://sqlitebrowser.org/) with GUI and check what's inside. - -[![13](/img/developing/dipdup13.png)](https://indexers.tezos.org.ua/static/8128e903d77665f4df43171d078c3953/5ca22/13.png) - -We use the iteration function forEach to sequentially get user addresses and balances from $row and then show them on the page. - -[![14](/img/developing/dipdup14.png)](https://indexers.tezos.org.ua/static/5a2d6682a268ce9bb9c9172a0a7c6725/d4913/14.png) - -Then we close the connection to the database with the $db\_handle = null command. - -In the terminal, go to the folder with tzbtc\_dipdup.php and start the PHP server with the command: - -```bash -php -S localhost:8000 -``` - -[![15](/img/developing/dipdup15.png)](https://indexers.tezos.org.ua/static/b6a8333f49c6dc19a1bd91e22bb1457f/abe8a/15.png) - -In the browser, open the page at [http://localhost:8000/tzbtc\_dipdup.php](http://localhost:8000/tzbtc_dipdup.php) - -[![16](/img/developing/dipdup16.png)](https://indexers.tezos.org.ua/static/4f88b13e8135e3a6897fd3c5bcdd4104/3f970/16.png) - -Since we already divided the balance by 10^8 when writing transaction handlers and minting tzBTC, they are displayed in the database with the correct number of decimal places. - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#installing-and-configuring-dappetizer)Installing and configuring Dappetizer - -With Dappetizer, we will index the same tzBTC contract, but this time we will display the most recent tzBTC transfers. To do this, we will index its "transfer" entry point and write to the database data about each transaction's sender, receiver, and transfer amount. - -First, install [Node.js](https://nodejs.org/) to work with the npm package manager. Then we will create a folder for the project and install Dappetizer there. The easiest way to do everything is from the terminal: - -```bash -mkdir dappetizer -cd dappetizer -npm install @tezos-dappetizer/cli -``` - -Check if the installation was successful with the version check command: - -```bash -npx dappetizer --version -``` - -[![17](/img/developing/dipdup17.png)](https://indexers.tezos.org.ua/static/228af948a104a02c0587fb9cfb252081/1855e/17.png) - -Now initialize a new project with the tzBTC name and contract address: - -```bash -npx dappetizer init --contractName=tzBTC KT1PWx2mnDueood7fEmfbBDKx1D9BAnnXitn -``` - -[![18](/img/developing/dipdup18.png)](https://indexers.tezos.org.ua/static/513ca79035e04adb259631c74649b680/5ca22/18.png) - -The Dappetizer will create the necessary files in the project folder. Among them are four main ones you will need to work with: - -* dappetizer.config.ts: parameters to connect to a database. -* index.ts: main indexing module, we will import our indexer config there. -* tz-btc-indexer.ts: indexing logic of tzBTC contract. Dappetizer generates a list with templates for all entry points of the specified contract, and we need to delete unused ones and write our handler logic in the indexTransfer method. -* entities.ts: it doesn't exist yet, as we need to create it ourselves and write in it names and types of tables and columns that will store our data. - -dappetizer.config.ts contains parameters for connecting to the database. Dappetizer uses SQLite by default, but PostgreSQL is fine, too. - -[![19](/img/developing/dipdup19.png)](https://indexers.tezos.org.ua/static/19952b3ff1adf15374abec328edd6b78/0f849/19.png) - -You need to create entities.ts and specify the class (table) and database columns. We will record transactions with the tzBTC token, creating a transaction class with columns id (internal id in the database), sender, receiver, and amount. - -[![20](/img/developing/dipdup20.png)](https://indexers.tezos.org.ua/static/879869b9b7c64aea21f136dc236c71cb/5dd2a/20.png) - -The src folder contains tz-btc-indexer.ts, in which Dappetizer will generate functions for indexing all entry points of the specified contract. In the code of the corresponding entrypoint, you need to describe the indexing logic: which entry point call parameter to write to the database. - -[![21](/img/developing/dipdup21.png)](https://indexers.tezos.org.ua/static/4458d88f72d3cc315e464ea698b84da4/5ca22/21.png) - -In tzBTC, the entry point for sending tokens is called transfer. Accordingly, we will find the code for this entry point and add logic to it: - -* at the beginning, we import the Transaction class to use it to write data, -* in the indexTransfer function, we describe the indexing logic: create a tzBTCtransfer constant, in which we write the sender and recipient addresses, as well as the transaction volume, -* at the end of the indexTransfer function, we will call the insert function to write the contents of tzBTCtransfer to the database. - -[![22](/img/developing/dipdup22.png)](https://indexers.tezos.org.ua/static/92bd0f1229f79e6bbf945c17cf6d14d7/5ca22/22.png) - -Now we need to add the Transaction model to the index.ts module. To do this, we import Transaction from the entities.ts file and add Transaction to the dbEntities array. - -[![23](/img/developing/dipdup23.png)](https://indexers.tezos.org.ua/static/d63ac8bfb337449b014639d25817d083/5ca22/23.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#starting-the-indexer)Starting the indexer - -In the terminal, go to the project folder and execute the command to build the project: - -```bash -npm run build -``` - -Then start indexing: - -```bash -npx dappetizer start -``` - -Dappetizer will begin indexing the blocks and writing data to the database. - -[![24](/img/developing/dipdup24.png)](https://indexers.tezos.org.ua/static/1183e87c13a4569fc11aeca211dba725/5ca22/24.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#connecting-the-database-to-the-project)Connecting the database to the project - -Let's make a page similar to the DipDup project. Let's create a tzbtc\_dappetizer.php file and describe the page structure in it. Since we are using the same SQLite database, we can directly copy the code from the DipDup project. You only need to change the path to the database, the SQL query, and the function for displaying data. - -In our example, we use SQL query to select sender, receiver, and amount fields from the "transaction" table and then sort them by internal id, so the most recent ones (with a higher id value) will be displayed first. - -[![25](/img/developing/dipdup25.png)](https://indexers.tezos.org.ua/static/6e12e4741cc38df7849681462501071f/46eb0/25.png) - -Let's start the PHP server in the Dappetizer folder and open the page in the browser: - -[![26](/img/developing/dipdup26.png)](https://indexers.tezos.org.ua/static/dd2993d74350f2f3cfc65f1fb276145d/5ca22/26.png) - -### [](https://indexers.tezos.org.ua/dipdup-dappetizer#homework)Homework - -Calculate the total volume of tzBTC transactions for the indexed period. Hint: it's better to work with ready-made data rather than with what you add. - -#### [](https://indexers.tezos.org.ua/dipdup-dappetizer#answers)Answers - -Working with indexers is closely related to databases and SQL queries. To develop an application, you must learn SQL! - -In the DipDup example, we considered the volume (turnover) by adding the volume of the indexed transaction to the previous turnover value. Still, using the SUM() SQL command is easier to avoid bloating the database. - -Open the tzbtc\_dappetizer file and create a $sql\_sum SQL query with SELECT SUM(amount) FROM “transaction”: - -[![27](/img/developing/dipdup27.png)](https://indexers.tezos.org.ua/static/ff992168c8b74794cd92405809640c37/c7591/27.png) - -Then add a query($sql\_sum) function to query the database and output the value to the page. - -[![28](/img/developing/dipdup28.png)](https://indexers.tezos.org.ua/static/e6fcc58f872ad695eb84cadb7d5509e5/5ca22/28.png) - - -:::note -This content was created by [Tezos Ukraine](https://tezos.org.ua/en) under MIT Licence. The original version can be found [here](https://indexers.tezos.org.ua/) in multiple languages. -::: diff --git a/docs/developing/information/indexers.md b/docs/developing/information/indexers.md index 832e4f558..ae2195217 100644 --- a/docs/developing/information/indexers.md +++ b/docs/developing/information/indexers.md @@ -1,429 +1,186 @@ --- title: Indexers -authors: Tezos Ukraine +authors: Tezos Ukraine, Tim McMackin last_update: - date: 10 July 2023 + date: 2 January 2024 --- -:::note -If you're just looking for API docs for a hosted indexer, [here is the one from TzKT](https://api.tzkt.io/), and [here is the one from TzPro](https://docs.tzpro.io/) -::: +Indexers are off-chain applications that retrieve blockchain data, process it, and store it in a way that makes it easier to search and use. +Indexers are an important component of [Block explorers](./block-explorers). -## What is an indexer ? -A block explorer, or an app that tracks activity on the blockchain, is typically made of: +You can use indexers to provide the data that you need for your dApps. -- an indexer that is a node operator that extracts the on-chain data and stores it in a database -- an API that queries the database -- a frontend that displays the data +## Why indexers are needed -Since the on-chain data is already in the database of the node itself, why is an indexer needed? The indexer is relevant because it optimizes the way a node stores data. The purpose is to provide very quick access to blockchain data according to some specific criteria related to a given address. +Tezos nodes store copies of blocks, but they provide only certain information about those blocks through the [RPC protocol](../../architecture/rpc). -So, the indexer is the part of the explorer that fetches the raw data from the node, then processes it and stores it in the database in an efficient way. They simplify querying and searching through blockchain data. Almost any decentralized application uses indexers, and if you want to build your dApp, chances are you’ll need to use them too. +For example, assume that you want information about an operation and all you have is its hash. +The RPC protocol can't provide this information directly, so you would have to search each block until you found the block with this operation, which is very inefficient. +Instead, you can use an indexer that records information about each operation. +This kind of indexer can provide information about a specific operation by its hash or a list of operations sent by a certain account. -## TzKT +## How indexers work -For instance, [TzKT](https://github.com/baking-bad/tzkt) is a lightweight Tezos blockchain indexer with an advanced API created by [Baking Bad](https://baking-bad.org/docs).The indexer fetches raw data from the Tezos node, then processes it and stores it in the database in such a way as to provide effective access to the blockchain data. For example, getting operations by hash, getting all operations of the particular account, or getting detailed baking rewards, etc. None of this can be accessed via node RPC, but TzKT indexer makes this data (and much more) available. +You can imagine indexers as the card catalog in a library. +Libraries typically sort books by genre and author name, so readers can find a book if they know its genre or author. -You can [install](https://github.com/baking-bad/tzkt#installation-docker) or [build](https://github.com/baking-bad/tzkt#installation-from-source) it, and [configure](https://github.com/baking-bad/tzkt#install-tzkt-indexer-and-api-for-testnets) it for the testnet. +However, suppose a reader wants to find a poem about acacia trees, a story about a gas station architect, or every book written in 1962. +Without any other information, they must check all the books in the library until they find what they are looking for. -To work with the **TzKT API** you can use this [public endpoint](https://api.tzkt.io/#section/Introduction). +To simplify the process of searching for books, libraries add each new book they receive to their card catalogs. +Card catalogs list all of the books on a certain topic and provide other metadata such as the authors' names and publication dates. +Readers can look up a topic in the catalog and get a list of all books on that topic. +Libraries can index books in as many ways as are necessary to help people search for books more efficiently. -### Traditional Blockchain Explorer Backends +Similarly, relational databases can have indexes to speed up queries. +In the same way, blockchain indexers create a database with the blockchain data organized in certain ways. -Indexers are node operators. The **ETL** extract, transform and load data into the **SQL database** by mapping the data into a pre-defined schema of tables with referential integrity in order to provide indexing and query processing services via the **API**. +## Types of indexers -* A **Tezos Node** is the heart of the blockchain, it manages the protocol. -* **ETL** stands for _extract, transform, and load_. The process of ETL plays a key role in data integration strategies. ETL allows businesses to gather data from multiple sources and consolidate it into a single, centralized location. -* **API** is the acronym for _Application Programming Interface_, which is a software intermediary that allows two applications to talk to each other. +There are two main types of blockchain indexers: full and selective. -### Focus on BlockWatch Indexer (TzIndex) +### Full indexers -The Blockwatch Indexer [TzIndex](https://github.com/blockwatch-cc/tzindex) is used for the [TzStats explorer](https://tzstats.com/). +Full indexers process and write all data from blocks, from simple transactions to validator's node software versions. +Blockchain explorers commonly use them to provide users with advanced blockchain analytics and allow them to search for any type of on-chain data. +Also, those who host full indexers can offer public APIs that other projects can use without hosting the indexer themselves. -The **Blockwatch indexer** replaces the slow and expensive SQL datastore with a high-performance columnar database that allows for extremely fast analytical queries. +You can get data from these full indexers, which allow you to find almost any information in the Tezos blockchain: - **Columnar database** is a column-oriented storage for databases. It is optimized for fast retrieval of data columns, for example for analytical applications. It significantly reduces the overall disk I/O requirements and limits the amount of data you need to load from the disk. +- [TzKT](https://api.tzkt.io/) +- [TzPro](https://docs.tzpro.io/) +- [TzIndex](https://github.com/blockwatch-cc/tzindex) -It's a custom-made database for blockchain analytics. Avoiding the storage bottleneck allows for more complex data processing. +### Selective indexers -> **Storage bottleneck** is a situation where the flow of data gets impaired or stopped completely due to bad performance or lack of resources. +Selective indexers store only selected data, which means that they need less space and resources to maintain. +Creating a selective indexer requires you to write the logic for the indexes, including what data to fetch, how to process it, and how to store it. -State updates happen at each block, which means all the balance updates are always verified, and the indexer will follow chain reorganizations in real-time. +Usually, they are used in projects that require only specific on-chain data, such as active user balances, balances of their smart contracts, and NFT metadata. +You can optimize a custom selective indexer for fast execution of specific project queries. +You can also use them to work with data in the way that you want to, such as storing token amounts with decimal places. +You can use these frameworks to create your own selective indexer to provide the data that you need in the format that you need: -## A simple explanation of how indexers work +- [Que Pasa](https://github.com/tzConnectBerlin/que-pasa) +- [DipDup](https://dipdup.io/) +- [Dappetizer](https://dappetizer.dev/) -Understanding indexers is easy if we use a library as an example. There are thousands of books on the shelves in the library. They are usually sorted by genre and authors' names. The reader will quickly find the stand with novels or gardening books. +For example, [Teia.art](https://teia.art/) and other NFT marketplaces use their indexers based on DipDup, optimized for working with NFTs. -But let's say the reader wants to find something specific: a poem about an acacia tree, a story about a gas station architect, or all the books written in 1962. They will have to check all the books in the library until they come across what they are looking for. +## Setting up indexers -The librarian got themselves ready for such visitors and compiled a special database. He wrote down the details of each book in separate indexes. For example, one index would list all the names of the characters of every book in alphabetical order and the name of the book where they appear. The second index will contain the key events, and the third—unique terms like “horcruxes” or “shai-hulud,” etc. The librarian can make as many indexes as he needs to simplify the search for a specific book. +For information on setting up indexers, see the documentation for the indexer or [Indexers](https://opentezos.com/dapp/indexers/introduction/) on OpenTezos. -Now let’s return to the blockchain. To find a transaction by its hash or by the address of the caller, you will have to check every transaction in every block. The best way out is to create a separate database with all the contents of the blockchain with pointers to search for information on specific queries and access it quickly. This is what indexers do. +## Using indexers +The exact list of tables, index schemas, and command syntax depend on the indexer and database it uses. +For information about how to use a specific indexer, see its documentation. -## Why blockchain indexers are needed +Some indexers provide information about networks other than Mainnet, so check the indexer's documentation for information about selecting a network. -Blockchain protocols work thanks to nodes that process transactions, ensure network security, and, most importantly, store copies of blocks in chronological order. Validator nodes are conditionally divided into light ones with a copy of the last blocks and archival ones with an entire blockchain. Users can get on-chain data from their nodes or third-party nodes using the Remote Procedure Call API to request data about the blockchain. - -However, nodes and RPC API aren't designed to work with complex searches and queries. -For example, a developer can access an operation's details only if they know where to look for them. - -Let's try to get data about this transaction using Tezos client and RPC API — [opRjkzJxJ1xZaUnBDykGUjrRV8qgHFvchcYnbkkcotS1Y7idCSL](https://tzstats.com/opRjkzJxJ1xZaUnBDykGUjrRV8qgHFvchcYnbkkcotS1Y7idCSL). +For example, this TzKT query gets an account's balance of the USDT token: ``` -user:~/tezos % tezos-client rpc get /chains/main/blocks/2283690/operations/3/2 - -Disclaimer: -The Tezos network is a new blockchain technology. -Users are solely responsible for any risks associated -with usage of the Tezos network. Users should do their -own research to determine if Tezos is the appropriate -platform for their needs and should apply judgement and -care in their network interactions. - -{ "protocol": "Psithaca2MLRFYargivpo7YvUr7wUDqyxrdhC5CQq78mRvimz6A", -"chain_id": "NetXdQprcVkpawu", -"hash": "opRjkzJxJ1xZaUnBDykGUjrRV8qgHFvchcYnbkkcotS1Y7idCSL", -"branch": "BLPELRQsDcAbeXzVcKnqtT7XG7qmtcSuleddD6235sCyN4x21q6", -"contents": -[ { "kind": "transaction", -"source": "tzlUEQzJbuaGJgwvkekké6HwGwaKvjZ7rr9v4", "fee": "543", -"counter": "18321546", "gas_limit": "1521", "storage_limit": "@", -"amount": "1500000", -"destination": "tz1dFq5gcAMi6éTAUJarNJeP6DffyjrPXzhid" } ], -"signature": -"sigPWea9tsDXrornNziEMWPeEpFfTkf4YdPz7GmgPDPVXQyKwsrpze7wLIXDgZCglL2aaciMa8J1QHAUr2b9zhrK26kpvv8Jz" } - -user:~/tezos % +https://api.tzkt.io/v1/tokens/balances?token.contract=KT1XnTn74bUtxHfDtBmm2bGZAQfhPbvKWR8o&account=tz1a1RTsGUbads3VucUQDxJF4EDXkDWcDHPK ``` +For information about the TZKT indexer's API, see https://api.tzkt.io. -The highlighted part at the top of the screenshot is the command we used: - -`tezos-client rpc get /chains/main/blocks/2283698/operations/3/2` - -The numbers in the command mean the following: - -- 2283698 — the level of the block in which the operation took place. -- 3 — the index of the operation's source type. "3" means the user initiated it. -- 2 — the index of the operation in a particular block. - -To get the details of a specific operation, we need to know its location in the blockchain: when it was included, what operation type table it was written in, and its number in that index. To know it, we need to check all blocks in turn for the presence of the specified hash, which is very slow and ineffective. - -Indexers simplify the task of getting the data you need from the blockchain. They request whole blocks from public nodes and write them to their databases. Then indexers create indexes—-additional data structures optimized for fast data retrieval that store either the data or a link to the corresponding place in the central database. When searching for any data, the indexer looks for it in the corresponding index, not in the central database. For even faster searching, some indexers don't store all data in one table but create separate tables for any valuable data. - -![](/img/usdt_que_pasa.png) + -Here is a USDT contract indexed with Que Pasa. It made tables for every entry point and big_map in the contract and an index for every table. It is an efficient way to fetch data. For example, the execution of a query to find a balance of a random USDT holder took only 0.064 milliseconds. + + -![](/img/que_pasa_sql.png) +Indexers can organize data in custom ways. +For example, TzKT indexes FA1.2 and FA2 tokens and gives each an internal ID. +This ID can be faster and easier to work with and compare than using the contract ID and token ID. -The exact list of tables, indexes schemas, and command syntax depends on the indexer and database it uses. While using TzKT, a query for a USDT balance will look like this: +This TzKT query gets information about an FA1.2 token based on its internal ID instead of its contract address and token ID: ``` -https://api.tzkt.io/v1/tokens/balances?token.contract=KT1XnTn74bUtxHfDtBmm2bGZAQfhPbvKWR8o&account=tz1a1RTsGUbads3VucUQDxJF4EDXkDWcDHPK +https://api.tzkt.io/v1/tokens?id=42290944933889 ``` -Let's look at the different parts of this url: -- `https://api.tzkt.io/v1/`: link to TzKT API -- `tokens/balances`: the path to the table with token balances -- `?token.contract=KT1XnTn74bUtxHfDtBmm2bGZAQfhPbvKWR8o`: first search filter, where we specify the contract address we need -- `&account=tz1a1RTsGUbads3VucUQDxJF4EDXkDWcDHPK`: second search filter, where we specify holder address - -In addition to search speed, indexing has another advantage: the ability to modify indexing rules. For example, TzKT provides an additional index, where each Tezos FA1.2 and FA2 token has its internal id. So instead of comparing relatively long contract addresses, it will compare small numbers and retrieve data even faster. - -``` -staging.api.tzkt.io/v1/tokens?id=85 -[{"id":85,"contract": -{"alias":"kUSD", "address":"KT1K9gCRgaLRFKTErYt1wVxA3Frb9FjasjTV"},"to -kenId":"0","standard":"fal.2","firstLevel":1330112,"firstTime":"2021- -02-04T05:43:23Z","lastLevel":2667753,"lastTime":"2022-08- -30T20:13:29Z", "transfersCount":323378,"balancesCount":9030, "holdersCo unt":3854,"totalMinted":"31442022884393231737144909","totalBurned":"2 9832264735683726828828184","totalSupply":"1609758148709504908316725", "metadata":{"name":"Kolibri USD","symbol": "kUSD","decimals":"18"}}] +The response provides information about the token: + +```json +[ + { + "id": 42290944933889, + "contract": { + "alias": "kUSD", + "address": "KT1K9gCRgaLRFKTErYt1wVxA3Frb9FjasjTV" + }, + "tokenId": "0", + "standard": "fa1.2", + "firstMinter": { + "address": "tz1eDj5UuChVcZpA7gofUtyVS6mdQAcyEbZ5" + }, + "firstLevel": 1330112, + "firstTime": "2021-02-04T05:43:23Z", + "lastLevel": 4860455, + "lastTime": "2024-01-03T12:23:49Z", + "transfersCount": 556579, + "balancesCount": 11270, + "holdersCount": 5313, + "totalMinted": "34975975281693622711131319", + "totalBurned": "33819058060274650139662474", + "totalSupply": "1156917221418972571468845", + "metadata": { + "name": "Kolibri USD", + "symbol": "kUSD", + "decimals": "18" + } + } +] ``` -There are two types of blockchain indexers: full and selective. - -Full indexers process and write all data from blocks, from simple transactions to validator's node software versions. Blockchain explorers commonly use them to provide users with advanced blockchain analytics and allow them to search for any type of on-chain data. Also, those who host full indexers often offer public APIs that other projects can use without hosting the indexer themselves. - -The best examples of full indexers with APIs are [TzKT](https://tzkt.io/) and [TzPro](https://tzpro.io/). They allow you to find almost any information that once got into the Tezos blockchain. - -Selective indexers store only selected data. Usually, they find their use in projects requiring only specific on-chain data: active user balances, balances of their smart contracts, and NFT metadata. A custom selective indexer can be optimized for fast execution of specific project queries and also needs less space and resources to maintain. - -Popular selective indexers like [Que Pasa](https://github.com/tzConnectBerlin/que-pasa) and frameworks like [DipDup](https://dipdup.io/) and [Dappetizer](https://dappetizer.dev/) can be used to build the indexer you need. For example, [Teia.art](https://teia.art/) and other NFT marketplaces use their indexers based on DipDup, optimized for working with NFTs. +## Data available on indexers -## What data can be obtained through a blockchain indexer +The exact list of queries and filters depends on the selected indexer. +Here are some examples of information you can get from full indexers: -The exact list of queries and filters depends on the selected indexer. But in general, from a full indexer, you can get information about: +- Account: Balance in tez, transaction history, address type, and status like "baker" or "delegator" +- Block: Header, content, and metadata like address and socials of the baker who made the block +- Contract: Description, entrypoints, balance, code in Michelson, storage, and the content of a specific big map +- Bakers and delegators: Who earned how much, who endorsed a particular block, and how much users delegate +- Protocol: What cycle is now, what is on the vote, how many tez are in circulation -- Account: tez balance, transaction history, address type, and status like "baker" or “delegator,” etc. -- Block: header, content, and metadata like address and socials of the baker who made the block. -- Contract: description, entry points, balance, code in Michelson, storage, and the content of a specific big map. -- Bakers and delegators: who earned how much, who endorsed a particular block, how much users delegate. -- Protocol: what cycle is now, what is on the vote, how many tez are in circulation. +For example, here are some examples for getting information from TzKT. +Follow the links and paste your address instead of `tz1…9v4` into the address bar to get data about your account: -We took some interesting and simple examples for calling BetterCallDev and TzKT indexers. Follow the links and paste your address instead of tz1…9v4 into the address bar to get data about your wallet: - -- [fxhash NFTs you own](https://api.tzkt.io/v1/tokens/balances?account=tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4&token.contract=KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton). The fxhash contract address filters the query, so try to change the address to see your NFTs from other marketplaces. -- [Your balance history in tez](https://api.tzkt.io/v1/accounts/tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4/balance_history). Nodes and indexers display balances without decimals, and ""balance": 500000" means only five tez. +- [fxhash NFTs you own](https://api.tzkt.io/v1/tokens/balances?account=tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4&token.contract=KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton). +The fxhash contract address filters the query, so try to change the address to see your NFTs from other marketplaces. +- [Your balance history in tez](https://api.tzkt.io/v1/accounts/tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4/balance_history). +Nodes and indexers display balances without decimals, and ""balance": 500000" means only five tez. - [List of FA1.2 and FA2 token transfers](https://api.tzkt.io/v1/tokens/transfers?from=tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4) where you were the sender. Change "from" to "to" in the query to see the list of transfers where you were a recipient. -## Where Indexers Are Used - -Many applications that work with on-chain data use an indexer. - -The simplest example of working with an indexer is a blockchain wallet. For example, to display a user's token balances, Temple Wallet queries this data from the TzKT indexer and gets tokens' tickers and logos from the contract metadata. - -![temple query](/img/temple_query.png) - -Try it yourself: [Go here](https://api.tzkt.io/v1/tokens/balances?account=tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4) and replace tz1...9v4 with your wallet address to see which tokens you have. This is the same query to TzKT API that Temple uses: '/tokens/balances' in constants getTokenBalances and getNFTBalances. - -In the same way, Temple Wallet receives transaction details and displays NFTs, delegation rewards, the value of your tokens, and other data. For different queries, it uses various sources. For example, it requests the XTZ price from Coingecko. - -Other blockchain applications work similarly: +TzKT provides information about Ghostnet. +For example, you can run any of the previous queries on Ghostnet by changing the host name in the URL to `https://api.ghostnet.tzkt.io/`. -- Decentralized exchanges' websites use indexers to obtain historical data on the balances of each pool, transactions with them, and the value of tez. Based on this data, they calculate the volume of transactions in tez and dollars, as well as the historical prices of tokens in dollars. -- NFT marketplaces' websites index transactions with their contracts to display new NFTs, transaction history, and statistics of the most popular tokens. -- Blockchain explorers display data from all blocks with the help of indexers and allow users to find anything, like operation details by its hash. Thanks to this, users can find the information they need in a user-friendly graphical interface. Browsers also run public APIs that other projects use. +## Where indexers are used +Many applications that work with on-chain data use an indexer. -## Self study - -Let's say you plan to launch a DeFi dashboard that will display the user's token balances: tez, stablecoins, fungible tokens, and NFTs. Think about how you would implement this based on what you have learned about blockchain indexers. - -### Solution +The simplest example of working with an indexer is a blockchain wallet. +For example, to display a user's token balances, Temple Wallet queries data from the TzKT indexer and gets tokens' tickers and logos from the contract metadata. -The simple way to do this is to make a list of all token smart contracts addresses and then run an indexer to store the contents of their storage in a database. Then you could run queries to get all balance records about the selected address. TzKT does it this way and even has a specific API endpoint: +You can do the same thing by calling the `GET tokens/balances` endpoint TzKT API and including your address as the value of the `account` parameter, as in this example: ``` https://api.tzkt.io/v1/tokens/balances?account=tz1UEQzJbuaGJgwvkekk6HwGwaKvjZ7rr9v4 ``` +In the same way, Temple Wallet receives transaction details and displays NFTs, delegation rewards, the value of your tokens, and other data. +It uses other sources for other data. +For example, it requests the XTZ price from Coingecko. -Using an indexer is not necessary, but the alternative — to use public RPC nodes and fetch data directly from the blockchain — is much more complicated. Token balances can be obtained by reading *big_map* contracts that store balance records. First, you would need to get the hash of the user's address in script-expression format. To do this, run the command - -`octez-client hash data '"{address}"' of type address` - -``` -user:~ % tezos-client hash data '"KT1Gxqznvqznkd3fQRRf95ftYByLpE8uqfTg"' of type address -Raw packed data: 0x050a00000016015beddb4e4cc337681090176a860473a7eba9c4a000 Script-expression-ID-Hash: exprvD3jLXJyLguSwR7D95ZZORHvahH2jdbBbv83m5yrZzyUFR2pVY -Raw Script-expression-ID-Hash: 0xdee7e7a90d3a74fe619d8c201181230a911d4e0427648cd175b7bab29e64a4d2 Ledger Blake2b hash: G18cCFYnGr9yviP24UTXUXasqYiqx5uC4fbx1nRHkFuT -Raw Sha256 hash: 0x08ebb0f9abe29615e6a6ca128e15b4811a70de097b595de72192dfe72755985b -Raw Sha512 hash: 0x1b4a9ecdc318d7f1fa0f6dd67f1ff11dfcfda7530667f07ebcd0301f96fb9377cedcac567bfe36ab60c976b -009b99c791eee28917b3bbe7a10ea73e24c3c288f -Gas remaining: 1039995.417 units remaining -``` - -Then you would need to find out the big_map id of the token contracts, pools, and farms you want to map. For example, Kolibri USD (kUSD) stores user balances in a big_map with id 380. - -`octez-client get element exprvEJ9kYbvt2rmka1jac8voDT4xJSAiy48YJdtrXEVxrdZJRpLYr of big map 380` - -``` -user:~ % tezos-client get element exprvD3jLXJyLguSwR7D95ZZoRHvahH2jdbBbv83m5yrZzyUfR2pVY of big map 380 -Pair {} 784604439440371 -user:~ % -``` - -That is, to find out the balances of FA1.2 and FA2 tokens, you need to collect in advance the id of the necessary big_map contracts of tokens, and in turn, ask them for values ​​by address. - -See how much more complicated it is to query data from blockchain rather than to use indexers. - - - -# How to Use the TzKT API in a Simple Project on Tezos - -Previously, we talked about how indexers work and showed a couple of examples. Now let's explain how to use them for real-life tasks. First, let's make a simple website for displaying the balance of an address, then a more complex dashboard for displaying liquidity baking data and calculating indicators. - -We will use public TzKT and JavaScript with the jQuery library in our examples. Nothing complicated, just one HTML file with a script. - -## Displaying Address Balance Using the TzKT Public Indexer - -The most popular use case for blockchain explorers and indexers is to check the balance of different tokens for a given address. It is convenient when storing cryptocurrencies in a cold wallet: you do not need to connect it to the network again, thus endangering the funds. - -We will make a simple page where users can enter their addresses and check the balance in tez. Then we will add the display of token balances and some other information. To do this, we will use the TzKT API. - -First, let's create an empty HTML file in VS Code (you may use another editor, of course) and add the essential elements: doctype, head, title, and body. - -![check balance 1](/img/check_balance_html_1.png) - -We will use AJAX and the jQuery library to request data via the API and process it. Incorporating a library is simple: just provide a link to it in the script element. - -![check balance 2](/img/check_balance_html_2.png) - -Let's get the balance of our address via AJAX. - -![check balance ajax](/img/check_balance_ajax.png) - -First, we added the $(document).ready() command. It ensures the page is loaded before the scripts are processed. - -Then we wrote a request to TzKT using AJAX: in "URL,” the request link to get the balance; in "type," the type of the GET request to get information; and in "success," the function that will process the response. - -In the function, we will declare the variable balance, assign the value of the answer (data), and immediately divide it by a million. You must do this because the indexer returns the balance in mutez, millionths of tez. - -To use the balance variable in HTML, you need to assign an internal id to this variable. Let's do this with the document.getElementById method. - -In the end, we will add a h2 element, in which we will display the balance. To display the variable's value, we use the span element and the id previously assigned to the balance variable. - -Let's open the page and check the result. - -![](/img/check_balance_result.png) - -### Adding a button and field to check specific address balance - -AJAX now sends an API request as soon as the page loads. Let's add a button, pressing which will launch the request. - -![](/img/check_balance_button.png) - -To do this, wrap the h2 in a div element and make it hidden with the style="display:none" parameter. - -Let's create a button and add a call to the check function to it, in which we will place the entire request code. At the end of the function, add a change in the display style of the div to a visible block. - -Now we need to add a field for entering the user's address and tweak the check() function to insert it into the API request. - -![](/img/check_balance_input.png) - -To do this, we did the following: - -1. Added the address parameter to the check() function. -2. Changed the value of the "URL" field. When run, the script will make a valid API request using the received address. -3. Added a field for entering an address with an id. -4. Changed the button's code so that pressing it would launch the check() function and pass the entered address to it. - -Now you can enter any address and, by pressing the button, get its balance in tez. - -![](/img/check_balance_input_result.png) - -Experiment: take [the code of this page](https://gist.github.com/pavelTU/e48c71d09ff5dcfb5343699d485760d9), paste it into an empty HTML file, and open it in a browser. - -This is a simple example: the TzKT API returns the user's balance as JSON with only one number. The answer does not even need to be further processed: everything works as it is. - -![](/img/tzkt_api_balance.png) - -When solving real-life cases, however, you will have to work with JSON arrays and carefully read API responses to select the necessary data. The next chapter explores this matter further. - -## Displaying liquidity baking statistics - -That contract with 5 million tez is [Sirius DEX](https://siriustoken.io/), better known as liquidity baking. Here is its [contract on TzKT explorer](https://tzkt.io/KT1TxqZ8QtKvLu3V3JH7Gx58n7Co8pgtpQU5/operations/). - -Liquidity Baking is Tezos' unique DeFi protocol. Users contribute tez and tzBTC to it to provide liquidity for the exchange, and the Tezos network itself adds another 2.5 tez to the pool in each block. Thus, the balances of liquidity providers in this pool are constantly growing. - -Since all liquidity backing data is stored on-chain, we can get it using API requests to public indexers and then calculate the annual yield and other helpful information. - -First, we are interested in the balance of Sirius DEX, particularly how many tez and tzBTC the contract holds. We will receive these numbers from the indexer. - -We need to calculate how much tez the protocol subsidies per year. Here you can calculate the number of seconds in a year, divide this value by the average block creation time—30 seconds—and multiply by one subsidy. - -It remains to find out the current value of assets in Sirius DEX and the value of subsidized tez for the year and divide these values​​​​, which would be the annual return or APY. - -We are starting a new page. First, let's try to get something simple, like the Sirius DEX internal contract id in the TzKT database. - -![](/img/sirius_contract_id_html.png) - -Let's copy the code of the first example and remove the button and the field for entering the address from it. Let's replace the API request URL with https://api.tzkt.io/v1/contracts/KT1TxqZ8QtKvLu3V3JH7Gx58n7Co8pgtpQU5 to get information about the contract from the indexer. - -Now it’s time to change the function for data processing. To get specific values, you need to specify the key. In our case, the key is id, and requesting a value using this key will look like var contractId = data.id. - -Ultimately, we assign an internal ID, "contractId", to the corresponding HTML element and display it on the page in the h2 element. - -![](/img/sirius_contract_id_result.png) - -We made sure everything worked, so now we can get the data we need: tez and tzBTC balances. - -First, we should examine TzKT's response to our API request. It has a balance field in tez, so we can get it without changing the request. - -![](/img/tzkt_sirius_contract.png) - -Now, assign the value from the received array to the balanceInTez variable by the balance key. - -![](/img/sirius_balance_html.png) - -![](/img/sirius_balance_result.png) - -Now we need to get the amount of tzBTC. Let's examine the storage of the liquidity backing contract on TzKT: it shows the amount of tzBTC under the tokenPool key. You can access it by requesting the contents of the storage. - -![](/img/tzkt_sirius_storage.png) - -Now we should create another AJAX request. In the URL, we specify the API request for the storage content and assign the value of the corresponding entry, tokenPool, to the balanceInTZBTC variable. - -![](/img/sirius_balance_tzbtc_html.png) - -On the page, we will display the balance in tzBTC. - -![](/img/sirius_balance_tzbtc_result.png) - -Now it’s time to calculate the subsidies for the year. The average year has 31,556,926 seconds. We divide this value by the block creation time and multiply it by the subsidy amount. - -We can get the block creation time and subsidy amount from TzKT at https://api.tzkt.io/v1/protocols/current. They will be referred to as timeBetweenBlocks and lbSubsidy in the response. - -![](/img/tzkt_protocols.png) - -So, we need to get data from two API requests in one function and use it for calculations. But as the variables are local and can't be used in other functions, we need to use nested functions: - -1. Get the number of tez. -2. Write them to a variable. -3. Call the function to calculate annual percentage yield (APY) and pass it the number of tez as an argument. -4. In the APY function, get the necessary data and make calculations. -5. Write the results to variables and assign an ID. -6. Return to the first AJAX function and add a variable ID assignment at the end with the amount of tez. - -First, let's add a call to the checkTimeAndSubsidy function to get the Sirius DEX balance in tez. - -![](/img/call_time_and_subsidy.png) - -Below, we will declare the same function and add an AJAX call to request protocol data from TzKT. - -![](/img/time_and_subsidy_function_1.png) - -We should assign three new variables: - -- secondsInYear: the average number of seconds in a year, in fact, a constant. -- timeBetweenBlocks: block creation time. -- lbSubsidy: Sirius DEX pool subsidy in mutez. We divide it by a million to get the value in tez instead of mutez. - -We now have all the data to calculate the annual subsidy and APY liquidity backing. - -![](/img/time_and_subsidy_function_2.png) - -Now we can calculate the necessary values by creating the variables yearlySubsidy and APY. - -To calculate APY, it is not necessary to obtain asset prices. At any given moment, the value of all tez in the pool equals the value of all tzBTC. To simplify APY calculation, we assume that the user adds to the liquidity pool not tez and tzBTC, but twice as much tez. His APY would be a share of the yearly subsidy divided by the liquidity he provided. Roughly speaking, APY = yearlySubsidy / (balanceInTez × 2) × 100%. - -It’s now possible to give the annual amount of subsidies and APY internal ID values and add them to the page. - -![](/img/time_and_subsidy_result.png) - -[Example code on Github](https://gist.github.com/pavelTU/5a56c940952f01e838a3ca98215eab15). - -## Self study - -Try to calculate the value of tez in the Sirius DEX liquidity pool. - -1. Find a smart contract of any liquidity pool with a stablecoin against tez: tez/USDt, tez/kUSD, or tez/uUSD. -2. Get the number of tokens in the pool using an API request. -3. Divide the number of tez by the number of stablecoins to find the price of tez. -4. Multiply the number of tez in the Sirius DEX pool by the resulting exchange rate. -5. Add the result to the appropriate line. - -### Solution - -First, check [the tez/kUSD contract](https://tzkt.io/KT1K4EwTpbvYN9agJdjpyJm4ZZdhpUNKB3F6/storage/) on an exchange on TzKT. Then examine the storage and find the necessary keys: tez_pool is the amount of tez, and token_pool stands for the amount of kUSD. - -![](/img/tez_kusd_contract.png) - -And we need to remember that the token's amount in the contract's storage is a natural number. And to display the actual amount of tokens, we need to divide that number by the corresponding number of decimals written in the contract's metadata. For example, kUSD has 18 decimals, and when we get raw token amount data, we need to divide it by 10^18 to get a human-readable amount. - -![](/img/kolibri_usd.png) - -Since we need to know the number of tez in the pool to calculate the cost of all tez, we need to use nested functions. After receiving the balanceInTez variable, we call the checkValueOfTez function with the balanceInTez argument. In this function, we use AJAX to get data from the tez/kUSD pool (remember to divide the number of tokens depending on the required decimals). - -Next, we calculate the price of one tez and the cost of all tez in the pool. In the end, we will add the readableValue variable: using the toLocaleString () method, we will add comma separators to the number. - -![](/img/homework_value_of_tez.png) - -As a result, we get the cost of all tez in the Sirius DEX pool, which we get completely from on-chain data. - -![](/img/homework_value_of_tez_result.png) +Other blockchain applications use indexers in similar ways: -:::note -This content was created by [Tezos Ukraine](https://tezos.org.ua/en) under MIT Licence. The original version can be found [here](https://indexers.tezos.org.ua/) in multiple languages. +- Decentralized exchanges use indexers to get historical data on the balances of each pool, transactions with them, and the value of tez. +Based on this data, they calculate the volume of transactions in tez and other currencies, as well as the historical prices of tokens in fiat currencies. +- NFT marketplace websites index transactions with their contracts to display new NFTs, transaction history, and statistics of the most popular tokens. +- Block explorers display data from all blocks with the help of indexers and allow users to find anything, like operation details by hash. +Thanks to this, users can find the information they need in a user-friendly graphical interface. diff --git a/sidebars.js b/sidebars.js index 624a63a2d..6214b3051 100644 --- a/sidebars.js +++ b/sidebars.js @@ -106,7 +106,6 @@ const sidebars = { ], }, 'developing/information/indexers', - 'developing/information/custom-indexers', ], }, ], diff --git a/static/img/call_time_and_subsidy.png b/static/img/call_time_and_subsidy.png deleted file mode 100644 index 8e7c5679f..000000000 Binary files a/static/img/call_time_and_subsidy.png and /dev/null differ diff --git a/static/img/check_balance_ajax.png b/static/img/check_balance_ajax.png deleted file mode 100644 index ecd80bef5..000000000 Binary files a/static/img/check_balance_ajax.png and /dev/null differ diff --git a/static/img/check_balance_button.png b/static/img/check_balance_button.png deleted file mode 100644 index 9f76fcc15..000000000 Binary files a/static/img/check_balance_button.png and /dev/null differ diff --git a/static/img/check_balance_html_1.png b/static/img/check_balance_html_1.png deleted file mode 100644 index 56cd36e6d..000000000 Binary files a/static/img/check_balance_html_1.png and /dev/null differ diff --git a/static/img/check_balance_html_2.png b/static/img/check_balance_html_2.png deleted file mode 100644 index 750ecc7d7..000000000 Binary files a/static/img/check_balance_html_2.png and /dev/null differ diff --git a/static/img/check_balance_input.png b/static/img/check_balance_input.png deleted file mode 100644 index 71c0684f0..000000000 Binary files a/static/img/check_balance_input.png and /dev/null differ diff --git a/static/img/check_balance_input_result.png b/static/img/check_balance_input_result.png deleted file mode 100644 index 9c210ad1c..000000000 Binary files a/static/img/check_balance_input_result.png and /dev/null differ diff --git a/static/img/check_balance_result.png b/static/img/check_balance_result.png deleted file mode 100644 index 0fecede82..000000000 Binary files a/static/img/check_balance_result.png and /dev/null differ diff --git a/static/img/developing/dipdup1.png b/static/img/developing/dipdup1.png deleted file mode 100644 index 10a67f700..000000000 Binary files a/static/img/developing/dipdup1.png and /dev/null differ diff --git a/static/img/developing/dipdup10.png b/static/img/developing/dipdup10.png deleted file mode 100644 index 4c61b835a..000000000 Binary files a/static/img/developing/dipdup10.png and /dev/null differ diff --git a/static/img/developing/dipdup11.png b/static/img/developing/dipdup11.png deleted file mode 100644 index 95aff33c2..000000000 Binary files a/static/img/developing/dipdup11.png and /dev/null differ diff --git a/static/img/developing/dipdup12.png b/static/img/developing/dipdup12.png deleted file mode 100644 index a8408d83f..000000000 Binary files a/static/img/developing/dipdup12.png and /dev/null differ diff --git a/static/img/developing/dipdup13.png b/static/img/developing/dipdup13.png deleted file mode 100644 index 1e08a3d85..000000000 Binary files a/static/img/developing/dipdup13.png and /dev/null differ diff --git a/static/img/developing/dipdup14.png b/static/img/developing/dipdup14.png deleted file mode 100644 index 91b2b0167..000000000 Binary files a/static/img/developing/dipdup14.png and /dev/null differ diff --git a/static/img/developing/dipdup15.png b/static/img/developing/dipdup15.png deleted file mode 100644 index 1177c5491..000000000 Binary files a/static/img/developing/dipdup15.png and /dev/null differ diff --git a/static/img/developing/dipdup16.png b/static/img/developing/dipdup16.png deleted file mode 100644 index afc5e1f94..000000000 Binary files a/static/img/developing/dipdup16.png and /dev/null differ diff --git a/static/img/developing/dipdup17.png b/static/img/developing/dipdup17.png deleted file mode 100644 index 11ffaf12c..000000000 Binary files a/static/img/developing/dipdup17.png and /dev/null differ diff --git a/static/img/developing/dipdup18.png b/static/img/developing/dipdup18.png deleted file mode 100644 index 9b2c0f574..000000000 Binary files a/static/img/developing/dipdup18.png and /dev/null differ diff --git a/static/img/developing/dipdup19.png b/static/img/developing/dipdup19.png deleted file mode 100644 index 457adcb3b..000000000 Binary files a/static/img/developing/dipdup19.png and /dev/null differ diff --git a/static/img/developing/dipdup2.png b/static/img/developing/dipdup2.png deleted file mode 100644 index 3c0a018b5..000000000 Binary files a/static/img/developing/dipdup2.png and /dev/null differ diff --git a/static/img/developing/dipdup20.png b/static/img/developing/dipdup20.png deleted file mode 100644 index a0497594d..000000000 Binary files a/static/img/developing/dipdup20.png and /dev/null differ diff --git a/static/img/developing/dipdup21.png b/static/img/developing/dipdup21.png deleted file mode 100644 index 968acf298..000000000 Binary files a/static/img/developing/dipdup21.png and /dev/null differ diff --git a/static/img/developing/dipdup22.png b/static/img/developing/dipdup22.png deleted file mode 100644 index 6c0a59eb9..000000000 Binary files a/static/img/developing/dipdup22.png and /dev/null differ diff --git a/static/img/developing/dipdup23.png b/static/img/developing/dipdup23.png deleted file mode 100644 index 310c9dfa3..000000000 Binary files a/static/img/developing/dipdup23.png and /dev/null differ diff --git a/static/img/developing/dipdup24.png b/static/img/developing/dipdup24.png deleted file mode 100644 index 7d6dc0c60..000000000 Binary files a/static/img/developing/dipdup24.png and /dev/null differ diff --git a/static/img/developing/dipdup25.png b/static/img/developing/dipdup25.png deleted file mode 100644 index b2f556cf2..000000000 Binary files a/static/img/developing/dipdup25.png and /dev/null differ diff --git a/static/img/developing/dipdup26.png b/static/img/developing/dipdup26.png deleted file mode 100644 index e2a45288d..000000000 Binary files a/static/img/developing/dipdup26.png and /dev/null differ diff --git a/static/img/developing/dipdup27.png b/static/img/developing/dipdup27.png deleted file mode 100644 index 8a99e9b14..000000000 Binary files a/static/img/developing/dipdup27.png and /dev/null differ diff --git a/static/img/developing/dipdup28.png b/static/img/developing/dipdup28.png deleted file mode 100644 index b5eecd15f..000000000 Binary files a/static/img/developing/dipdup28.png and /dev/null differ diff --git a/static/img/developing/dipdup3.png b/static/img/developing/dipdup3.png deleted file mode 100644 index 3eefcfe1c..000000000 Binary files a/static/img/developing/dipdup3.png and /dev/null differ diff --git a/static/img/developing/dipdup4.png b/static/img/developing/dipdup4.png deleted file mode 100644 index 3788819cb..000000000 Binary files a/static/img/developing/dipdup4.png and /dev/null differ diff --git a/static/img/developing/dipdup5.png b/static/img/developing/dipdup5.png deleted file mode 100644 index a750097d9..000000000 Binary files a/static/img/developing/dipdup5.png and /dev/null differ diff --git a/static/img/developing/dipdup6.png b/static/img/developing/dipdup6.png deleted file mode 100644 index 9d097eda6..000000000 Binary files a/static/img/developing/dipdup6.png and /dev/null differ diff --git a/static/img/developing/dipdup7.png b/static/img/developing/dipdup7.png deleted file mode 100644 index ef4dde143..000000000 Binary files a/static/img/developing/dipdup7.png and /dev/null differ diff --git a/static/img/developing/dipdup8.png b/static/img/developing/dipdup8.png deleted file mode 100644 index fec8d03d4..000000000 Binary files a/static/img/developing/dipdup8.png and /dev/null differ diff --git a/static/img/developing/dipdup9.png b/static/img/developing/dipdup9.png deleted file mode 100644 index 07f8293f6..000000000 Binary files a/static/img/developing/dipdup9.png and /dev/null differ diff --git a/static/img/homework_value_of_tez.png b/static/img/homework_value_of_tez.png deleted file mode 100644 index 2bec8fdf4..000000000 Binary files a/static/img/homework_value_of_tez.png and /dev/null differ diff --git a/static/img/homework_value_of_tez_result.png b/static/img/homework_value_of_tez_result.png deleted file mode 100644 index 0df2a45fc..000000000 Binary files a/static/img/homework_value_of_tez_result.png and /dev/null differ diff --git a/static/img/kolibri_usd.png b/static/img/kolibri_usd.png deleted file mode 100644 index c39920b10..000000000 Binary files a/static/img/kolibri_usd.png and /dev/null differ diff --git a/static/img/que_pasa_sql.png b/static/img/que_pasa_sql.png deleted file mode 100644 index ecc74b893..000000000 Binary files a/static/img/que_pasa_sql.png and /dev/null differ diff --git a/static/img/sirius_balance_html.png b/static/img/sirius_balance_html.png deleted file mode 100644 index c016d2a9e..000000000 Binary files a/static/img/sirius_balance_html.png and /dev/null differ diff --git a/static/img/sirius_balance_result.png b/static/img/sirius_balance_result.png deleted file mode 100644 index 82c57d669..000000000 Binary files a/static/img/sirius_balance_result.png and /dev/null differ diff --git a/static/img/sirius_balance_tzbtc_html.png b/static/img/sirius_balance_tzbtc_html.png deleted file mode 100644 index 3c0e547cb..000000000 Binary files a/static/img/sirius_balance_tzbtc_html.png and /dev/null differ diff --git a/static/img/sirius_balance_tzbtc_result.png b/static/img/sirius_balance_tzbtc_result.png deleted file mode 100644 index 74ddaecee..000000000 Binary files a/static/img/sirius_balance_tzbtc_result.png and /dev/null differ diff --git a/static/img/sirius_contract_id_html.png b/static/img/sirius_contract_id_html.png deleted file mode 100644 index 2b8c23de8..000000000 Binary files a/static/img/sirius_contract_id_html.png and /dev/null differ diff --git a/static/img/sirius_contract_id_result.png b/static/img/sirius_contract_id_result.png deleted file mode 100644 index 5ba0fb401..000000000 Binary files a/static/img/sirius_contract_id_result.png and /dev/null differ diff --git a/static/img/temple_query.png b/static/img/temple_query.png deleted file mode 100644 index 24b2a65d6..000000000 Binary files a/static/img/temple_query.png and /dev/null differ diff --git a/static/img/tez_kusd_contract.png b/static/img/tez_kusd_contract.png deleted file mode 100644 index 9ea9fc9dd..000000000 Binary files a/static/img/tez_kusd_contract.png and /dev/null differ diff --git a/static/img/time_and_subsidy_function_1.png b/static/img/time_and_subsidy_function_1.png deleted file mode 100644 index 1ee6a0d66..000000000 Binary files a/static/img/time_and_subsidy_function_1.png and /dev/null differ diff --git a/static/img/time_and_subsidy_function_2.png b/static/img/time_and_subsidy_function_2.png deleted file mode 100644 index 29527f74a..000000000 Binary files a/static/img/time_and_subsidy_function_2.png and /dev/null differ diff --git a/static/img/time_and_subsidy_result.png b/static/img/time_and_subsidy_result.png deleted file mode 100644 index 0c3550de4..000000000 Binary files a/static/img/time_and_subsidy_result.png and /dev/null differ diff --git a/static/img/tzkt_api_balance.png b/static/img/tzkt_api_balance.png deleted file mode 100644 index 943835906..000000000 Binary files a/static/img/tzkt_api_balance.png and /dev/null differ diff --git a/static/img/tzkt_protocols.png b/static/img/tzkt_protocols.png deleted file mode 100644 index 3da37821a..000000000 Binary files a/static/img/tzkt_protocols.png and /dev/null differ diff --git a/static/img/tzkt_sirius_contract.png b/static/img/tzkt_sirius_contract.png deleted file mode 100644 index e408f5bb2..000000000 Binary files a/static/img/tzkt_sirius_contract.png and /dev/null differ diff --git a/static/img/tzkt_sirius_storage.png b/static/img/tzkt_sirius_storage.png deleted file mode 100644 index 9a5a0db81..000000000 Binary files a/static/img/tzkt_sirius_storage.png and /dev/null differ diff --git a/static/img/usdt_que_pasa.png b/static/img/usdt_que_pasa.png deleted file mode 100644 index 51ca04521..000000000 Binary files a/static/img/usdt_que_pasa.png and /dev/null differ