Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

fix: --embedded-database does not work when using fuel-indexer binary directly #892

Merged
merged 27 commits into from
Jun 6, 2023

Conversation

lostman
Copy link
Contributor

@lostman lostman commented May 10, 2023

Description

This PR makes fuel-indexer start run an embedded postgres database when --embedded-database flag is specified.

When fuel-indexer exits, the embedded database is shut down. For this purpose, fuel-indexer now listens to SIGHUP/INT/TERM signals.

Testing steps

fuel-indexer

  1. --embedded-database
cargo run -p fuel-indexer -- run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --auth-enabled --auth-strategy jwt --jwt-secret $FUEL_JWT_TOKEN --jwt-issuer FuelLabs --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database
  1. --embedded-database and --local-fuel-node
cargo run -p fuel-indexer --features api-server,fuel-core-lib -- run --run-migrations --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database --local-fuel-node

Note: fuel-core-lib must be enabled.

3 --embedded-database and --all-features, --no-default-features

cargo run -p fuel-indexer --all-features -- run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database
cargo run -p fuel-indexer --no-default-features -- run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database
  1. fuel-api-server --all-features, --no-default-features
cargo run -p fuel-indexer-api-server --all-features -- run --fuel-node-host beta-3.fuel.network --fuel-node-port 80
cargo run -p fuel-indexer-api-server --no-default-features -- run --fuel-node-host beta-3.fuel.network --fuel-node-port 80

And check it works by visiting:
http://localhost:29987/api/playground/fuel_examples/explorer_indexer

Changelog

  • start an embedded database when --embedded-database is specified
  • listen for termination signals
  • stop the embedded database when a termination signal is received
  • cleanup: forc index start passes --embedded-database flag to fuel-indexer instead of calling forc_postgres::commands::create::exec
  • change how handles are created for various features—no nested cfg's
  • vendored openssl
    it's a fix for
    ./fuel-node: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
    
    which was suggested here:
    Cannot build due to libssl.so.1.1 link error launchbadge/sqlx#473
    "the issue is because procedural macro executed in a slightly different environment"
    I think we're running into the same situation with our Docker build—we build with cargo chef image, but run on ubuntu:22.04

@lostman lostman force-pushed the maciej/813-fuel-indexer-embedded-database branch from fe336ad to f10df37 Compare May 10, 2023 19:35
@lostman lostman force-pushed the maciej/813-fuel-indexer-embedded-database branch from d10b40e to 9c92a65 Compare May 11, 2023 15:38
@lostman lostman marked this pull request as ready for review May 11, 2023 17:25
@lostman lostman requested review from ra0x3, deekerno and 0xmovses May 12, 2023 12:17
Copy link
Contributor

@ra0x3 ra0x3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lostman

packages/fuel-indexer/Cargo.toml Outdated Show resolved Hide resolved
packages/fuel-indexer/src/commands/run.rs Outdated Show resolved Hide resolved
packages/fuel-indexer/src/commands/run.rs Outdated Show resolved Hide resolved
packages/fuel-indexer/src/commands/run.rs Outdated Show resolved Hide resolved
plugins/forc-index/src/ops/forc_index_start.rs Outdated Show resolved Hide resolved
@@ -57,56 +100,82 @@ pub async fn exec(args: IndexerArgs) -> anyhow::Result<()> {

let service_handle = tokio::spawn(service.run());

// for graceful shutdown
Copy link
Contributor

@ra0x3 ra0x3 May 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lostman

  • Just a heads up here.
  • I see the logic of how the service starts and is feature flagged has been changed
  • Totally fine with that overall, but just an FYI that the manual testing to verify this works is gonna be a bit vigorous
  • You will need to test:
    • The fuel-indexer service starts with all/any features --all-features, --no-default-features
    • The fuel-indexer service starts with --local-fuel-node
    • The fuel-indexer-api-server starts with all/any features --all-features, --no-default-features
    • Native execution starts with all/any features --all-features, --no-default-features
    • Native execution starts with --local-fuel-node
      • Native execution is here btw
      • It's pretty much a copy/paste of how the non-native fuel-indexer service is started
    • Docker builds still work
      • Hello-indexer example
      • Block-explorer example
      • Hello-indexer-native example
  • And we'd need to update the PR testing steps to reflect this

packages/fuel-indexer/src/commands/run.rs Show resolved Hide resolved
@lostman
Copy link
Contributor Author

lostman commented May 17, 2023

After the fix yesterday, I re-ran various variations. Pasting outputs here:

  1. fuel-indexer --embedded-database --local-fuel-node
cargo run -p fuel-indexer --features api-server,fuel-core-lib -- run --run-migrations --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database --local-fuel-node
    Finished dev [unoptimized + debuginfo] target(s) in 0.90s
     Running `target/debug/fuel-indexer run --run-migrations --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database --local-fuel-node`
▪▪▪▪▪ ⏱  Setting up database...
2023-05-17T12:12:50.565433Z  INFO fuel_indexer::commands::run: 66: Configuration: IndexerConfig { verbose: false, local_fuel_node: true, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "localhost", port: "4000" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "12345", database: "postgres9", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
2023-05-17T12:12:50.592033Z  INFO sqlx::postgres::notice: 157: relation "_sqlx_migrations" already exists, skipping
2023-05-17T12:12:53.730486Z  WARN fuel_indexer::executor: 77: No end_block specified in manifest. Indexer will run forever.
2023-05-17T12:12:53.730513Z  INFO fuel_indexer::executor: 96: Subscribing to Fuel node at localhost:4000
2023-05-17T12:12:53.731412Z  INFO fuel_indexer::service: 142: Registered Indexer(fuel_indexer_test.index1)
2023-05-17T12:12:53.731456Z  INFO fuel_indexer::commands::run: 92: ✨ ✨ GraphQL Playground at: http://localhost:29987/api/playground/:namespace/:identifier
2023-05-17T12:12:53.731727Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:12:53.731843Z  INFO fuel_core_chain_config::config::chain: 96: Initial Accounts
2023-05-17T12:12:53.732086Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
2023-05-17T12:12:53.733004Z  INFO fuel_core_chain_config::config::chain: 105: PrivateKey(0xde97d8624a438121b86a1956544bd72ed68cd69f2c99555b08b1e8c51ffd511c), Address(0x6b63804cfbf9856e68e5b6e7aef238dc8311ec55bec04df774003a2c96e0418e [bech32: fuel1dd3cqn8mlxzku689kmn6au3cmjp3rmz4hmqymam5qqaze9hqgx8qtjpwn9]), Balance(10000000)
2023-05-17T12:12:53.733474Z  INFO fuel_core_chain_config::config::chain: 105: PrivateKey(0x37fa81c84ccd547c30c176b118d5cb892bdb113e8e80141f266519422ef9eefd), Address(0x54944e5b8189827e470e5a8bacfc6c3667397dc4e1eef7ef3519d16d6d6c6610 [bech32: fuel12j2yukup3xp8u3cwt296elrvxennjlwyu8h00me4r8gk6mtvvcgqmtakkk]), Balance(10000000)
2023-05-17T12:12:53.733897Z  INFO fuel_core_chain_config::config::chain: 105: PrivateKey(0x862512a2363db2b3a375c0d4bbbd27172180d89f23f2e259bac850ab02619301), Address(0xe10f526b192593793b7a1559a391445faba82a1d669e3eb2dcd17f9c121b24b1 [bech32: fuel1uy84y6ceykfhjwm6z4v68y2yt746s2sav60ravku69lecysmyjcss0yrdx]), Balance(10000000)
2023-05-17T12:12:53.734317Z  INFO fuel_core_chain_config::config::chain: 105: PrivateKey(0x976e5c3fa620092c718d852ca703b6da9e3075b9f2ecb8ed42d9f746bf26aafb), Address(0x577e424ee53a16e6a85291feabc8443862495f74ac39a706d2dd0b9fc16955eb [bech32: fuel12alyynh98gtwd2zjj8l2hjzy8p3yjhm54su6wpkjm59elstf2h4swddd2d]), Balance(10000000)
2023-05-17T12:12:53.734748Z  INFO fuel_core_chain_config::config::chain: 105: PrivateKey(0x7f8a325504e7315eda997db7861c9447f5c3eff26333b20180475d94443a10c6), Address(0xc36be0e14d3eaf5d8d233e0f4a40b3b4e48427d25f84c460d2b03b242a38479e [bech32: fuel1cd47pc2d86h4mrfr8c855s9nknjggf7jt7zvgcxjkqajg23cg70qnxg0hd]), Balance(10000000)
2023-05-17T12:12:53.736208Z ERROR fuel_indexer::executor: 127: Failed to retrieve blocks: Error making HTTP request: error sending request for url (http://localhost:4000/graphql): error trying to connect: tcp connect error: Connection refused (os error 61)
2023-05-17T12:12:53.736962Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:53.845704Z  WARN fuel_core::service: 89: No RocksDB path configured, initializing database with a tmp directory
2023-05-17T12:12:54.085581Z  INFO new{name=}:_commit_result{block_id=b1807ca9f2eec7e459b866ecf69b68679fc6b205a9a85c16bd4943d1bfc6fb2a height=0 tx_status=[]}: fuel_core_importer::importer: 233: Committed block
2023-05-17T12:12:54.090531Z  INFO new{name=}: fuel_core::graphql_api::service: 207: Binding GraphQL provider to 127.0.0.1:4000
2023-05-17T12:12:54.752627Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:55.761440Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:56.769358Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:57.778084Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:58.786502Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:12:59.794753Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:13:00.801950Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
^C2023-05-17T12:13:00.914303Z  INFO fuel_indexer::commands::run: 154: Received SIGINT. Stopping services
2023-05-17T12:13:00.914368Z  INFO forc_postgres::pg: 51: Using database directory at "/Users/mw/.fuel/indexer/postgres9"
2023-05-17T12:13:00.934761Z  INFO sqlx::query: select exists(SELECT 1 from …; rows affected: 0, rows returned: 1, elapsed: 731.625µs

select
  exists(
    SELECT
      1
    from
      pg_database
    WHERE
      datname = $1
  )

2023-05-17T12:13:00.934858Z  INFO forc_postgres::ops::forc_postgres_stopdb: 29:
Stopping database at 'postgres://postgres:postgres@localhost:12345/postgres9'.

2023-05-17T12:13:00.934932Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.934 CEST [75684] LOG:  could not receive data from client: Connection reset by peer
2023-05-17T12:13:00.939556Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.939 CEST [75672] LOG:  received fast shutdown request
2023-05-17T12:13:00.939606Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.939 CEST [75672] LOG:  aborting any active transactions
2023-05-17T12:13:00.939994Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.939 CEST [75683] FATAL:  terminating connection due to administrator command
2023-05-17T12:13:00.940088Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.939 CEST [75682] FATAL:  terminating connection due to administrator command
2023-05-17T12:13:00.940151Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.940 CEST [75681] FATAL:  terminating connection due to administrator command
2023-05-17T12:13:00.940240Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.940 CEST [75672] LOG:  background worker "logical replication launcher" (PID 75679) exited with exit code 1  
2023-05-17T12:13:00.940821Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.940 CEST [75674] LOG:  shutting down
2023-05-17T12:13:00.944104Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:13:00.944 CEST [75672] LOG:  database system is shut down
2023-05-17T12:13:01.045214Z  INFO forc_postgres::ops::forc_postgres_stopdb: 35: ✅ Successfully stopped database.
2023-05-17T12:13:01.045348Z  INFO pg_embed::command_executor: 144: waiting for server to shut down.... done
2023-05-17T12:13:01.045381Z  INFO pg_embed::command_executor: 144: server stopped
  1. fuel-indexer --all-features --embedded-database
cargo run -p fuel-indexer --all-features -- run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database
    Finished dev [unoptimized + debuginfo] target(s) in 0.94s
     Running `target/debug/fuel-indexer run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database`
▪▪▪▪▪ ⏱  Setting up database...
2023-05-17T12:14:36.127670Z  INFO fuel_indexer::commands::run: 66: Configuration: IndexerConfig { verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "beta-3.fuel.network", port: "80" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "12345", database: "postgres9", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
2023-05-17T12:14:36.154882Z  INFO sqlx::postgres::notice: 157: relation "_sqlx_migrations" already exists, skipping
2023-05-17T12:14:39.318450Z  WARN fuel_indexer::executor: 77: No end_block specified in manifest. Indexer will run forever.
2023-05-17T12:14:39.318476Z  INFO fuel_indexer::executor: 96: Subscribing to Fuel node at beta-3.fuel.network:80
2023-05-17T12:14:39.319507Z  INFO fuel_indexer::service: 142: Registered Indexer(fuel_indexer_test.index1)
2023-05-17T12:14:39.319576Z  INFO fuel_indexer::commands::run: 92: ✨ ✨ GraphQL Playground at: http://localhost:29987/api/playground/:namespace/:identifier
2023-05-17T12:14:39.319675Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:14:39.320065Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
2023-05-17T12:14:54.601558Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.612604Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:14:54.617425Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.619109Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:14:54.622305Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.623897Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:14:54.629227Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.630991Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:14:54.635465Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.647524Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:14:54.651655Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:14:54.653619Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.319273Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.321394Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.344658Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.352705Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.356261Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.357768Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.362024Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.363475Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.367429Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.368859Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.371872Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.373382Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.381159Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.382634Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.386097Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:04.387466Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:04.387919Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_messageout handling MessageOut event
^C2023-05-17T12:15:04.783511Z  INFO fuel_indexer::commands::run: 154: Received SIGINT. Stopping services
2023-05-17T12:15:04.783632Z  INFO forc_postgres::pg: 51: Using database directory at "/Users/mw/.fuel/indexer/postgres9"
2023-05-17T12:15:04.806695Z  INFO sqlx::query: select exists(SELECT 1 from …; rows affected: 0, rows returned: 1, elapsed: 642.750µs

select
  exists(
    SELECT
      1
    from
      pg_database
    WHERE
      datname = $1
  )

2023-05-17T12:15:04.806795Z  INFO forc_postgres::ops::forc_postgres_stopdb: 29:
Stopping database at 'postgres://postgres:postgres@localhost:12345/postgres9'.

2023-05-17T12:15:04.806865Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.806 CEST [75862] LOG:  could not receive data from client: Connection reset by peer
2023-05-17T12:15:04.813425Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.813 CEST [75789] LOG:  received fast shutdown request
2023-05-17T12:15:04.813465Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.813 CEST [75789] LOG:  aborting any active transactions
2023-05-17T12:15:04.813950Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.813 CEST [75799] FATAL:  terminating connection due to administrator command
2023-05-17T12:15:04.813984Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.813 CEST [75798] FATAL:  terminating connection due to administrator command
2023-05-17T12:15:04.814057Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.814 CEST [75789] LOG:  background worker "logical replication launcher" (PID 75796) exited with exit code 1  
2023-05-17T12:15:04.814637Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.814 CEST [75791] LOG:  shutting down
2023-05-17T12:15:04.818698Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:15:04.818 CEST [75789] LOG:  database system is shut down
2023-05-17T12:15:04.915923Z  INFO forc_postgres::ops::forc_postgres_stopdb: 35: ✅ Successfully stopped database.
2023-05-17T12:15:04.916036Z  INFO pg_embed::command_executor: 144: waiting for server to shut down.... done
2023-05-17T12:15:04.916060Z  INFO pg_embed::command_executor: 144: server stopped
  1. fuel-indexer --no-default-features --embedded-database
cargo run -p fuel-indexer --no-default-features -- run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database
   Compiling fuel-indexer-postgres v0.13.0 (/Users/mw/Data/Code/Work/FuelLabs/fuel-indexer/packages/fuel-indexer-database/postgres)
   Compiling forc-postgres v0.13.0 (/Users/mw/Data/Code/Work/FuelLabs/fuel-indexer/plugins/forc-postgres)
   Compiling fuel-indexer-database v0.13.0 (/Users/mw/Data/Code/Work/FuelLabs/fuel-indexer/packages/fuel-indexer-database)
   Compiling fuel-indexer-schema v0.13.0 (/Users/mw/Data/Code/Work/FuelLabs/fuel-indexer/packages/fuel-indexer-schema)
   Compiling fuel-indexer v0.13.0 (/Users/mw/Data/Code/Work/FuelLabs/fuel-indexer/packages/fuel-indexer)
    Finished dev [unoptimized + debuginfo] target(s) in 8.22s
     Running `target/debug/fuel-indexer run --run-migrations --fuel-node-host beta-3.fuel.network --fuel-node-port 80 --postgres-user postgres --postgres-password postgres --postgres-database postgres9 --postgres-port 12345 --embedded-database`
▪▪▪▪▪ ⏱  Setting up database...
2023-05-17T12:15:32.319309Z  INFO fuel_indexer::commands::run: 66: Configuration: IndexerConfig { verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "beta-3.fuel.network", port: "80" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "12345", database: "postgres9", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
2023-05-17T12:15:32.344063Z  INFO sqlx::postgres::notice: 157: relation "_sqlx_migrations" already exists, skipping
2023-05-17T12:15:35.508785Z  WARN fuel_indexer::executor: 77: No end_block specified in manifest. Indexer will run forever.
2023-05-17T12:15:35.508816Z  INFO fuel_indexer::executor: 96: Subscribing to Fuel node at beta-3.fuel.network:80
2023-05-17T12:15:35.509855Z  INFO fuel_indexer::service: 142: Registered Indexer(fuel_indexer_test.index1)
2023-05-17T12:15:35.509888Z  INFO fuel_indexer::commands::run: 92: ✨ ✨ GraphQL Playground at: http://localhost:29987/api/playground/:namespace/:identifier
2023-05-17T12:15:49.788462Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.791939Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:49.796597Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.797899Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:49.800469Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.804594Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:49.817520Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.819213Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:49.839511Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.843995Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:49.851134Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:49.852460Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.358823Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.361053Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.372395Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.374058Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.380904Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.384130Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.390086Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.407643Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.413042Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.417672Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.420831Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.422566Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.426974Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.430538Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.436155Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:15:59.438591Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:15:59.439468Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_messageout handling MessageOut event
2023-05-17T12:16:08.088714Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.090562Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:08.091419Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_messageout handling MessageOut event
2023-05-17T12:16:08.099821Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.099851Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.099862Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.099870Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.099879Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.101402Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:08.111980Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.112017Z  WARN fuel_indexer::ffi: 109: Unknown message type ID; check ABI to make sure that message types are correct.
2023-05-17T12:16:08.112045Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.112054Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.112061Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:08.113826Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:08.114744Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_messageout handling MessageOut event
2023-05-17T12:16:16.725032Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.725061Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.725069Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.725076Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.725083Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.736736Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:16.787082Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.787149Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.811972Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:16.857546Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.857576Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.857584Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.857592Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.857599Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.859338Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:16.867419Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.867453Z  WARN fuel_indexer::ffi: 109: Unknown message type ID; check ABI to make sure that message types are correct.
2023-05-17T12:16:16.867484Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.867495Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.867504Z  INFO fuel_indexer::ffi: 110: Not subscribed to this contract. Will skip this receipt event. <('-'<)
2023-05-17T12:16:16.872906Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_scriptresult handling ScriptResult event.
2023-05-17T12:16:16.874199Z  INFO fuel_indexer::ffi: 110: fuel_indexer_test_messageout handling MessageOut event
^C2023-05-17T12:16:40.977624Z  INFO fuel_indexer::commands::run: 154: Received SIGINT. Stopping services
2023-05-17T12:16:40.977778Z  INFO forc_postgres::pg: 51: Using database directory at "/Users/mw/.fuel/indexer/postgres9"
2023-05-17T12:16:41.023546Z  INFO sqlx::query: select exists(SELECT 1 from …; rows affected: 0, rows returned: 1, elapsed: 2.977ms

select
  exists(
    SELECT
      1
    from
      pg_database
    WHERE
      datname = $1
  )

2023-05-17T12:16:41.023644Z  INFO forc_postgres::ops::forc_postgres_stopdb: 29:
Stopping database at 'postgres://postgres:postgres@localhost:12345/postgres9'.

2023-05-17T12:16:41.024256Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.023 CEST [76002] LOG:  could not receive data from client: Connection reset by peer
2023-05-17T12:16:41.031295Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.031 CEST [75932] LOG:  received fast shutdown request
2023-05-17T12:16:41.031324Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.031 CEST [75932] LOG:  aborting any active transactions
2023-05-17T12:16:41.032856Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.031 CEST [75944] FATAL:  terminating connection due to administrator command
2023-05-17T12:16:41.032879Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.031 CEST [75942] FATAL:  terminating connection due to administrator command
2023-05-17T12:16:41.032886Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.032 CEST [75941] FATAL:  terminating connection due to administrator command
2023-05-17T12:16:41.032893Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.032 CEST [75932] LOG:  background worker "logical replication launcher" (PID 75939) exited with exit code 1  
2023-05-17T12:16:41.033174Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.033 CEST [75934] LOG:  shutting down
2023-05-17T12:16:41.042005Z  INFO pg_embed::command_executor: 144: 2023-05-17 14:16:41.041 CEST [75932] LOG:  database system is shut down
2023-05-17T12:16:41.135822Z  INFO forc_postgres::ops::forc_postgres_stopdb: 35: ✅ Successfully stopped database.
2023-05-17T12:16:41.135874Z  INFO pg_embed::command_executor: 144: waiting for server to shut down.... done
2023-05-17T12:16:41.135889Z  INFO pg_embed::command_executor: 144: server stopped
  1. fuel-indexer-api-server --all-features
cargo run -p fuel-indexer-api-server --all-features -- run --fuel-node-host beta-3.fuel.network --fuel-node-port 80
    Finished dev [unoptimized + debuginfo] target(s) in 0.33s
     Running `target/debug/fuel-indexer-api-server run --fuel-node-host beta-3.fuel.network --fuel-node-port 80`
2023-05-17T12:38:48.175425Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:38:48.175909Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
^C
  1. fuel-indexer-api-server --no-default-features
cargo run -p fuel-indexer-api-server --no-default-features -- run --fuel-node-host beta-3.fuel.network --fuel-node-port 80
    Finished dev [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/fuel-indexer-api-server run --fuel-node-host beta-3.fuel.network --fuel-node-port 80`
2023-05-17T12:40:02.970192Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:40:02.970630Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
^C
  1. native execution --all-features
cargo run -p hello_indexer_native --all-features -- --manifest examples/hello-world-native/hello-indexer-native/hello_indexer_native.manifest.yaml --run-migrations
    Finished dev [unoptimized + debuginfo] target(s) in 4.54s
     Running `target/debug/hello_indexer_native --manifest examples/hello-world-native/hello-indexer-native/hello_indexer_native.manifest.yaml --run-migrations`
2023-05-17T12:45:43.162755Z  INFO hello_indexer_native: 25: Configuration: IndexerConfig { verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "localhost", port: "4000" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "5432", database: "postgres", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
2023-05-17T12:45:43.199532Z  INFO sqlx::postgres::notice: 157: relation "_sqlx_migrations" already exists, skipping
2023-05-17T12:45:43.232701Z  WARN fuel_indexer::executor: 77: No end_block specified in manifest. Indexer will run forever.
2023-05-17T12:45:43.232742Z  INFO fuel_indexer::executor: 96: Subscribing to Fuel node at localhost:4000
2023-05-17T12:45:43.234059Z  INFO fuel_indexer::service: 184: Registered NativeIndex(fuel_examples.hello_indexer_native)
2023-05-17T12:45:43.234488Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:45:43.234960Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
2023-05-17T12:45:43.265207Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:45:44.277104Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:45:45.298159Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
^C
  1. native execution --no-default-features
cargo run -p hello_indexer_native --no-default-features -- --manifest examples/hello-world-native/hello-indexer-native/hello_indexer_native.manifest.yaml --run-migrations
    Finished dev [unoptimized + debuginfo] target(s) in 0.81s
     Running `target/debug/hello_indexer_native --manifest examples/hello-world-native/hello-indexer-native/hello_indexer_native.manifest.yaml --run-migrations`
2023-05-17T12:45:56.403559Z  INFO hello_indexer_native: 25: Configuration: IndexerConfig { verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "localhost", port: "4000" }, graphql_api: GraphQLConfig { host: "localhost", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "localhost", port: "5432", database: "postgres", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
2023-05-17T12:45:56.448811Z  INFO sqlx::postgres::notice: 157: relation "_sqlx_migrations" already exists, skipping
2023-05-17T12:45:56.474936Z  WARN fuel_indexer::executor: 77: No end_block specified in manifest. Indexer will run forever.
2023-05-17T12:45:56.474976Z  INFO fuel_indexer::executor: 96: Subscribing to Fuel node at localhost:4000
2023-05-17T12:45:56.476140Z  INFO fuel_indexer::service: 184: Registered NativeIndex(fuel_examples.hello_indexer_native)
2023-05-17T12:45:56.476693Z  WARN fuel_indexer_lib::utils: 105: Failed to parse 'localhost:29987' as a SocketAddr due to 'invalid socket address syntax'. Retrying using ToSocketAddrs.
2023-05-17T12:45:56.477099Z  INFO fuel_indexer_lib::utils: 117: Parsed SocketAddr '127.0.0.1:29987' from 'localhost:29987'
2023-05-17T12:45:56.498280Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
2023-05-17T12:45:57.506598Z  INFO fuel_indexer::executor: 246: No new blocks to process, sleeping.
^C
  1. Examples/Docker:

hello-world builds images from scratch. On my laptop, postgres and indexer start but node fails with

hello-world-fuel-node-1     | ./fuel-node: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

block-explorer uses a pre-built image image: ghcr.io/fuellabs/fuel-indexer:latest and everything works fine.

@lostman lostman requested a review from ra0x3 May 17, 2023 13:15
@lostman lostman force-pushed the maciej/813-fuel-indexer-embedded-database branch from 2622986 to f3abd25 Compare May 17, 2023 14:36
Copy link
Contributor

@ra0x3 ra0x3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spent some time reviewing this:

The following things work:

  • Ctrl+C stops cargo run --embedded-database
  • forc index kill does not stop forc-postgres (but still stops forc-index)
  • Ctrl+C stops ./target/release/fuel-indexer
  • ./target/release/fuel-indexer --embedded-database works
  • ./target/release/fuel-indexer --no-default-features starts & stops
  • hello-native works

I was not able to get the following things to work:

  • explorer-indexer works via docker
  • hello-world works via docker

When trying to run the dockerized examples I'm consistently getting the following error

> docker compose up 

block-explorer-fuel-indexer-1  | ./fuel-indexer: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
block-explorer-fuel-indexer-1 exited with code 127

My local image was built with the following command:

docker build -t fuel-indexer/local:latest -f deployment/Dockerfile .

@lostman I'm assuming the Dockerized examples worked for you?

@lostman
Copy link
Contributor Author

lostman commented May 25, 2023

@ra0x3 forc index kill uses SIGKILL, so it leaves the postgres process hanging.

We could switch to SIGTERM as a default and add a -9 flag for those cases when the users need it.

I'll re-check the Docker builds.

@ra0x3
Copy link
Contributor

ra0x3 commented May 25, 2023

@ra0x3 forc index kill uses SIGKILL, so it leaves the postgres process hanging.

We could switch to SIGTERM as a default and add a -9 flag for those cases when the users need it.

I'll re-check the Docker builds.

@lostman Yep, we can definitely handle the postgres killing separately :) Lmk how the Docker thing works for you. @deekerno Could you also test the docker parts of this when you get a minute? I've already verified the other parts are working 👌🏽

@lostman
Copy link
Contributor Author

lostman commented May 26, 2023

With vendored openssl:

docker compose up

[+] Running 1/1
 ! fuel-node Warning                                                                                  1.9s
[+] Building 979.9s (22/22) FINISHED
 => [internal] load .dockerignore                                                                     0.0s
 => => transferring context: 280B                                                                     0.0s
 => [internal] load build definition from Dockerfile.fuel-node                                        0.0s
 => => transferring dockerfile: 1.46kB                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                       1.9s
 => [internal] load metadata for docker.io/lukemathwalker/cargo-chef:latest-rust-1.69.0               1.6s
 => [run 1/8] FROM docker.io/library/ubuntu:22.04@sha256:dfd64a3b4296d8c9b62aa3309984f8620b98d87e474  0.0s
 => [chef 1/3] FROM docker.io/lukemathwalker/cargo-chef:latest-rust-1.69.0@sha256:4d72accbf1c4f6af2b  0.0s
 => [internal] load build context                                                                     1.3s
 => => transferring context: 52.04MB                                                                  1.3s
 => CACHED [chef 2/3] WORKDIR /build/                                                                 0.0s
 => [chef 3/3] RUN apt-get update -y &&     apt-get install -y --no-install-recommends     build-es  55.4s
 => [planner 1/2] COPY . .                                                                            0.4s
 => [planner 2/2] RUN cargo chef prepare --recipe-path recipe.json                                    0.4s
 => [builder 1/3] COPY --from=planner /build/recipe.json recipe.json                                  0.0s
 => [builder 2/3] COPY . .                                                                            0.2s
 => [builder 3/3] RUN cargo build --release -p fuel-node --all-targets                              920.3s
 => CACHED [run 2/8] WORKDIR /root/                                                                   0.0s
 => CACHED [run 3/8] RUN DEBIAN_FRONTEND=noninteractive apt-get update -y     && apt-get install -y   0.0s
 => [run 4/8] COPY --from=builder /build/target/release/fuel-node .                                   0.1s
 => [run 5/8] COPY --from=builder /build/target/release/fuel-node.d .                                 0.0s
 => [run 6/8] COPY --from=builder /build/packages/fuel-indexer-tests/test-chain-config.json .         0.0s
 => [run 7/8] COPY --from=builder /build/packages/fuel-indexer-tests/contracts/fuel-indexer-test/out  0.0s
 => [run 8/8] COPY --from=builder /build/examples/hello-world/contracts/greeting/out/debug/greeting.  0.0s
 => exporting to image                                                                                0.1s
 => => exporting layers                                                                               0.1s
 => => writing image sha256:1f10c10ae48731e59baaa8b829db88a124c88aff4d0b34536e8d1863f28b9f39          0.0s
 => => naming to docker.io/fuel-indexer/local:fuel-node                                               0.0s
[+] Running 3/3
 ✔ Container hello-world-postgres-1      Created                                                      0.2s
 ✔ Container hello-world-fuel-node-1     Created                                                      0.1s
 ✔ Container hello-world-fuel-indexer-1  Created                                                      0.1s
Attaching to hello-world-fuel-indexer-1, hello-world-fuel-node-1, hello-world-postgres-1
hello-world-postgres-1      | The files belonging to this database system will be owned by user "postgres".
hello-world-postgres-1      | This user must also own the server process.
hello-world-postgres-1      |
hello-world-postgres-1      | The database cluster will be initialized with locale "en_US.utf8".
hello-world-postgres-1      | The default database encoding has accordingly been set to "UTF8".
hello-world-postgres-1      | The default text search configuration will be set to "english".
hello-world-postgres-1      |
hello-world-postgres-1      | Data page checksums are disabled.
hello-world-postgres-1      |
hello-world-postgres-1      | fixing permissions on existing directory /var/lib/postgresql/data ... ok
hello-world-postgres-1      | creating subdirectories ... ok
hello-world-postgres-1      | selecting dynamic shared memory implementation ... posix
hello-world-postgres-1      | selecting default max_connections ... 100
hello-world-postgres-1      | selecting default shared_buffers ... 128MB
hello-world-postgres-1      | selecting default time zone ... Etc/UTC
hello-world-postgres-1      | creating configuration files ... ok
hello-world-postgres-1      | running bootstrap script ... ok
hello-world-fuel-node-1     | Contract deployed at: fuel18hchrf7f4hnpkl84sqf8k0sk8gcauzeemzwgweea8dgr7eachv4s86r9t9
hello-world-postgres-1      | performing post-bootstrap initialization ... ok
hello-world-postgres-1      | syncing data to disk ... ok
hello-world-postgres-1      |
hello-world-postgres-1      | initdb: warning: enabling "trust" authentication for local connections
hello-world-postgres-1      | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
hello-world-postgres-1      |
hello-world-postgres-1      | Success. You can now start the database server using:
hello-world-postgres-1      |
hello-world-postgres-1      |     pg_ctl -D /var/lib/postgresql/data -l logfile start
hello-world-postgres-1      |
hello-world-postgres-1      | waiting for server to start....2023-05-26 15:53:50.939 UTC [48] LOG:  starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
hello-world-postgres-1      | 2023-05-26 15:53:50.941 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
hello-world-postgres-1      | 2023-05-26 15:53:50.947 UTC [51] LOG:  database system was shut down at 2023-05-26 15:53:50 UTC
hello-world-postgres-1      | 2023-05-26 15:53:50.954 UTC [48] LOG:  database system is ready to accept connections
hello-world-postgres-1      |  done
hello-world-postgres-1      | server started
hello-world-postgres-1      |
hello-world-postgres-1      | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
hello-world-postgres-1      |
hello-world-postgres-1      | waiting for server to shut down....2023-05-26 15:53:51.136 UTC [48] LOG:  received fast shutdown request
hello-world-postgres-1      | 2023-05-26 15:53:51.141 UTC [48] LOG:  aborting any active transactions
hello-world-postgres-1      | 2023-05-26 15:53:51.149 UTC [48] LOG:  background worker "logical replication launcher" (PID 54) exited with exit code 1
hello-world-postgres-1      | 2023-05-26 15:53:51.179 UTC [49] LOG:  shutting down
hello-world-postgres-1      | 2023-05-26 15:53:51.191 UTC [49] LOG:  checkpoint starting: shutdown immediate
hello-world-postgres-1      | 2023-05-26 15:53:51.245 UTC [49] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.020 s, sync=0.003 s, total=0.066 s; sync files=2, longest=0.002 s, average=0.002 s; distance=0 kB, estimate=0 kB
hello-world-postgres-1      | 2023-05-26 15:53:51.287 UTC [48] LOG:  database system is shut down
hello-world-postgres-1      |  done
hello-world-postgres-1      | server stopped
hello-world-postgres-1      |
hello-world-postgres-1      | PostgreSQL init process complete; ready for start up.
hello-world-postgres-1      |
hello-world-postgres-1      | 2023-05-26 15:53:51.388 UTC [1] LOG:  starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
hello-world-postgres-1      | 2023-05-26 15:53:51.397 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
hello-world-postgres-1      | 2023-05-26 15:53:51.398 UTC [1] LOG:  listening on IPv6 address "::", port 5432
hello-world-postgres-1      | 2023-05-26 15:53:51.406 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
hello-world-postgres-1      | 2023-05-26 15:53:51.415 UTC [62] LOG:  database system was shut down at 2023-05-26 15:53:51 UTC
hello-world-postgres-1      | 2023-05-26 15:53:51.430 UTC [1] LOG:  database system is ready to accept connections
hello-world-fuel-indexer-1  | 2023-05-26T15:53:52.663957Z  INFO fuel_indexer::commands::run: 66: Configuration: IndexerConfig { verbose: false, local_fuel_node: false, indexer_net_config: false, fuel_node: FuelNodeConfig { host: "fuel-node", port: "4000" }, graphql_api: GraphQLConfig { host: "0.0.0.0", port: "29987", max_body_size: 5242880 }, database: PostgresConfig { user: "postgres", password: "XXXX", host: "postgres", port: "5432", database: "postgres", verbose: "false" }, metrics: false, stop_idle_indexers: false, run_migrations: true, authentication: AuthenticationConfig { enabled: false, strategy: None, jwt_secret: "XXXX", jwt_issuer: None, jwt_expiry: None } }
hello-world-fuel-indexer-1  | 2023-05-26T15:53:52.847831Z  INFO sqlx::postgres::notice: 157: identifier "graph_registry_graph_root_version_schema_name_schema_identifier_key" will be truncated to "graph_registry_graph_root_version_schema_name_schema_identifier"
hello-world-fuel-indexer-1  | 2023-05-26T15:53:52.895483Z  INFO fuel_indexer::commands::run: 92: ✨ ✨ GraphQL Playground at: http://localhost:29987/api/playground/:namespace/:identifier

Copy link
Contributor

@deekerno deekerno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker examples run correctly without --embedded-database, but not with the flag. @lostman, did you happen to run the Docker examples with the postgres container commented out? I may be missing something here.

packages/fuel-indexer/src/commands/run.rs Outdated Show resolved Hide resolved
@lostman lostman linked an issue Jun 1, 2023 that may be closed by this pull request
@lostman
Copy link
Contributor Author

lostman commented Jun 5, 2023

Docker examples run correctly without --embedded-database, but not with the flag. @lostman, did you happen to run the Docker examples with the postgres container commented out? I may be missing something here.

What is the error you're encountering?

deekerno
deekerno previously approved these changes Jun 5, 2023
@lostman lostman requested review from ra0x3 and removed request for 0xmovses June 6, 2023 10:39
@lostman lostman requested a review from a team as a code owner June 6, 2023 10:45
@deekerno
Copy link
Contributor

deekerno commented Jun 6, 2023

Docker examples run correctly without --embedded-database, but not with the flag. @lostman, did you happen to run the Docker examples with the postgres container commented out? I may be missing something here.

What is the error you're encountering?

After speaking about it with @lostman, I totally forgot that the fix itself would not be a part of the Docker compose workflow because it hasn't been included as part of an image yet. 😅 so I'm fine with this. Re-reviewing now.

Copy link
Contributor

@ra0x3 ra0x3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello-world is working via docker now ✅

@ra0x3 ra0x3 merged commit a420b1e into master Jun 6, 2023
@ra0x3 ra0x3 deleted the maciej/813-fuel-indexer-embedded-database branch June 6, 2023 15:54
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants