Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection refused running control-plane-lite via docker-compose #2894

Closed
mkraft opened this issue Jan 21, 2023 · 3 comments
Closed

Connection refused running control-plane-lite via docker-compose #2894

mkraft opened this issue Jan 21, 2023 · 3 comments

Comments

@mkraft
Copy link

mkraft commented Jan 21, 2023

I'm getting "connection refused" errors following the control-plane-lite Docker setup steps and any guidance would be much appreciated please.

rudder-docker.yml

version: "3.7"

services:

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_USER=rudder
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=jobsdb
    ports:
      - "6432:5432"

  backend:
    depends_on:
      - db
      - metrics-exporter
      - d-transformer
    image: rudderlabs/rudder-server:latest # latest version that matches 0.*.*
    entrypoint: sh -c '/wait-for db:5432 -- /rudder-server'
    ports:
      - "8080:8080"
    environment:
      - JOBS_DB_HOST=db
      - JOBS_DB_USER=rudder
      - JOBS_DB_PORT=5432
      - JOBS_DB_DB_NAME=jobsdb
      - JOBS_DB_PASSWORD=password
      - DEST_TRANSFORM_URL=http://d-transformer:9090
      - CONFIG_BACKEND_URL=https://api.rudderstack.com
      - WORKSPACE_TOKEN=<your_workspace_token> # Must be written without quotes e.g. WORKSPACE_TOKEN=20TokEO34NWitou2t3quO8teS7
      - STATSD_SERVER_URL=metrics-exporter:9125
      - RSERVER_BACKEND_CONFIG_CONFIG_FROM_FILE=true
      - RSERVER_BACKEND_CONFIG_CONFIG_JSONPATH=/etc/rudderstack/workspaceConfig.json
    volumes:
      - /Users/astrophage/Desktop/rudderindocker/workspaceConfig.json:/etc/rudderstack/workspaceConfig.json

  d-transformer:
    depends_on:
      - metrics-exporter
    image: rudderlabs/rudder-transformer:latest
    ports:
      - "9090:9090"
    environment:
      - STATSD_SERVER_HOST=metrics-exporter
      - STATSD_SERVER_PORT="9125"
  
  minio:
    image: minio/minio
    ports:
      - "9000:9000"
    command: server /data
  
  metrics-exporter:
    image: prom/statsd-exporter:v0.22.4
    ports:
      - "9102:9102"

workspaceConfig.json

{
    "source": {
        "config": {},
        "id": "1im8yfXQsocRBGcQLXAaq5M8dYV",
        "name": "test-JS",
        "writeKey": "1im8yk2sz8oyHVCGVqViSNfKVDF",
        "enabled": true,
        "sourceDefinitionId": "1TW48i2bIzEl1HPf825cEznfIM8",
        "deleted": false,
        "createdAt": "Mon Oct 12 2020 16:51:54 GMT+0530 (India Standard Time)",
        "updatedAt": "Mon Oct 12 2020 16:51:54 GMT+0530 (India Standard Time)",
        "sourceDefinition": {
            "id": "1TW48i2bIzEl1HPf825cEznfIM8",
            "name": "JavaScript",
            "displayName": "JavaScript",
            "category": null,
            "createdAt": "2019-11-12T12:39:19.885Z",
            "updatedAt": "2020-06-18T11:54:06.114Z"
        }
    },
    "metadata": {
        "version": "1.0.2"
    }
}

starting up with

docker-compose -f rudder-docker.yml up

logs

[+] Running 5/0
 ⠿ Container rudderindocker-minio-1             Created                                                                                                                                                                                                                        0.0s
 ⠿ Container rudderindocker-db-1                Created                                                                                                                                                                                                                        0.0s
 ⠿ Container rudderindocker-metrics-exporter-1  Created                                                                                                                                                                                                                        0.0s
 ⠿ Container rudderindocker-d-transformer-1     Created                                                                                                                                                                                                                        0.0s
 ⠿ Container rudderindocker-backend-1           Created                                                                                                                                                                                                                        0.0s
Attaching to rudderindocker-backend-1, rudderindocker-d-transformer-1, rudderindocker-db-1, rudderindocker-metrics-exporter-1, rudderindocker-minio-1
rudderindocker-metrics-exporter-1  | level=info ts=2023-01-21T11:31:44.670Z caller=main.go:291 msg="Starting StatsD -> Prometheus Exporter" version="(version=0.22.4, branch=HEAD, revision=7e2fe6c2e61552062f03c6a5dddc219512456832)"
rudderindocker-metrics-exporter-1  | level=info ts=2023-01-21T11:31:44.670Z caller=main.go:292 msg="Build context" context="(go=go1.17.3, user=root@f597e9e3e4dd, date=20211126-15:23:50)"
rudderindocker-metrics-exporter-1  | level=info ts=2023-01-21T11:31:44.670Z caller=main.go:341 msg="Accepting StatsD Traffic" udp=:9125 tcp=:9125 unixgram=
rudderindocker-metrics-exporter-1  | level=info ts=2023-01-21T11:31:44.670Z caller=main.go:342 msg="Accepting Prometheus Requests" addr=:9102
rudderindocker-db-1                |
rudderindocker-db-1                | PostgreSQL Database directory appears to contain a database; Skipping initialization
rudderindocker-db-1                |
rudderindocker-db-1                | 2023-01-21 11:31:44.697 UTC [1] LOG:  starting PostgreSQL 15.1 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
rudderindocker-db-1                | 2023-01-21 11:31:44.697 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
rudderindocker-db-1                | 2023-01-21 11:31:44.697 UTC [1] LOG:  listening on IPv6 address "::", port 5432
rudderindocker-db-1                | 2023-01-21 11:31:44.698 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
rudderindocker-db-1                | 2023-01-21 11:31:44.704 UTC [24] LOG:  database system was shut down at 2023-01-21 11:30:53 UTC
rudderindocker-db-1                | 2023-01-21 11:31:44.708 UTC [1] LOG:  database system is ready to accept connections
rudderindocker-minio-1             | WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
rudderindocker-minio-1             | MinIO Object Storage Server
rudderindocker-minio-1             | Copyright: 2015-2023 MinIO, Inc.
rudderindocker-minio-1             | License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
rudderindocker-minio-1             | Version: RELEASE.2023-01-20T02-05-44Z (go1.19.4 linux/arm64)
rudderindocker-minio-1             |
rudderindocker-minio-1             | Status:         1 Online, 0 Offline.
rudderindocker-minio-1             | API: http://172.21.0.2:9000  http://127.0.0.1:9000
rudderindocker-minio-1             | Console: http://172.21.0.2:40093 http://127.0.0.1:40093
rudderindocker-minio-1             |
rudderindocker-minio-1             | Documentation: https://min.io/docs/minio/linux/index.html
rudderindocker-minio-1             | Warning: The standard parity is set to 0. This can lead to data loss.
rudderindocker-backend-1           | 2023/01/21 11:31:45 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
rudderindocker-backend-1           | INFO: No .env file found.
rudderindocker-backend-1           | [Config] :: Failed to parse config file from path "./config/config.yaml", using default values: open ./config/config.yaml: no such file or directory
rudderindocker-backend-1           | 2023-01-21T11:31:45.630Z	WARN	event-schema	event-schema/event_schema.go:202	[EventSchemas] You are using default password. Please change it by setting env variable RUDDER_ADMIN_PASSWORD
rudderindocker-backend-1           | 2023-01-21T11:31:45.638Z	INFO	app	app/app.go:55	Open source version of rudder-server
rudderindocker-backend-1           | 2023-01-21T11:31:45.663Z	ERROR	processor.transformer	transformer/transformer.go:177	Unable to make a transfomer build version call with error : Get "http://d-transformer:9090/transformerBuildVersion": dial tcp 172.21.0.5:9090: connect: connection refused
rudderindocker-backend-1           | 2023-01-21T11:31:45.667Z	ERROR	utils.misc	misc/misc.go:1165	Unable to make a warehouse databricks build version call with error : Get "http://localhost:8082/databricksVersion": dial tcp 127.0.0.1:8082: connect: connection refused
rudderindocker-backend-1           | 2023-01-21T11:31:45.748Z	INFO	runner	runner/runner.go:411	warehousemode embedded
rudderindocker-backend-1           | 2023-01-21T11:31:45.752Z	INFO	backend-config	backend-config/single_workspace.go:127	Read workspace config from JSON file
rudderindocker-backend-1           | 2023-01-21T11:31:45.753Z	INFO	backend-config	backend-config/backend-config.go:213	Workspace Config changed
rudderindocker-backend-1           | 2023-01-21T11:31:45.756Z	INFO	db.recovery	db/embeddedRecovery.go:42	Starting in normal mode
rudderindocker-db-1                | 2023-01-21 11:31:45.779 UTC [30] FATAL:  terminating connection due to administrator command
rudderindocker-db-1                | 2023-01-21 11:31:45.779 UTC [29] FATAL:  terminating connection due to administrator command
rudderindocker-backend-1           | 2023-01-21T11:31:45.782Z	WARN	validators.envValidator	validators/envValidator.go:181	Terminated 2 dangling connection(s)
rudderindocker-backend-1           | 2023-01-21T11:31:45.783Z	WARN	validators.envValidator	validators/envValidator.go:183	dangling connection #1: {pid:29 queryStart:0xc00051d240 waitEventType:Client waitEvent:ClientRead state:idle query:SELECT pg_advisory_unlock($1) terminated:true}
rudderindocker-backend-1           | 2023-01-21T11:31:45.783Z	WARN	validators.envValidator	validators/envValidator.go:183	dangling connection #2: {pid:30 queryStart:0xc00051d3e0 waitEventType:Client waitEvent:ClientRead state:idle query:INSERT INTO config_cache (key, config) VALUES ($1, $2)
rudderindocker-backend-1           | 		on conflict (key)
rudderindocker-backend-1           | 		do update set
rudderindocker-backend-1           | 		config = $2,
rudderindocker-backend-1           | 		updated_at = NOW() terminated:true}
rudderindocker-backend-1           | 2023-01-21T11:31:45.869Z	INFO	runner	runner/runner.go:411	warehousemode embedded
rudderindocker-backend-1           | 2023-01-21T11:31:45.869Z	INFO	admin	profiler/profiler.go:82	Starting server on port 7777
rudderindocker-backend-1           | 2023-01-21T11:31:45.869Z	INFO	apphandlers.EMBEDDED	apphandlers/embeddedAppHandler.go:91	Embedded mode: Starting Rudder Core
rudderindocker-backend-1           | 2023-01-21T11:31:45.874Z	INFO	warehouse	warehouse/warehouse.go:2040	WH: Starting Warehouse service...
rudderindocker-backend-1           | 2023-01-21T11:31:45.875Z	INFO	warehouse.pgnotifier	pgnotifier/pgnotifier.go:128	PgNotifier: Initializing PgNotifier...
rudderindocker-backend-1           | 2023-01-21T11:31:45.875Z	INFO	warehouse.pgnotifier	pgnotifier/pgnotifier.go:621	PgNotifier: Creating Job Queue Tables
rudderindocker-backend-1           | 2023-01-21T11:31:45.893Z	INFO	utils.misc	misc/misc.go:412	RUDDER_TMPDIR not found, falling back to /tmp
rudderindocker-backend-1           |
rudderindocker-backend-1           | 2023-01-21T11:31:45.893Z	INFO	admin	admin/admin.go:272	Serving on admin interface @ /tmp/rudder-server.sock
rudderindocker-backend-1           | 2023-01-21T11:31:45.908Z	INFO	jobsdb.readonly-gw	jobsdb/readonly_jobsdb.go:107	Readonly user connected to gw DB
rudderindocker-backend-1           | 2023-01-21T11:31:45.932Z	INFO	warehouse	warehouse/warehouse.go:2108	WH: Starting warehouse slave...
rudderindocker-backend-1           | 2023-01-21T11:31:45.935Z	INFO	jobsdb.readonly-rt	jobsdb/readonly_jobsdb.go:107	Readonly user connected to rt DB
rudderindocker-backend-1           | 2023-01-21T11:31:45.936Z	INFO	warehouse	warehouse/warehouse.go:2115	[WH]: Starting warehouse master...
rudderindocker-backend-1           | 2023-01-21T11:31:45.939Z	INFO	warehouse.pgnotifier	pgnotifier/pgnotifier.go:172	PgNotifier: Deleting all jobs for workspace: ::77bf0b886fd262a44b244144a05fb412
rudderindocker-backend-1           | 2023-01-21T11:31:45.954Z	INFO	warehouse	warehouse/warehouse.go:1921	WH: Warehouse master service waiting for BackendConfig before starting on 8082
rudderindocker-backend-1           | 2023-01-21T11:31:45.961Z	INFO	warehouse	warehouse/warehouse.go:1944	WH: Starting warehouse master service in 8082
rudderindocker-backend-1           | 2023-01-21T11:31:45.962Z	INFO	asyncjob	jobs/runner.go:120	[WH-Jobs]: Initializing async job runner
rudderindocker-backend-1           | 2023-01-21T11:31:45.962Z	INFO	asyncjob	jobs/runner.go:152	[WH-Jobs]: Cleaning up the zombie asyncjobs
rudderindocker-backend-1           | 2023-01-21T11:31:45.964Z	INFO	jobsdb.readonly-batch_rt	jobsdb/readonly_jobsdb.go:107	Readonly user connected to batch_rt DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.005Z	INFO	asyncjob	jobs/runner.go:129	[WH-Jobs]: successfully cleanup async table with error
rudderindocker-backend-1           | 2023-01-21T11:31:46.005Z	INFO	asyncjob	jobs/runner.go:171	[WH-Jobs]: Starting async job runner
rudderindocker-backend-1           | 2023-01-21T11:31:46.010Z	INFO	jobsdb.readonly-proc_error	jobsdb/readonly_jobsdb.go:107	Readonly user connected to proc_error DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.010Z	INFO	apphandlers.EMBEDDED	apphandlers/embeddedAppHandler.go:104	Configured deployment type: "DEDICATED"
rudderindocker-backend-1           | 2023-01-21T11:31:46.010Z	INFO	apphandlers.EMBEDDED	apphandlers/embeddedAppHandler.go:113	Clearing DB false
rudderindocker-db-1                | 2023-01-21 11:31:46.037 UTC [44] ERROR:  relation "rsources_failed_keys" already exists
rudderindocker-db-1                | 2023-01-21 11:31:46.037 UTC [44] STATEMENT:  create table "rsources_failed_keys" (
rudderindocker-db-1                | 			id BIGSERIAL,
rudderindocker-db-1                | 			db_name text not null default 'db',
rudderindocker-db-1                | 			job_run_id text not null,
rudderindocker-db-1                | 			task_run_id text not null,
rudderindocker-db-1                | 			source_id text not null,
rudderindocker-db-1                | 			destination_id text not null,
rudderindocker-db-1                | 			record_id jsonb not null,
rudderindocker-db-1                | 			ts timestamp not null default NOW(),
rudderindocker-db-1                | 			primary key (job_run_id, task_run_id, source_id, destination_id, db_name, id)
rudderindocker-db-1                | 		)
rudderindocker-db-1                | 2023-01-21 11:31:46.039 UTC [44] ERROR:  relation "rsources_stats" already exists
rudderindocker-db-1                | 2023-01-21 11:31:46.039 UTC [44] STATEMENT:  create table "rsources_stats" (
rudderindocker-db-1                | 			db_name text not null default 'db',
rudderindocker-db-1                | 			job_run_id text not null,
rudderindocker-db-1                | 			task_run_id text not null,
rudderindocker-db-1                | 			source_id text not null,
rudderindocker-db-1                | 			destination_id text not null,
rudderindocker-db-1                | 			in_count integer not null default 0,
rudderindocker-db-1                | 			out_count integer not null default 0,
rudderindocker-db-1                | 			failed_count integer not null default 0,
rudderindocker-db-1                | 			ts timestamp not null default NOW(),
rudderindocker-db-1                | 			primary key (db_name, job_run_id, task_run_id, source_id, destination_id)
rudderindocker-db-1                | 		)
rudderindocker-backend-1           | 2023-01-21T11:31:46.059Z	INFO	jobsdb.gw	jobsdb/jobsdb.go:858	Connected to gw DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.145Z	INFO	jobsdb.rt	jobsdb/jobsdb.go:858	Connected to rt DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.169Z	ERROR	runner	runner/runner.go:258	error sending server features: non retriable: unexpected status code 404: Not Found
rudderindocker-backend-1           | 2023-01-21T11:31:46.248Z	INFO	jobsdb.batch_rt	jobsdb/jobsdb.go:858	Connected to batch_rt DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.352Z	INFO	jobsdb.proc_error	jobsdb/jobsdb.go:858	Connected to proc_error DB
rudderindocker-d-transformer-1     |
rudderindocker-d-transformer-1     | > [email protected] start /home/node/app
rudderindocker-d-transformer-1     | > node src/index.js
rudderindocker-d-transformer-1     |
rudderindocker-backend-1           | 2023-01-21T11:31:46.439Z	INFO	apphandlers.EMBEDDED	apphandlers/embeddedAppHandler.go:194	using Static Cluster Manager
rudderindocker-backend-1           | 2023-01-21T11:31:46.459Z	INFO	jobsdb.gw	jobsdb/jobsdb.go:858	Connected to gw DB
rudderindocker-backend-1           | 2023-01-21T11:31:46.554Z	INFO	enterprise.suppress-user	suppress-user/factory.go:30	Suppress User feature is enterprise only
rudderindocker-backend-1           | 2023-01-21T11:31:46.560Z	INFO	gateway	gateway/gateway.go:1244	AdminHandler waiting for BackendConfig before starting on 8089
rudderindocker-backend-1           | 2023-01-21T11:31:46.561Z	INFO	gateway	gateway/gateway.go:1170	WebHandler waiting for BackendConfig before starting on 8080
rudderindocker-backend-1           | 2023-01-21T11:31:46.561Z	INFO	gateway	gateway/gateway.go:1172	WebHandler Starting on 8080
rudderindocker-backend-1           | 2023-01-21T11:31:46.561Z	INFO	gateway	gateway/gateway.go:1246	AdminHandler starting on 8089
rudderindocker-backend-1           | 2023-01-21T11:31:46.562Z	WARN	rsources	rsources/handler.go:419	shared database is not configured, skipping logical replication monitoring
rudderindocker-backend-1           | 2023-01-21T11:31:46.563Z	INFO	cluster	cluster/dynamic.go:110	Got trigger to change the mode, new mode: NORMAL, old mode: DEGRADED
rudderindocker-backend-1           | 2023-01-21T11:31:46.564Z	INFO	cluster	cluster/dynamic.go:238	Transiting the server from DegradedMode to NormalMode
rudderindocker-backend-1           | 2023-01-21T11:31:46.564Z	INFO	cluster	cluster/dynamic.go:150	Starting the server
rudderindocker-backend-1           | 2023-01-21T11:31:46.575Z	INFO	jobsdb.proc_error	jobsdb/backup.go:42	BackupDS loop is running
rudderindocker-backend-1           | 2023-01-21T11:31:46.585Z	INFO	jobsdb.gw	jobsdb/backup.go:42	BackupDS loop is running
rudderindocker-backend-1           | 2023-01-21T11:31:46.595Z	INFO	jobsdb.rt	jobsdb/backup.go:42	BackupDS loop is running
rudderindocker-backend-1           | 2023-01-21T11:31:46.605Z	INFO	jobsdb.batch_rt	jobsdb/backup.go:42	BackupDS loop is running
rudderindocker-backend-1           | 2023-01-21T11:31:46.612Z	ERROR	processor	processor/processor.go:556	error sending request - %sGet "http://d-transformer:9090/features": dial tcp 172.21.0.5:9090: connect: connection refused
rudderindocker-backend-1           | 2023-01-21T11:31:46.614Z	INFO	processor	processor/processor.go:2402	Processor mainPipeline started, subJobSize=2000 pipelineBufferedItems=0
rudderindocker-backend-1           | 2023-01-21T11:31:46.616Z	INFO	processor.stash	stash/stash.go:296	Processor errors stash loop started
rudderindocker-backend-1           | 2023-01-21T11:31:46.814Z	ERROR	processor	processor/processor.go:556	error sending request - %sGet "http://d-transformer:9090/features": dial tcp 172.21.0.5:9090: connect: connection refused
...

The backend container has network access to the transformer as I'm able to do

docker exec -it rudderindocker-backend-1 bash -c 'curl http://d-transformer:9090/transformerBuildVersion'
19012023.101129%
@mkraft
Copy link
Author

mkraft commented Jan 30, 2023

@achettyiitr Do you happen to know if this is the best place to ask this question?

@atzoum
Copy link
Contributor

atzoum commented Feb 9, 2023

Hey @mkraft,
I wasn't able to reproduce your issue. Would it be possible to pull the latest versions of the docker images first, and then try again?

docker pull rudderlabs/rudder-transformer:latest 
docker pull rudderlabs/rudder-server:latest

@gitcommitshow
Copy link
Collaborator

As Control Plane Lite is deprecated now and it will fail with the latest rudder-server versions so I am going to close this issue for now. Likely the change needed will be at the Control Plane Lite side so there's no action for rudder-server team. If you plan to fix Control Plane Lite on your own and need support, feel free to ask in slack community.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mkraft @atzoum @gitcommitshow