Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workflow which ensures that talawa Api app starts in docker #2759

Open
wants to merge 24 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
64bee20
added workflow
prayanshchh Dec 14, 2024
4fa5812
Merge branch 'develop' into api_fail_docker
prayanshchh Dec 14, 2024
6ca6131
syntax errors
prayanshchh Dec 14, 2024
57ae541
Merge branch 'api_fail_docker' of https://github.com/prayanshchh/pray…
prayanshchh Dec 14, 2024
cfec867
suggestion-1
prayanshchh Dec 14, 2024
1544502
indentation 1
prayanshchh Dec 14, 2024
0b6ef08
indentation 2
prayanshchh Dec 14, 2024
d8db396
Merge branch 'develop' into api_fail_docker
prayanshchh Dec 16, 2024
4adec1f
removing unions file
prayanshchh Dec 19, 2024
45b7fcc
Merge branch 'api_fail_docker' of https://github.com/prayanshchh/pray…
prayanshchh Dec 19, 2024
338a7b8
fixing tests
prayanshchh Dec 21, 2024
bb63fc2
Merge branch 'develop' into api_fail_docker
prayanshchh Dec 21, 2024
d540981
docker-compose issue
prayanshchh Dec 21, 2024
5c54d6a
Merge branch 'api_fail_docker' of https://github.com/prayanshchh/pray…
prayanshchh Dec 21, 2024
c942e2c
increasing timeout
prayanshchh Dec 22, 2024
fbbb7ea
test fix
prayanshchh Dec 22, 2024
798b4f7
using mongosh
prayanshchh Dec 22, 2024
05f9c97
idenation fix
prayanshchh Dec 22, 2024
006092e
pwd variable fix, redi-stack-server fix
prayanshchh Dec 23, 2024
1c57fba
health-check-api
prayanshchh Dec 23, 2024
08c8d16
api fix
prayanshchh Dec 23, 2024
548e83b
Merge branch 'develop' into api_fail_docker
prayanshchh Dec 23, 2024
a228c07
removing unnecessary files
prayanshchh Dec 23, 2024
4c7280c
Merge branch 'api_fail_docker' of https://github.com/prayanshchh/pray…
prayanshchh Dec 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -104,4 +104,7 @@ MINIO_DATA_DIR=

# this environment variable is for setting the environment variable for Image Upload size

IMAGE_SIZE_LIMIT_KB=3000
IMAGE_SIZE_LIMIT_KB=3000

# This environment variable is for setting the working directory when Docker is used
PWD=.
129 changes: 128 additions & 1 deletion .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,134 @@ jobs:
with:
path: './coverage/lcov.info'
min_coverage: 95.0


Docker-Check:
needs: Test-Application
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22.x'

- name: Cache Node.js dependencies
uses: actions/cache@v4
with:
path: |
~/.npm
node_modules
key: ${{ runner.os }}-docker-check-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-docker-check-

- name: Install Docker Compose
run: |
sudo apt-get update
sudo apt-get install -y docker-compose

- name: Check if Talawa API starts in Docker
run: |
# Ensure no containers are running
docker-compose -f docker-compose.dev.yaml down -v || true

# Verify docker-compose file exists
if [ ! -f "docker-compose.dev.yaml" ]; then
echo "Error: docker-compose.dev.yaml not found"
docker-compose -f docker-compose.dev.yaml logs
exit 1
fi

# Start containers
if ! timeout 300 docker-compose -f docker-compose.dev.yaml up -d --build; then
echo "Failed to start containers"
docker-compose -f docker-compose.dev.yaml logs
exit 1
fi
Comment on lines +331 to +343
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve error handling for docker-compose file check

The current implementation logs docker-compose logs even when the file doesn't exist, which would fail.

           if [ ! -f "docker-compose.dev.yaml" ]; then
             echo "Error: docker-compose.dev.yaml not found"
-            docker-compose -f docker-compose.dev.yaml logs
             exit 1
           fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Verify docker-compose file exists
if [ ! -f "docker-compose.dev.yaml" ]; then
echo "Error: docker-compose.dev.yaml not found"
docker-compose -f docker-compose.dev.yaml logs
exit 1
fi
# Start containers
if ! timeout 300 docker-compose -f docker-compose.dev.yaml up -d --build; then
echo "Failed to start containers"
docker-compose -f docker-compose.dev.yaml logs
exit 1
fi
# Verify docker-compose file exists
if [ ! -f "docker-compose.dev.yaml" ]; then
echo "Error: docker-compose.dev.yaml not found"
exit 1
fi
# Start containers
if ! timeout 300 docker-compose -f docker-compose.dev.yaml up -d --build; then
echo "Failed to start containers"
docker-compose -f docker-compose.dev.yaml logs
exit 1
fi


# Wait for MongoDB and Redis to be ready
echo "Waiting for MongoDB..."
timeout=30
until docker-compose -f docker-compose.dev.yaml exec -T mongodb mongosh --eval "db.runCommand({ ping: 1 }).ok">/dev/null 2>&1 || [ $timeout -eq 0 ]; do
echo "Waiting for MongoDB to be ready..."
sleep 1
((timeout--))
done
Comment on lines +348 to +352
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix MongoDB health check indentation

The indentation in the MongoDB health check loop is inconsistent.

 until docker-compose -f docker-compose.dev.yaml exec -T mongodb mongosh --eval "db.runCommand({ ping: 1 }).ok">/dev/null 2>&1 || [ $timeout -eq 0 ]; do
-echo "Waiting for MongoDB to be ready..."
-sleep 1
- ((timeout--))
+  echo "Waiting for MongoDB to be ready..."
+  sleep 1
+  ((timeout--))
 done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
until docker-compose -f docker-compose.dev.yaml exec -T mongodb mongosh --eval "db.runCommand({ ping: 1 }).ok">/dev/null 2>&1 || [ $timeout -eq 0 ]; do
echo "Waiting for MongoDB to be ready..."
sleep 1
((timeout--))
done
until docker-compose -f docker-compose.dev.yaml exec -T mongodb mongosh --eval "db.runCommand({ ping: 1 }).ok">/dev/null 2>&1 || [ $timeout -eq 0 ]; do
echo "Waiting for MongoDB to be ready..."
sleep 1
((timeout--))
done

if [ $timeout -eq 0 ]; then
echo "Error: MongoDB failed to start within timeout"
echo "Fetching MongoDB logs..."
docker-compose -f docker-compose.dev.yaml logs mongodb
echo "Shutting down MongoDB..."
docker-compose -f docker-compose.dev.yaml down -v
exit 1

else
echo "MongoDB is ready!"
fi

echo "Waiting for Redis..."
timeout=30
until docker-compose -f docker-compose.dev.yaml exec -T redis-stack-server redis-cli ping >/dev/null 2>&1 || [ $timeout -eq 0 ]; do
sleep 1
((timeout--))
done
if [ $timeout -eq 0 ]; then
echo "Error: Redis failed to start within timeout"
echo "Fetching Redis logs..."
docker-compose -f docker-compose.dev.yaml logs redis-stack-server
echo "Shutting down Redis..."
docker-compose -f docker-compose.dev.yaml down -v
exit 1

else
echo "Redis is ready!"
fi

# Wait for TALAWA API to be healthy
timeout=60
until docker-compose -f docker-compose.dev.yaml exec -T talawa-api-dev curl -v -X OPTIONS "http://talawa-api-dev:4000/graphql" 2>&1 || [ $timeout -eq 0 ]; do
echo "Waiting for API to start... ($timeout seconds remaining)"
sleep 1
((timeout--))
done
Comment on lines +385 to +389
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve API health check reliability

The current API health check has a few issues:

  1. Using OPTIONS request might not accurately verify the GraphQL endpoint
  2. Internal container DNS resolution might be unreliable

Apply this diff to improve the health check:

-          until docker-compose -f docker-compose.dev.yaml exec -T talawa-api-dev curl -v -X OPTIONS "http://talawa-api-dev:4000/graphql" 2>&1 || [ $timeout -eq 0 ]; do
+          until docker-compose -f docker-compose.dev.yaml exec -T talawa-api-dev curl -sf -X POST "http://localhost:4000/graphql" \
+            -H 'Content-Type: application/json' \
+            -d '{"query":"{ __typename }"}' 2>&1 || [ $timeout -eq 0 ]; do

Committable suggestion skipped: line range outside the PR's diff.


if [ $timeout -eq 0 ]; then
echo "Error: API failed to start within timeout"
docker-compose -f docker-compose.dev.yaml logs
docker-compose -f docker-compose.dev.yaml down -v
exit 1
fi

echo "API started successfully"

# Ensure cleanup runs even if the script fails
cleanup() {
local exit_code=$?
echo "Cleaning up containers..."
if ! docker-compose -f docker-compose.dev.yaml down -v; then
echo "Warning: Failed to cleanup containers"
fi
exit $exit_code
}
trap cleanup EXIT
env:
HEALTH_CHECK_URL: http://localhost:4000
COMPOSE_PROJECT_NAME: pr-${{ github.event.pull_request.number }}
MONGO_DB_URL: mongodb://mongodb:27017/talawa-test-db
REDIS_HOST: redis-stack-server
REDIS_PORT: 6379
ACCESS_TOKEN_SECRET: ${{ secrets.GITHUB_TOKEN }}_${{ github.run_id }}_${{ github.run_number }}
REFRESH_TOKEN_SECRET: ${{ secrets.GITHUB_TOKEN }}_${{ github.run_id }}_${{ github.run_attempt }}
LAST_RESORT_SUPERADMIN_EMAIL: "[email protected]"
COLORIZE_LOGS: "true"
LOG_LEVEL: "info"
RECAPTCHA_SITE_KEY: ${{secrets.RECAPTCHA_SITE_KEY}}
RECAPTCHA_SECRET_KEY: ${{secrets.RECAPTCHA_SECRET_KEY}}
MAIL_USERNAME: ${{secrets.MAIL_USERNAME}}
MAIL_PASSWORD: ${{secrets.MAIL_PASSWORD}}

JSDocs:
if: ${{ github.actor != 'dependabot[bot]' }}
name: 'JSDocs comments and pipeline'
Expand Down
26 changes: 7 additions & 19 deletions docker-compose.dev.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "3.8"

services:
mongodb:
image: mongo:latest
Expand Down Expand Up @@ -37,6 +35,9 @@ services:
build:
context: .
dockerfile: Dockerfile.dev
container_name: talawa-api-dev
ports:
- "4000:4000"
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules
Expand All @@ -45,30 +46,17 @@ services:
- redis-stack-server
- minio
environment:
- MONGO_DB_URL=mongodb://mongodb:27017
- MONGO_DB_URL=mongodb://mongodb:27017/talawa-api
- REDIS_HOST=redis-stack-server
- REDIS_PORT=6379

caddy:
image: caddy/caddy:2.2.1-alpine
container_name: caddy-service
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- $PWD/site:/srv
- caddy_data:/data
- caddy_config:/config
networks:
- talawa-network

volumes:
mongodb-data:
redis-data:
caddy_data:
caddy_config:
minio-data:

networks:
talawa-network:
driver: bridge
driver: bridge
11 changes: 11 additions & 0 deletions setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1015,25 +1015,36 @@ async function main(): Promise<void> {
const REDIS_PASSWORD = "";
const MINIO_ENDPOINT = "http://minio:9000";

const { pwdVariable } = await inquirer.prompt({
type: "input",
name: "pwdVariable",
message:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this added? It should have been fixed in a previously merged PR. Please merge your code with the latest upstream/postgres-develop to be certain.

Copy link
Author

@prayanshchh prayanshchh Dec 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sir my branch is working against develop branch, not develop-postgres, merging develop-postgres would result in a lot of conflicts, I tried to pull from develop but I am already up to date with the upstream/develop branch

"Please enter the value for PWD (working directory for Docker setup):",
default: ".",
});

const config = dotenv.parse(fs.readFileSync(".env"));

config.MONGO_DB_URL = DB_URL;
config.REDIS_HOST = REDIS_HOST;
config.REDIS_PORT = REDIS_PORT;
config.REDIS_PASSWORD = REDIS_PASSWORD;
config.MINIO_ENDPOINT = MINIO_ENDPOINT;
config.PWD = pwdVariable;

process.env.MONGO_DB_URL = DB_URL;
process.env.REDIS_HOST = REDIS_HOST;
process.env.REDIS_PORT = REDIS_PORT;
process.env.REDIS_PASSWORD = REDIS_PASSWORD;
process.env.MINIO_ENDPOINT = MINIO_ENDPOINT;
process.env.PWD = pwdVariable;

updateEnvVariable(config);
console.log(`Your MongoDB URL is:\n${process.env.MONGO_DB_URL}`);
console.log(`Your Redis host is:\n${process.env.REDIS_HOST}`);
console.log(`Your Redis port is:\n${process.env.REDIS_PORT}`);
console.log(`Your MinIO endpoint is:\n${process.env.MINIO_ENDPOINT}`);
console.log(`Your PWD value is:\n${process.env.PWD}`);
}

if (!isDockerInstallation) {
Expand Down
2 changes: 0 additions & 2 deletions tests/helpers/volunteers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,6 @@ export const createVolunteerAndActions = async (): Promise<
});

const today = new Date();
const yesterday = new Date(today);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please exclude this file. It is not relevant to the original issue. You have also changed its logic

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was included to pass the getVolunteerRanks test, this changed was merged in this pr

yesterday.setDate(today.getDate() - 1);
const twoWeeksAgo = new Date(today);
twoWeeksAgo.setDate(today.getDate() - 14);
const twoMonthsAgo = new Date(today);
Expand Down
Loading