Skip to content

Commit

Permalink
bind SFTP on an ephemeral port of the Docker host
Browse files Browse the repository at this point in the history
Trying to bind 22 can be problematic; for one thing, it's tripping up our
CI/CD. Note that the containers address the original port number when speaking
to each other directly.
  • Loading branch information
barend-xebia committed Dec 11, 2024
1 parent f6642bb commit a40014a
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 3 deletions.
4 changes: 2 additions & 2 deletions envs/sftp-mysql-example/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ services:
ftp-server:
image: iowoi/sftp-server:latest
ports:
- 22:22
- "22" # Bind on an ephemeral port, user `docker ps` to find out where it landed.
env_file:
- sftp.env

Expand All @@ -30,6 +30,6 @@ services:
- ${ENVIRONMENT_FOLDER}/whirl.setup.d:${WHIRL_SETUP_FOLDER}/env.d/
- ${DAG_FOLDER}/whirl.setup.d:${WHIRL_SETUP_FOLDER}/dag.d/
- ${MOCK_DATA_FOLDER}:/mock-data
depends_on:
links:
- ftp-server
- mysql
4 changes: 3 additions & 1 deletion examples/sftp-mysql-example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,6 @@ The environment to be used is set in the `.whirl.env` in the DAG directory. In t

The DAG contains 2 tasks that use:
- The SFTPOperator to copy the file from the FTPserver to the local directory
- The Python operator using the MySQL Hook to load the csv data into the database.
- The Python operator using the MySQL Hook to load the csv data into the database.

The SFTP server is exposed to the host on an ephemeral port. Run `docker ps` to find out the local port number.

0 comments on commit a40014a

Please sign in to comment.