Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: central-settlement tb integration #379

Draft
wants to merge 171 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
171 commits
Select commit Hold shift + click to select a range
c100b20
TigerBeetle required functions for settlement.
koekiebox Jul 3, 2022
476108a
TigerBeetle creating transfers for settlement.
koekiebox Jul 5, 2022
d4d9277
Install Tb into base settlement-ledger image.
koekiebox Jul 7, 2022
bfd70e3
Install Tb into base settlement-ledger image.
koekiebox Jul 11, 2022
0a9a48d
Update host to `mysql-int` from `localhost`
koekiebox Jul 11, 2022
127d3ef
Print out config contents.
koekiebox Jul 11, 2022
0ec5a8c
Remove out sanity check.
koekiebox Jul 11, 2022
a7ce5c0
Merge branch 'master' into feature/tigerbeetle-integration
koekiebox Jul 15, 2022
480c70e
Add wget for Docker files.
koekiebox Jul 15, 2022
1ffda3f
Add wget for Docker files.
koekiebox Jul 15, 2022
6e802e1
Merge branch 'master' into feature/tigerbeetle-integration
koekiebox Jul 28, 2022
07bd056
Bump to latest tiger-beetle. TigerBeetle node 0.10.0.
koekiebox Sep 1, 2022
a7ccce1
Bump to latest tiger-beetle. TigerBeetle node 0.10.0 with unit test.
koekiebox Sep 7, 2022
a462c06
Merge branch 'master' into feature/tigerbeetle-integration
koekiebox Sep 8, 2022
9913b7b
Updated unit testing for TB client.
koekiebox Sep 8, 2022
2a636fa
Updated unit testing for TB client unit testing.
koekiebox Sep 9, 2022
1bf3f29
Lint fixed with test cases.
koekiebox Sep 12, 2022
ca831d2
Simplified TB client. Better coverage.
koekiebox Sep 12, 2022
1b52f77
Reworked config.js
koekiebox Sep 12, 2022
a303d74
Test $DOCKER_ORG.
koekiebox Sep 12, 2022
95cec62
Test $DOCKER_ORG.
koekiebox Sep 12, 2022
3486712
Test start TB controller for integration.
koekiebox Sep 12, 2022
2dac241
Integration test for tigerbeetle.
koekiebox Sep 12, 2022
ba7de6a
Integration test for tigerbeetle.
koekiebox Sep 12, 2022
9d923e8
Descriptive objectives for integration tests.
koekiebox Sep 12, 2022
7c46b1c
Initial unit tests for central-settlement.
koekiebox Sep 13, 2022
8dd220b
Initial unit tests for central-settlement.
koekiebox Sep 13, 2022
2b42c8c
Initial unit tests for central-settlement.
koekiebox Sep 13, 2022
f0110a6
Initial unit tests for central-settlement.
koekiebox Sep 15, 2022
52a5aa9
Fix docker group issue.
koekiebox Sep 15, 2022
1ef70f7
Circle-CI fixes.
koekiebox Sep 15, 2022
7ccbbbd
Circle-CI fixes.
koekiebox Sep 15, 2022
3bb4e44
Circle-CI fixes.
koekiebox Sep 15, 2022
45e7137
Circle-CI fixes.
koekiebox Sep 15, 2022
5c869c6
Circle-CI fixes.
koekiebox Sep 15, 2022
c1446b2
Circle-CI fixes.
koekiebox Sep 15, 2022
e47ea1f
Circle-CI fixes.
koekiebox Sep 15, 2022
50bd7f1
Circle-CI fixes.
koekiebox Sep 20, 2022
fc4764f
Circle-CI fixes.
koekiebox Sep 20, 2022
ec41658
Circle-CI fixes.
koekiebox Sep 20, 2022
8617e9a
Circle-CI fixes.
koekiebox Sep 20, 2022
f7d5c2d
Circle-CI fixes.
koekiebox Sep 20, 2022
c0e9ed6
Circle-CI fixes.
koekiebox Sep 20, 2022
3295ff0
Circle-CI fixes.
koekiebox Sep 20, 2022
bbc2943
Circle-CI fixes.
koekiebox Sep 20, 2022
637c0db
Circle-CI fixes.
koekiebox Sep 20, 2022
722d634
Circle-CI fixes.
koekiebox Sep 20, 2022
f345049
Circle-CI fixes.
koekiebox Sep 23, 2022
38feb90
Circle-CI fixes.
koekiebox Sep 26, 2022
7ca8ee7
Circle-CI fixes.
koekiebox Sep 26, 2022
509f9e3
Circle-CI fixes.
koekiebox Sep 26, 2022
2381d92
Circle-CI fixes.
koekiebox Sep 26, 2022
e612a38
Circle-CI fixes.
koekiebox Sep 26, 2022
3b0ed5f
Circle-CI fixes.
koekiebox Sep 26, 2022
bb7aee8
Circle-CI fixes.
koekiebox Sep 26, 2022
bfea92d
Circle-CI fixes.
koekiebox Sep 26, 2022
34ff892
Circle-CI fixes.
koekiebox Sep 27, 2022
f709407
Circle-CI fixes.
koekiebox Sep 27, 2022
e854abd
Circle-CI fixes.
koekiebox Sep 28, 2022
2239812
Circle-CI fixes.
koekiebox Sep 28, 2022
7927c1f
Circle-CI fixes.
koekiebox Sep 28, 2022
ea3fa92
Circle-CI fixes.
koekiebox Sep 28, 2022
4765690
Circle-CI fixes.
koekiebox Oct 4, 2022
2f9e9ea
Circle-CI fixes.
koekiebox Oct 4, 2022
3b9259a
Circle-CI fixes.
koekiebox Oct 4, 2022
312f14c
Circle-CI fixes.
koekiebox Oct 4, 2022
4b85157
Circle-CI fixes.
koekiebox Oct 4, 2022
fbd0d4d
Circle-CI fixes.
koekiebox Oct 4, 2022
8b03296
Circle-CI fixes.
koekiebox Oct 4, 2022
e6874c0
Circle-CI fixes.
koekiebox Oct 4, 2022
995dcf9
Circle-CI fixes.
koekiebox Oct 4, 2022
aeafd03
Circle-CI fixes.
koekiebox Oct 4, 2022
f8b96a6
Circle-CI fixes.
koekiebox Oct 4, 2022
5680acc
Circle-CI fixes.
koekiebox Oct 4, 2022
0dbb458
Circle-CI fixes.
koekiebox Oct 4, 2022
d3ac266
Circle-CI fixes.
koekiebox Oct 4, 2022
e8f7d3f
Circle-CI fixes.
koekiebox Oct 4, 2022
76d4f83
Circle-CI fixes.
koekiebox Oct 4, 2022
47278de
Circle-CI fixes.
koekiebox Oct 4, 2022
aae0269
Circle-CI fixes.
koekiebox Oct 4, 2022
91c463d
Circle-CI fixes.
koekiebox Oct 4, 2022
01ba3ec
Circle-CI fixes.
koekiebox Oct 4, 2022
7d80510
Circle-CI fixes.
koekiebox Oct 4, 2022
918bafc
Circle-CI fixes.
koekiebox Oct 5, 2022
16f7391
Circle-CI fixes.
koekiebox Oct 5, 2022
7dbe166
Circle-CI fixes.
koekiebox Oct 5, 2022
47afe65
Circle-CI fixes.
koekiebox Oct 5, 2022
bb94123
Circle-CI fixes.
koekiebox Oct 5, 2022
dd1c147
Circle-CI fixes.
koekiebox Oct 5, 2022
0597694
Circle-CI fixes.
koekiebox Oct 5, 2022
69445ec
Circle-CI fixes.
koekiebox Oct 7, 2022
c18b142
Circle-CI fixes.
koekiebox Oct 7, 2022
c31e48d
Circle-CI fixes.
koekiebox Oct 11, 2022
a973667
Circle-CI fixes.
koekiebox Oct 11, 2022
dc4ab98
Circle-CI fixes.
koekiebox Oct 11, 2022
02cb3a8
Circle-CI fixes.
koekiebox Oct 11, 2022
5cca588
Circle-CI fixes.
koekiebox Oct 11, 2022
77bd1ee
Circle-CI fixes.
koekiebox Oct 11, 2022
725fe2d
Circle-CI fixes.
koekiebox Oct 11, 2022
4223893
Circle-CI fixes.
koekiebox Oct 11, 2022
18a2fca
Circle-CI fixes.
koekiebox Oct 11, 2022
fd0d91d
Circle-CI fixes.
koekiebox Oct 11, 2022
0b9867a
Circle-CI fixes.
koekiebox Oct 11, 2022
f2a6d08
Circle-CI fixes.
koekiebox Oct 11, 2022
4360dd1
Circle-CI fixes.
koekiebox Oct 11, 2022
d863b8c
Circle-CI fixes.
koekiebox Oct 11, 2022
bb01d40
Circle-CI fixes.
koekiebox Oct 11, 2022
c3bf38e
Circle-CI fixes.
koekiebox Oct 11, 2022
50c42b0
Circle-CI fixes.
koekiebox Oct 11, 2022
20cd87c
Circle-CI fixes.
koekiebox Oct 12, 2022
b57be07
Circle-CI fixes.
koekiebox Oct 12, 2022
46886e5
Circle-CI fixes.
koekiebox Oct 12, 2022
d03b61a
Circle-CI fixes.
koekiebox Oct 12, 2022
8490353
Circle-CI fixes.
koekiebox Oct 12, 2022
c7cd843
Circle-CI fixes.
koekiebox Oct 12, 2022
f65dfe7
Circle-CI fixes.
koekiebox Oct 12, 2022
ca4af33
Circle-CI fixes.
koekiebox Oct 12, 2022
37f0656
Circle-CI fixes.
koekiebox Oct 12, 2022
ecbb815
Circle-CI fixes.
koekiebox Oct 12, 2022
b5c42d7
Circle-CI fixes.
koekiebox Oct 12, 2022
e6e65fd
Circle-CI fixes.
koekiebox Oct 13, 2022
a24cc15
Circle-CI fixes.
koekiebox Oct 13, 2022
8436c1a
Circle-CI fixes.
koekiebox Oct 13, 2022
9879d44
Update description. Remove unnecessary logs.
koekiebox Oct 17, 2022
01c50a0
Update description. Remove unnecessary logs.
koekiebox Oct 17, 2022
a8774fe
Updated printout.
koekiebox Oct 17, 2022
d7424dc
Updated printout.
koekiebox Oct 17, 2022
d603e24
Updated printout.
koekiebox Oct 17, 2022
b6266aa
Updated printout.
koekiebox Oct 17, 2022
54afccf
Updated printout.
koekiebox Oct 17, 2022
68fa74b
Updated printout.
koekiebox Oct 17, 2022
149ef93
Updated printout.
koekiebox Oct 17, 2022
c81e486
Updated printout.
koekiebox Oct 17, 2022
c34f528
Updated printout.
koekiebox Oct 17, 2022
124a085
Updated printout.
koekiebox Oct 17, 2022
c904aa6
Updated printout.
koekiebox Oct 17, 2022
a28eebf
Updated printout.
koekiebox Oct 17, 2022
13113db
Updated printout.
koekiebox Oct 17, 2022
019754b
Updated printout.
koekiebox Oct 17, 2022
81df7c3
Updated printout.
koekiebox Oct 17, 2022
9b8fb9f
Updated printout.
koekiebox Oct 17, 2022
c699c63
Updated printout.
koekiebox Oct 17, 2022
68cc57d
Updated printout.
koekiebox Oct 17, 2022
2f78e48
Updated printout.
koekiebox Oct 17, 2022
4b45563
Updated printout.
koekiebox Oct 17, 2022
52fdc3f
Updated printout.
koekiebox Oct 18, 2022
5ead0b1
Updated printout.
koekiebox Oct 18, 2022
939a2a4
Updated printout.
koekiebox Oct 18, 2022
671cc86
Updated printout.
koekiebox Oct 18, 2022
f5289ea
Updated printout.
koekiebox Oct 18, 2022
80da2a2
Updated printout.
koekiebox Oct 18, 2022
292c874
Updated printout.
koekiebox Oct 18, 2022
30784de
Updated printout.
koekiebox Oct 18, 2022
366b6b7
Updated printout.
koekiebox Oct 18, 2022
03b357b
Updated printout.
koekiebox Oct 18, 2022
5c2e7c1
Updated printout.
koekiebox Oct 18, 2022
1946e79
Updated printout.
koekiebox Oct 18, 2022
044e14a
Updated printout.
koekiebox Oct 18, 2022
d0fda1e
Updated printout.
koekiebox Oct 18, 2022
dd15941
Updated printout.
koekiebox Oct 18, 2022
a4665fc
Updated printout.
koekiebox Oct 18, 2022
fc11234
Updated printout.
koekiebox Oct 18, 2022
299d97b
Updated printout.
koekiebox Oct 18, 2022
13cfb6b
Updated printout.
koekiebox Oct 18, 2022
33d36d4
Updated printout.
koekiebox Oct 18, 2022
1ecc5b5
Updated printout.
koekiebox Oct 20, 2022
6cff565
Updated printout.
koekiebox Oct 20, 2022
3e30f3d
Updated printout.
koekiebox Oct 25, 2022
f0553e9
Updated printout.
koekiebox Oct 25, 2022
979239d
Updated printout.
koekiebox Oct 25, 2022
8f01134
Updated printout.
koekiebox Oct 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ defaults_Dependencies: &defaults_Dependencies |
apk --no-cache add ca-certificates
apk --no-cache add curl
apk --no-cache add openssh-client
apk add --no-cache -t build-dependencies make gcc g++ python3 libtool autoconf automake jq
apk add --no-cache -t build-dependencies make gcc g++ python3 libtool autoconf automake jq wget
apk add --no-cache -t openssl ncurses coreutils libgcc linux-headers grep util-linux binutils findutils
npm config set unsafe-perm true
npm install -g node-gyp
Expand Down Expand Up @@ -254,10 +254,10 @@ jobs:
name: Build Docker local image
command: |
echo "Building Docker image: local"
docker build -t $DOCKER_ORG/$CIRCLE_PROJECT_REPONAME:local .
docker build -t mojaloop/$CIRCLE_PROJECT_REPONAME:local .
- run:
name: Save docker image to workspace
command: docker save -o /tmp/docker-image-local.tar $DOCKER_ORG/$CIRCLE_PROJECT_REPONAME:local
command: docker save -o /tmp/docker-image-local.tar mojaloop/$CIRCLE_PROJECT_REPONAME:local
- persist_to_workspace:
root: /tmp
paths:
Expand Down Expand Up @@ -326,7 +326,7 @@ jobs:
apk --no-cache add ca-certificates
apk --no-cache add curl
apk --no-cache add openssh-client
apk add --no-cache -t build-dependencies make gcc g++ python3 libtool autoconf automake
apk add --no-cache -t build-dependencies make gcc g++ python3 libtool autoconf automake wget
npm config set unsafe-perm true
npm install -g node-gyp
- run:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,6 @@ junit.xml
# Ignore DevSpace configs, cache and log folder
.devspace/
devspace*

# TigerBeetle
tbdata/
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ USER root

WORKDIR /opt/app/

RUN apk add --no-cache -t build-dependencies git make gcc g++ python3 libtool autoconf automake \
RUN apk add --no-cache -t build-dependencies git make gcc g++ python3 libtool autoconf automake wget \
&& cd $(npm root -g)/npm \
&& npm config set unsafe-perm true \
&& npm install -g node-gyp
Expand Down
35 changes: 27 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ TBA

## Configuration

Currently the only configuration, necessary is kept [here](./config/default.json)
Currently, the only configuration necessary is kept [here](./config/default.json)

## Environmental variables

Currently all is set into the config.
Currently, all is set into the config.

## API

Expand All @@ -51,30 +51,50 @@ Includes unit tests at the moment. Functional and integration are outstanding.

Running the tests:

npm run test:all
```bash
npm run test:all
```

Tests include code coverage via istanbul. See the test/ folder for testing scripts.

Running integration tests (narrow)

npm run test:int:narrow
```bash
npm run test:int:narrow
```

### Running Integration Tests interactively

If you want to run integration tests in a repetitive manner, you can startup the test containers using `docker-compose`, login to running `central-settlement` container like so:
If you want to run integration tests in a repetitive manner, you can start-up the test containers using `docker-compose`, login to running `central-settlement` container like so:

```bash
docker-compose -f docker-compose.yml -f docker-compose.integration.yml up
docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d
docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d mysql kafka central-ledger
docker-compose -f docker-compose.yml -f docker-compose.integration.yml up mysql kafka central-ledger

#first time only:
docker exec -it cs_central-ledger sh
docker exec -it central-ledger-int sh
export CL_DATABASE_HOST=localhost
npm run migrate
exit #exit running container

#in a new shell
docker exec -it cs_central-settlement sh
npm run test:int
```

#remove
docker pull mojaloop/central-settlement:latest
docker pull mojaloop/central-ledger:latest
docker rm cs_central-settlement
docker rm central-ledger-int

TODO @jason remove the below:
```shell
apk add python
npm install tigerbeetle-node
```

## Auditing Dependencies

We use `npm-audit-resolver` along with `npm audit` to check dependencies for node vulnerabilities, and keep track of resolved dependencies with an `audit-resolve.json` file.
Expand All @@ -88,7 +108,6 @@ You can then check to see if the CI will pass based on the current dependencies
```bash
npm run audit:check
```

And commit the changed `audit-resolv.json` to ensure that CircleCI will build correctly.

## Container Scans
Expand Down
3 changes: 3 additions & 0 deletions config/default.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
"CREATE_RETRY_INTERVAL_MILLIS": 200,
"DEBUG": false
},
"TIGERBEETLE" : {
"ENABLED" : false
},
"WINDOW_AGGREGATION": {
"RETRY_COUNT": 3,
"RETRY_INTERVAL": 3000
Expand Down
Loading