Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing db caching functionality #211

Merged
merged 89 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from 88 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
02aff57
Implementing file cache functionality
mchavez Aug 20, 2024
b052284
Open or Create file
mchavez Aug 22, 2024
17f6ef0
Refactoring code
mchavez Aug 22, 2024
bec4265
Adding dbcache functionality
mchavez Aug 23, 2024
9136435
Adding dependencies
mchavez Aug 23, 2024
49bc2ad
Updating dbcache
mchavez Aug 23, 2024
97d4beb
Updating dbcache
mchavez Aug 24, 2024
5a36a0f
Updating dbcache
mchavez Aug 24, 2024
c3283ac
Removing unnecessary code
mchavez Aug 24, 2024
a4c8c9d
Fix linter issues
mchavez Aug 24, 2024
b7f7174
Adding tests
mchavez Aug 24, 2024
1b0e067
Adding tests
mchavez Aug 24, 2024
2b68c14
Adding interface
mchavez Aug 26, 2024
b0e8597
Setting up db cache within uhttp pkg
mchavez Aug 26, 2024
b4146b6
Adding validations
mchavez Aug 26, 2024
a38ba33
Adding expiration time
mchavez Aug 27, 2024
82cea4e
Adding expiration time
mchavez Aug 27, 2024
aa3ab04
Refactoring code
mchavez Aug 27, 2024
ec92df8
Refactoring code
mchavez Aug 27, 2024
0a0685e
Refactoring code
mchavez Aug 27, 2024
c7720d2
Refactoring code
mchavez Aug 27, 2024
c187713
Refactoring code
mchavez Aug 27, 2024
0acefd1
Refactoring code
mchavez Aug 27, 2024
ff0121a
Code review changes
mchavez Aug 27, 2024
daa9740
Code review changes
mchavez Aug 27, 2024
25d8672
Code review changes
mchavez Aug 27, 2024
4be7206
Code review changes
mchavez Aug 27, 2024
990ed37
Adding db transactions
mchavez Aug 28, 2024
ec58c35
Fix bug when database is locked
mchavez Aug 28, 2024
41323bb
Fix tests
mchavez Aug 28, 2024
d23b63d
Adding BATON_IN_MEMORY_HTTP_CACHE env
mchavez Aug 28, 2024
5a820fb
Adding stats structure
mchavez Aug 28, 2024
8327789
Adding statistics
mchavez Aug 29, 2024
0518bf9
Adding statistics
mchavez Aug 29, 2024
1e94994
Adding statistics
mchavez Aug 29, 2024
025c015
Adding statistics
mchavez Aug 29, 2024
d6dfe52
Refactoring code
mchavez Aug 29, 2024
f9dc57d
Refactoring code
mchavez Aug 30, 2024
549da81
Refactoring code
mchavez Sep 2, 2024
65c5ac1
Refactoring code
mchavez Sep 3, 2024
2a7eac7
Code review changes
mchavez Sep 3, 2024
176642e
Refactoring code
mchavez Sep 3, 2024
d272bb9
Code review changes
mchavez Sep 3, 2024
f084c8f
Code review changes part 1
mchavez Sep 3, 2024
d811ab9
Code review changes part 2
mchavez Sep 3, 2024
3c62eb7
Code review changes part 3
mchavez Sep 3, 2024
f4911ad
Code review changes part 3
mchavez Sep 3, 2024
d0b2f8e
Code review changes part 4
mchavez Sep 3, 2024
f63c4e7
Code review changes part 4
mchavez Sep 3, 2024
a758909
Code review changes part 5
mchavez Sep 3, 2024
b70ba01
Code review changes part 5
mchavez Sep 3, 2024
88c1b0a
Code review changes part 5
mchavez Sep 3, 2024
4e8d7c8
Code review changes part 5
mchavez Sep 4, 2024
2fe7450
Code review changes part 6
mchavez Sep 4, 2024
9fdab12
Refactoring code
mchavez Sep 5, 2024
c416256
Refactoring code
mchavez Sep 5, 2024
4661eb0
Code review changes
mchavez Sep 5, 2024
aa417ec
Code review changes
mchavez Sep 5, 2024
c6fa85b
Code review changes
mchavez Sep 5, 2024
1387057
Refactoring code
mchavez Sep 6, 2024
acbfed8
Refactoring code
mchavez Sep 6, 2024
1b92aa0
Refactoring code
mchavez Sep 6, 2024
2761b4e
Refactoring code
mchavez Sep 6, 2024
07e2529
Refactoring code
mchavez Sep 6, 2024
ab72b6a
Refactoring code
mchavez Sep 6, 2024
587f1c5
Refactoring code
mchavez Sep 6, 2024
ac5a8cb
Refactoring code
mchavez Sep 6, 2024
b1ba948
Use goqu for sqlite like we do elsewhere in baton-sdk, removing the d…
ggreer Sep 10, 2024
3b0a2d6
Fix linter error
ggreer Sep 12, 2024
a8d0c33
Fix error where failing tests didn't cause CI to fail.
ggreer Sep 12, 2024
18be0d9
Upgrade github action versions.
ggreer Sep 12, 2024
6a78141
Use latest version of golang-test-annotations. Output test results in…
ggreer Sep 12, 2024
6534213
Truncate tables on db cache initialization. Test setting same cache k…
ggreer Sep 12, 2024
1433b01
Update on conflict. Set default cache size to 50MB.
ggreer Sep 12, 2024
3ed79a1
Clean up cache interface. Accept request instead of having to generat…
ggreer Sep 12, 2024
edfee11
Clean up cache config. Add noop cache instead of initializing a disab…
ggreer Sep 12, 2024
468af3c
Move cache config stuff into cache file. Don't create cachekey since …
ggreer Sep 13, 2024
ffcab73
Remove unused functions.
ggreer Oct 31, 2024
b428bbf
Use one variable for the cache we're returning.
ggreer Nov 4, 2024
5756e76
Move all env var parsing into NewCacheConfigFromEnv. Clean up names a…
ggreer Nov 7, 2024
82ace9a
Add TODO.
ggreer Nov 8, 2024
93550b1
Add explicit GRPC call to clear caches/cleanup. Print cache stats.
ggreer Nov 8, 2024
d236ab6
Lots of DB cache cleanup. Use higher log levels for errors. Simplify …
ggreer Nov 9, 2024
7807106
Better function name for deleting expired cache entries. Fix stats ca…
ggreer Nov 9, 2024
0dfafaf
Make insert only accept bytes. Clean up pick() to build query and not…
ggreer Nov 9, 2024
32b2eef
Add support for sparse mode in memory cache. Reduce default cache siz…
ggreer Nov 21, 2024
f6b3098
Use request url, not response request url.
ggreer Nov 21, 2024
a271c5b
Error if we can't create the http cache.
ggreer Nov 21, 2024
3242eba
Remove unused SqliteError. Cache clearing log line should be debug, n…
ggreer Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Run linters
uses: golangci/golangci-lint-action@v6
with:
Expand All @@ -24,23 +24,26 @@ jobs:
steps:
- name: Install Go
if: success()
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: go tests
run: go test -v -covermode=count -json ./... | tee test.json
run: go test -v -covermode=count -json ./... > test.json
- name: Print go test results
if: always()
run: cat test.json
- name: annotate go tests
if: always()
uses: guyarb/golang-test-annotations@v0.5.1
uses: guyarb/golang-test-annotations@v0.8.0
with:
test-results: test.json
buf-lint-and-breaking-change-detection:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Setup
uses: bufbuild/buf-setup-action@v1
with:
Expand Down
19 changes: 11 additions & 8 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Setup
uses: bufbuild/buf-setup-action@v1
with:
Expand All @@ -25,13 +25,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v2
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Run linters
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v6
with:
version: latest
args: --timeout=3m
Expand All @@ -44,15 +44,18 @@ jobs:
steps:
- name: Install Go
if: success()
uses: actions/setup-go@v2
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: go tests
run: go test -v -covermode=count -json ./... | tee test.json
run: go test -v -covermode=count -json ./... > test.json
- name: Print go test results
if: always()
run: cat test.json
- name: annotate go tests
if: always()
uses: guyarb/golang-test-annotations@v0.5.1
uses: guyarb/golang-test-annotations@v0.8.0
with:
test-results: test.json
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ require (
github.com/lufia/plan9stats v0.0.0-20240408141607-282e7b5d6b74 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-sqlite3 v1.14.22 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
Expand Down
3 changes: 2 additions & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,9 @@ github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0V
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.7 h1:fxWBnXkxfM6sRiuH3bqJ4CfzZojMOLVc0UTsTglEghA=
github.com/mattn/go-sqlite3 v1.14.7/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
Expand Down
Loading
Loading