feat: refactor read store #615
Conversation
WalkthroughThe pull request introduces a comprehensive refactoring of the ledger system's query and resource management infrastructure. The changes primarily focus on transitioning from specific, tightly-coupled query types to a more flexible, generic resource querying approach. A new type Changes
Sequence DiagramsequenceDiagram
participant Client
participant API
participant Controller
participant Store
participant ResourceHandler
Client->>API: Send Query Request
API->>Controller: Process Query
Controller->>Store: Retrieve Paginated Resource
Store->>ResourceHandler: Build Dataset
ResourceHandler-->>Store: Return Filtered Query
Store-->>Controller: Return Paginated Results
Controller-->>API: Return Processed Data
API-->>Client: Send Response
Possibly related PRs
Suggested Reviewers
Poem
Tip CodeRabbit's docstrings feature is now available as part of our Early Access Program! Simply use the command Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 21
🧹 Outside diff range and nitpick comments (51)
internal/api/v2/controllers_balances_test.go (1)
Line range hint
39-88: Consider enhancing test coverage with additional scenarios.The test cases effectively cover basic querying scenarios. Consider adding tests for:
- Edge cases with empty or invalid options
- Multiple Builder conditions combined
- Different Expand configurations
- Boundary conditions for PIT values
Example test case structure:
{ name: "combined filters", body: `{"$and": [{"$match": {"address": "foo"}}, {"$exists": {"metadata": "bar"}}]}`, expectQuery: ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions]{ Opts: ledgercontroller.GetAggregatedVolumesOptions{}, PIT: &now, Builder: query.And( query.Match("address", "foo"), query.Exists("metadata", "bar"), ), Expand: make([]string, 0), }, },internal/api/v2/common.go (1)
56-62: Trim Whitespace ingetExpandFunctionWhen splitting the "expand" query parameter, consider trimming whitespace from each item to handle inputs with unintended spaces.
Apply this diff to trim whitespace:
func getExpand(r *http.Request) []string { return collectionutils.Flatten( collectionutils.Map(r.URL.Query()["expand"], func(from string) []string { - return strings.Split(from, ",") + items := strings.Split(from, ",") + for i, item := range items { + items[i] = strings.TrimSpace(item) + } + return items }), ) }internal/storage/ledger/legacy/queries.go (4)
4-9: Organize Imports for Better ClarityConsider separating standard library imports, third-party imports, and project-specific imports with blank lines to enhance readability.
22-27: Document Fields inFiltersForVolumesStructThe
FiltersForVolumesstruct containsUseInsertionDateandGroupLvlfields. Adding comments to explain these fields will improve code maintainability and assist other developers.
36-52: Add Comments toListTransactionsQueryMethodsIncluding method comments for
WithColumnand the constructorNewListTransactionsQuerywill provide clarity on their usage and any expectations for parameters.
54-76: Validate Transaction ID inGetTransactionQueryIn
GetTransactionQuery, ensure that theIDfield is validated to prevent invalid or malicious input when querying transactions.Consider adding validation logic:
if q.ID <= 0 { return errors.New("transaction ID must be positive") }internal/storage/ledger/legacy/adapters.go (1)
21-21: Address the TODO comment to ensure compatibility with v1The code includes a TODO comment
// todo; handle compat with v1. It's important to handle compatibility with version 1 to prevent potential issues. Please consider implementing the compatibility logic or creating a GitHub issue to track this task.Would you like assistance in implementing the compatibility handling or opening a GitHub issue to track this task?
internal/storage/ledger/resource_aggregated_balances.go (1)
Line range hint
167-168: Avoid usingpanic; return an error insteadUsing
panic("unreachable")in theresolveFiltermethod is risky. It's better to return an error to handle unexpected cases gracefully.Apply this diff:
- panic("unreachable") + return "", nil, fmt.Errorf("unhandled property: %s", property)internal/storage/ledger/resource_transactions.go (1)
130-133: Simplify reverted transaction filter logicThe filter for the "reverted" property can be simplified for clarity.
Apply this diff to simplify the condition:
- ret := "reverted_at is" - if value.(bool) { - ret += " not" - } - return ret + " null", nil, nil + if value.(bool) { + return "reverted_at IS NOT NULL", nil, nil + } + return "reverted_at IS NULL", nil, nilinternal/storage/ledger/resource_volumes.go (1)
198-200: Clarify the error message inexpandmethodThe
expandmethod currently returns a generic error message. Providing a more specific message can help with debugging.Apply this diff to improve the error message:
- return nil, nil, errors.New("no expansion available") + return nil, nil, errors.New("expansion not available for volumesResourceHandler")internal/storage/ledger/paginator_column.go (2)
155-160: Simplify the unnecessary for loopThe
forloop executes only once due to the immediatebreak. Simplify the code by removing the loop.Apply this diff to simplify:
- for { if field.Type.Kind() == reflect.Ptr { fieldType = field.Type.Elem() } break - } + if field.Type.Kind() == reflect.Ptr { + fieldType = field.Type.Elem() + }
20-20: Avoid suppressing linter warnings with//nolint:unusedMultiple functions are marked with
//nolint:unused. If these functions are intended for future use or are used indirectly (e.g., via reflection), consider adding comments explaining their purpose. Otherwise, remove unused code to improve code quality.Also applies to: 62-62, 140-140, 188-188, 233-233
internal/storage/ledger/store.go (1)
148-151: Maintain consistency in parameter namingThe parameter
ledgerwas renamed tolin theNewfunction. For clarity and consistency, consider using descriptive parameter names.Apply this diff to rename the parameter:
- func New(db bun.IDB, bucket bucket.Bucket, l ledger.Ledger, opts ...Option) *Store { + func New(db bun.IDB, bucket bucket.Bucket, ledger ledger.Ledger, opts ...Option) *Store { ret := &Store{ db: db, - ledger: l, + ledger: ledger, bucket: bucket, }internal/controller/ledger/store.go (1)
149-173: Document theResourceQuerystruct's JSON unmarshallingThe custom unmarshalling logic in
ResourceQuerymay be non-obvious. Consider adding comments to explain how JSON unmarshalling is handled, especially regarding theBuilderfield.internal/storage/ledger/balances_test.go (1)
242-261: Simplify Big Int Calculations in Test AssertionsIn the
aggregate on alltest case, the construction ofInputandOutputvolumes involves multiple nestedbig.NewInt(0)initializations and arithmetic operations. Consider simplifying this by precomputing the expected totals to improve readability.Apply this diff to simplify the calculations:
-Input: big.NewInt(0).Add( - big.NewInt(0).Mul(bigInt, big.NewInt(2)), - big.NewInt(0).Mul(smallInt, big.NewInt(2)), -), +totalInput := big.NewInt(0) +totalInput.Add(totalInput, big.NewInt(0).Mul(bigInt, big.NewInt(2))) +totalInput.Add(totalInput, big.NewInt(0).Mul(smallInt, big.NewInt(2))) +Input: totalInput, -Output: big.NewInt(0).Add( - big.NewInt(0).Mul(bigInt, big.NewInt(2)), - big.NewInt(0).Mul(smallInt, big.NewInt(2)), -), +totalOutput := big.NewInt(0) +totalOutput.Add(totalOutput, big.NewInt(0).Mul(bigInt, big.NewInt(2))) +totalOutput.Add(totalOutput, big.NewInt(0).Mul(smallInt, big.NewInt(2))) +Output: totalOutput,internal/storage/ledger/resource.go (1)
159-179: Ensure Proper Handling of Query ExpansionsIn the
Querymethod, when iterating overq.Expand, ensure that unsupported expansions are handled gracefully. Currently, ifselectQueryisnil, the loop continues, but there might be cases where an unsupported expansion could cause issues.Consider adding validation for the supported expansions and returning an informative error if an expansion is not supported.
internal/controller/ledger/controller_default.go (2)
190-192: Consider handling potential empty ledger more efficientlyIn the
importLogsfunction, you're checking if the ledger is empty by paginating logs withPageSize: 1. While this works, consider using a more efficient query to check for the existence of logs, such as counting logs or checking for the minimum ID.
292-297: MakePageSizeconfigurable inExportfunctionIn the
Exportmethod, thePageSizeis hardcoded to100. Consider making this value configurable or document the choice to ensure it meets various use cases, especially for larger datasets.internal/controller/ledger/stats.go (1)
16-24: LGTM! Consider enhancing error messages.The refactoring to use
ResourceQuery[any]aligns well with the broader effort to streamline query handling. The implementation is correct and maintains proper error handling.Consider making the error messages more specific:
- return stats, fmt.Errorf("counting transactions: %w", err) + return stats, fmt.Errorf("failed to count transactions in store: %w", err) - return stats, fmt.Errorf("counting accounts: %w", err) + return stats, fmt.Errorf("failed to count accounts in store: %w", err)internal/api/v2/controllers_transactions_count.go (1)
15-20: Consider extracting common count handling logicThe implementation is nearly identical to
controllers_accounts_count.go. Consider extracting the common counting logic into a shared helper function to reduce duplication and maintain consistency.Example refactor:
func handleResourceCount[T any]( w http.ResponseWriter, r *http.Request, countFn func(context.Context, ledgercontroller.ResourceQuery[T]) (uint64, error), ) { rq, err := getResourceQuery[T](r) if err != nil { api.BadRequest(w, common.ErrValidation, err) return } count, err := countFn(r.Context(), *rq) if err != nil { switch { case errors.Is(err, ledgercontroller.ErrInvalidQuery{}) || errors.Is(err, ledgercontroller.ErrMissingFeature{}): api.BadRequest(w, common.ErrValidation, err) default: common.HandleCommonErrors(w, r, err) } return } w.Header().Set("Count", fmt.Sprint(count)) api.NoContent(w) }This would allow the count endpoints to be simplified to:
func countTransactions(w http.ResponseWriter, r *http.Request) { handleResourceCount(w, r, common.LedgerFromContext(r.Context()).CountTransactions) }Also applies to: 21-31
internal/api/v1/controllers_accounts_list.go (1)
21-24: Consider combining error handling blocksThe error handling for
buildAccountsFilterQueryfollows the same pattern as the previous block. Consider combining these operations to reduce code duplication.- rq, err := getOffsetPaginatedQuery[any](r) - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } - - rq.Options.Builder, err = buildAccountsFilterQuery(r) - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } + rq, err := getOffsetPaginatedQuery[any](r) + if err == nil { + rq.Options.Builder, err = buildAccountsFilterQuery(r) + } + if err != nil { + api.BadRequest(w, common.ErrValidation, err) + return + }internal/api/v1/controllers_transactions_read.go (1)
Line range hint
30-39: Consider adding transaction ID validationWhile the error handling is comprehensive, consider adding validation for negative transaction IDs.
txId, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64) - if err != nil { + if err != nil || txId < 0 { api.BadRequest(w, common.ErrValidation, err) return }internal/api/v1/controllers_accounts_count.go (2)
Line range hint
28-37: Consider adding count range validationWhile the error handling is comprehensive, consider adding validation for the count value before setting the header.
count, err := l.CountAccounts(r.Context(), *rq) if err != nil { switch { case errors.Is(err, ledgercontroller.ErrInvalidQuery{}) || errors.Is(err, ledgercontroller.ErrMissingFeature{}): api.BadRequest(w, common.ErrValidation, err) default: common.HandleCommonErrors(w, r, err) } return } + if count < 0 { + api.InternalServerError(w, fmt.Errorf("unexpected negative count: %d", count)) + return + } w.Header().Set("Count", fmt.Sprint(count)) api.NoContent(w)
22-26: Consider combining error handling blocksSimilar to the accounts list controller, consider combining the error handling blocks for query building.
- rq, err := getResourceQuery[any](r) - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } - - rq.Builder, err = buildAccountsFilterQuery(r) - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } + rq, err := getResourceQuery[any](r) + if err == nil { + rq.Builder, err = buildAccountsFilterQuery(r) + } + if err != nil { + api.BadRequest(w, common.ErrValidation, err) + return + }internal/api/v1/controllers_balances_aggregates.go (1)
21-25: Consider documenting the UseInsertionDate behaviorThe closure pattern for configuring options is clean, but the
UseInsertionDateflag's implications should be documented.rq, err := getResourceQuery[ledgercontroller.GetAggregatedVolumesOptions](r, func(q *ledgercontroller.GetAggregatedVolumesOptions) error { + // UseInsertionDate: When true, uses the insertion timestamp instead of the effective date q.UseInsertionDate = true return nil })internal/api/v2/controllers_balances.go (1)
15-19: Consider standardizing parameter naming conventionWhile supporting both
use_insertion_dateanduseInsertionDatemaintains compatibility, we should consider standardizing on one style in future versions.Consider adding a TODO comment about deprecating one style:
rq, err := getResourceQuery[ledgercontroller.GetAggregatedVolumesOptions](r, func(options *ledgercontroller.GetAggregatedVolumesOptions) error { + // TODO: Deprecate camelCase parameter in favor of snake_case in v3 options.UseInsertionDate = api.QueryParamBool(r, "use_insertion_date") || api.QueryParamBool(r, "useInsertionDate") return nil })internal/api/v2/controllers_accounts_read.go (1)
Line range hint
17-21: Consider enhancing error handling for address parameterWhile the current error handling works, consider providing more specific error messages for URL unescaping failures to help with debugging.
param, err := url.PathUnescape(chi.URLParam(r, "address")) if err != nil { - api.BadRequestWithDetails(w, common.ErrValidation, err, err.Error()) + api.BadRequestWithDetails(w, common.ErrValidation, err, + "Invalid account address format: failed to unescape URL parameter") return }internal/api/v1/controllers_balances_list.go (1)
15-20: Consider consistent error handling patternWhile the error handling is functional, there's a slight inconsistency in the pattern:
// First error check if err != nil { api.BadRequest(w, common.ErrValidation, err) return } // Second error check if err != nil { api.BadRequest(w, common.ErrValidation, err) return }Consider combining these checks or extracting to a helper function to maintain consistency.
Also applies to: 21-26
internal/api/v2/controllers_volumes.go (1)
37-51: Consider consolidating time parameter handlingThe startTime and endTime parameter handling is duplicated. Consider extracting this into a helper function to reduce code duplication and improve maintainability.
+func setTimeOption(r *http.Request, paramName string, setter func(*time.Time)) error { + if value := r.URL.Query().Get(paramName); value != "" { + t, err := getDate(r, paramName) + if err != nil { + return err + } + setter(t) + } + return nil +} // Usage in readVolumes: -if r.URL.Query().Get("endTime") != "" { - rq.Options.PIT, err = getDate(r, "endTime") - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } +if err := setTimeOption(r, "endTime", func(t *time.Time) { rq.Options.PIT = t }); err != nil { + api.BadRequest(w, common.ErrValidation, err) + return } -if r.URL.Query().Get("startTime") != "" { - rq.Options.OOT, err = getDate(r, "startTime") - if err != nil { - api.BadRequest(w, common.ErrValidation, err) - return - } +if err := setTimeOption(r, "startTime", func(t *time.Time) { rq.Options.OOT = t }); err != nil { + api.BadRequest(w, common.ErrValidation, err) + return }internal/storage/ledger/resource_logs.go (2)
13-20: Address TODO: Implement date validatorsThe date filter is missing validators which could lead to invalid date formats being processed. Consider implementing validation for date format and range.
Would you like me to help implement the date validators? I can provide a solution that includes:
- Date format validation
- Range validation
- Common date format parsing
42-44: Consider implementing meaningful aggregationThe current aggregate implementation is a no-op, returning the query unchanged. Consider implementing meaningful aggregations for logs (e.g., count by date).
Common log aggregations might include:
- Count by date/hour
- Count by type/severity
- Distribution analysis
Would you like guidance on implementing these aggregations?internal/storage/ledger/paginator_offset.go (1)
11-23: Consider performance implications of OFFSET-based paginationOFFSET-based pagination can become inefficient with large offsets as the database must scan and discard rows before reaching the desired offset.
Consider alternatives:
- Keyset pagination using unique, indexed columns
- Cursor-based pagination using the last seen record
- If OFFSET must be used, consider implementing maximum offset limits
internal/storage/ledger/legacy/balances.go (1)
13-13: Consider breaking down the complex query logicWhile the function works correctly, its complexity (100+ lines) makes it harder to maintain. Consider extracting the query building logic into smaller, focused functions.
Suggested structure:
func (store *Store) GetAggregatedBalances(ctx context.Context, q GetAggregatedBalanceQuery) (ledger.BalancesByAssets, error) { builder := newAggregatedBalancesQueryBuilder(store) query, err := builder. withMetadataJoins(q). withPITFilter(q). withCustomFilters(q). build() if err != nil { return nil, err } return store.executeAggregatedBalancesQuery(ctx, query) }internal/api/v2/controllers_accounts_count_test.go (1)
98-101: Verify error handling test casesThe error handling test cases cover various scenarios (invalid query, missing feature, unexpected error) with appropriate query construction. However, consider adding test cases for:
- Malformed PIT parameter
- Invalid expand parameters
Also applies to: 109-112, 120-123
internal/controller/ledger/controller.go (1)
27-35: Consider adding method documentation for query parametersThe transition to generic query types (
ResourceQuery[any],OffsetPaginatedQuery[any],ColumnPaginatedQuery[any]) improves flexibility, but the interface would benefit from documentation explaining:
- Expected query fields for each method
- Behavior of different pagination types
- Valid options for each query type
Example documentation format:
// GetAccount retrieves an account using the provided query // Query fields: // - PIT: Point in time for the query (required) // - Builder: Query builder for filtering (optional) // - Expand: Fields to expand in the response (optional) GetAccount(ctx context.Context, query ResourceQuery[any]) (*ledger.Account, error)internal/api/v2/controllers_volumes_test.go (1)
43-49: Consider simplifying the nested query structureWhile the nested structure with
Optionsprovides good type safety, it might be worth considering a builder pattern to simplify the construction of these complex queries.Example approach:
-expectQuery: ledgercontroller.OffsetPaginatedQuery[ledgercontroller.GetVolumesOptions]{ - PageSize: DefaultPageSize, - Options: ledgercontroller.ResourceQuery[ledgercontroller.GetVolumesOptions]{ - PIT: &before, - Expand: make([]string, 0), - }, -}, +expectQuery: ledgercontroller.NewVolumeQueryBuilder(). + WithPageSize(DefaultPageSize). + WithPIT(&before). + Build(),internal/storage/ledger/legacy/volumes.go (2)
Line range hint
15-92: Consider standardizing error messages and simplifying query building logic.The function could benefit from the following improvements:
- Standardize error message format (some use
errors.New, others usenewErrInvalidQuery).- Consider extracting operator conversion logic to a separate function for reusability.
func (store *Store) volumesQueryContext(q GetVolumesWithBalancesQuery) (string, []any, bool, error) { + // Extract to a package-level function + convertOperatorToSQL := func(operator string) (string, error) { switch operator { case "$match": return "=" case "$lt": return "<" case "$gt": return ">" case "$lte": return "<=" case "$gte": return ">=" default: - panic("unreachable") + return "", fmt.Errorf("unsupported operator: %s", operator) } }
Line range hint
93-167: Add documentation to explain complex SQL query logic.The function builds complex SQL queries with multiple joins and subqueries. Consider adding documentation to:
- Explain the purpose of each subquery and join
- Document the expected behavior of different filtering options
- Add examples of resulting queries for common use cases
internal/storage/ledger/legacy/accounts.go (2)
Line range hint
58-136: Consider improving error handling consistency in accountQueryContext.The function uses multiple error creation patterns. Consider standardizing error handling:
- Use consistent error creation method (either
errors.NewornewErrInvalidQuery)- Add error wrapping for better error context
Line range hint
168-181: Add documentation for SQL queries in account-related functions.The functions contain complex SQL queries. Consider adding:
- Documentation explaining the query structure
- Comments describing the purpose of joins and subqueries
- Examples of resulting queries for common use cases
Also applies to: 182-198
internal/api/v2/controllers_logs_list_test.go (2)
44-52: LGTM! Consider adding validation for the expand field.The nominal test case looks good, but consider adding test cases that validate the behavior when the expand field contains invalid values.
122-131: Consider adding specific error message validation.While the test case covers invalid query scenarios, it would be more robust to validate the specific error message returned.
expectStatusCode: http.StatusBadRequest, expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ PageSize: DefaultPageSize, Column: "id", Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), Options: ledgercontroller.ResourceQuery[any]{ Expand: make([]string, 0), }, }, -expectedErrorCode: common.ErrValidation, +expectedErrorCode: common.ErrValidation, +expectedErrorMessage: "invalid query: ...", // Add specific error message validationinternal/storage/ledger/legacy/transactions.go (1)
Line range hint
67-142: Consider improving error messages in query context builder.The error messages in the query context builder could be more descriptive and consistent.
-return "", nil, newErrInvalidQuery("'account' column can only be used with $match") +return "", nil, newErrInvalidQuery("column 'account' only supports the $match operator")Also, consider extracting the repeated validation logic into a helper function to reduce code duplication.
internal/controller/ledger/controller_with_traces.go (1)
56-74: Consider adding trace attributes for query parameters.The tracing implementation could be enhanced by adding relevant query parameters as span attributes.
func (c *ControllerWithTraces) ListTransactions(ctx context.Context, q ColumnPaginatedQuery[any]) (*bunpaginate.Cursor[ledger.Transaction], error) { - return tracing.Trace(ctx, c.tracer, "ListTransactions", func(ctx context.Context) (*bunpaginate.Cursor[ledger.Transaction], error) { + return tracing.Trace(ctx, c.tracer, "ListTransactions", func(ctx context.Context) (*bunpaginate.Cursor[ledger.Transaction], error) { + span := trace.SpanFromContext(ctx) + span.SetAttributes( + attribute.Int("pageSize", q.PageSize), + attribute.String("column", q.Column), + ) return c.underlying.ListTransactions(ctx, q) }) }internal/api/v1/controllers_transactions_list_test.go (1)
Line range hint
38-192: Consider adding test cases for combined filters.The test suite thoroughly covers individual filter scenarios but could benefit from additional test cases that verify the behavior of combined filters (e.g., metadata + time range, account + reference).
Example test case to add:
+ { + name: "using combined filters", + queryParams: url.Values{ + "start_time": []string{now.Format(time.DateFormat)}, + "reference": []string{"xxx"}, + }, + expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ + PageSize: DefaultPageSize, + Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), + Column: "id", + Options: ledgercontroller.ResourceQuery[any]{ + Builder: query.And( + query.Gte("date", now.Format(time.DateFormat)), + query.Match("reference", "xxx"), + ), + Expand: []string{"volumes"}, + }, + }, + },internal/api/v2/controllers_accounts_list_test.go (1)
Line range hint
43-120: Consider adding order verification in test cases.While the test cases cover various filtering scenarios, they don't explicitly verify the order of results. Consider adding assertions to verify the ordering of returned accounts.
Example modification:
if tc.expectStatusCode < 300 && tc.expectStatusCode >= 200 { cursor := api.DecodeCursorResponse[ledger.Account](t, rec.Body) require.Equal(t, expectedCursor, *cursor) + // Verify order + if len(cursor.Data) > 1 { + for i := 0; i < len(cursor.Data)-1; i++ { + require.True(t, cursor.Data[i].Address >= cursor.Data[i+1].Address, + "Results should be ordered by address in descending order") + } + } }internal/storage/ledger/legacy/transactions_test.go (2)
Line range hint
162-167: Consider extracting the test helper function.The anonymous function for refreshing tx3 could be extracted into a named test helper function for better reusability and readability.
+func getTransactionWithVolumes(t *testing.T, store *LedgerStore, ctx context.Context, txID string) ledger.Transaction { + tx, err := store.Store.GetTransactionWithVolumes(ctx, ledgerstore.NewGetTransactionQuery(txID). + WithExpandVolumes(). + WithExpandEffectiveVolumes()) + require.NoError(t, err) + return *tx +} + // refresh tx3 -tx3 := func() ledger.Transaction { - tx3, err := store.Store.GetTransactionWithVolumes(ctx, ledgerstore.NewGetTransactionQuery(tx3BeforeRevert.ID). - WithExpandVolumes(). - WithExpandEffectiveVolumes()) - require.NoError(t, err) - return *tx3 -}() +tx3 := getTransactionWithVolumes(t, store, ctx, tx3BeforeRevert.ID)
Line range hint
119-136: Consider documenting the test data setup.The test uses magic numbers (100 USD) in transaction amounts. Consider documenting why these specific values were chosen or using named constants for better maintainability.
+// Test constants +const ( + // Standard test amount used across transactions + testAmount = 100 + testCurrency = "USD" +) tx1 := ledger.NewTransaction(). WithPostings( - ledger.NewPosting("world", "alice", "USD", big.NewInt(100)), + ledger.NewPosting("world", "alice", testCurrency, big.NewInt(testAmount)),internal/storage/ledger/legacy/accounts_test.go (1)
Line range hint
101-108: Consider adding edge cases for volume calculationsWhile the test covers basic volume calculations, consider adding test cases for:
- Negative volumes
- Zero volumes
- Very large numbers
internal/controller/ledger/controller_default_test.go (1)
232-238: Consider adding pagination edge casesWhile the basic functionality is tested, consider adding test cases for:
- Empty result sets
- Maximum page size
- Invalid page tokens
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint
73-312: Well-structured refactoring of query interfacesThe refactoring demonstrates good architectural decisions:
- Use of generic types reduces code duplication while maintaining type safety
- Consistent implementation across all API versions
- Clear separation between simple queries (
ResourceQuery[any]) and specialized ones (with concrete option types)- Appropriate use of pagination types based on the query nature
This change improves maintainability while preserving the type safety guarantees where needed.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
go.modis excluded by!**/*.modgo.sumis excluded by!**/*.sum,!**/*.sumtools/generator/go.sumis excluded by!**/*.sum,!**/*.sum
📒 Files selected for processing (82)
internal/README.md(2 hunks)internal/api/bulking/mocks_ledger_controller_test.go(9 hunks)internal/api/common/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/controllers_accounts_count.go(1 hunks)internal/api/v1/controllers_accounts_count_test.go(6 hunks)internal/api/v1/controllers_accounts_list.go(1 hunks)internal/api/v1/controllers_accounts_list_test.go(5 hunks)internal/api/v1/controllers_accounts_read.go(2 hunks)internal/api/v1/controllers_accounts_read_test.go(4 hunks)internal/api/v1/controllers_balances_aggregates.go(1 hunks)internal/api/v1/controllers_balances_aggregates_test.go(1 hunks)internal/api/v1/controllers_balances_list.go(1 hunks)internal/api/v1/controllers_logs_list.go(1 hunks)internal/api/v1/controllers_logs_list_test.go(3 hunks)internal/api/v1/controllers_transactions_count.go(1 hunks)internal/api/v1/controllers_transactions_count_test.go(3 hunks)internal/api/v1/controllers_transactions_list.go(1 hunks)internal/api/v1/controllers_transactions_list_test.go(4 hunks)internal/api/v1/controllers_transactions_read.go(2 hunks)internal/api/v1/controllers_transactions_read_test.go(2 hunks)internal/api/v1/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/utils.go(1 hunks)internal/api/v2/common.go(2 hunks)internal/api/v2/controllers_accounts_count.go(1 hunks)internal/api/v2/controllers_accounts_count_test.go(4 hunks)internal/api/v2/controllers_accounts_list.go(1 hunks)internal/api/v2/controllers_accounts_list_test.go(5 hunks)internal/api/v2/controllers_accounts_read.go(2 hunks)internal/api/v2/controllers_accounts_read_test.go(4 hunks)internal/api/v2/controllers_balances.go(1 hunks)internal/api/v2/controllers_balances_test.go(2 hunks)internal/api/v2/controllers_logs_list.go(1 hunks)internal/api/v2/controllers_logs_list_test.go(5 hunks)internal/api/v2/controllers_transactions_count.go(1 hunks)internal/api/v2/controllers_transactions_count_test.go(4 hunks)internal/api/v2/controllers_transactions_list.go(1 hunks)internal/api/v2/controllers_transactions_list_test.go(4 hunks)internal/api/v2/controllers_transactions_read.go(2 hunks)internal/api/v2/controllers_transactions_read_test.go(2 hunks)internal/api/v2/controllers_volumes.go(1 hunks)internal/api/v2/controllers_volumes_test.go(4 hunks)internal/api/v2/mocks_ledger_controller_test.go(9 hunks)internal/controller/ledger/controller.go(1 hunks)internal/controller/ledger/controller_default.go(4 hunks)internal/controller/ledger/controller_default_test.go(10 hunks)internal/controller/ledger/controller_generated_test.go(9 hunks)internal/controller/ledger/controller_with_traces.go(1 hunks)internal/controller/ledger/stats.go(1 hunks)internal/controller/ledger/stats_test.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/controller/ledger/store_generated_test.go(5 hunks)internal/storage/ledger/accounts.go(2 hunks)internal/storage/ledger/accounts_test.go(7 hunks)internal/storage/ledger/balances.go(0 hunks)internal/storage/ledger/balances_test.go(1 hunks)internal/storage/ledger/errors.go(0 hunks)internal/storage/ledger/legacy/accounts.go(6 hunks)internal/storage/ledger/legacy/accounts_test.go(19 hunks)internal/storage/ledger/legacy/adapters.go(2 hunks)internal/storage/ledger/legacy/balances.go(1 hunks)internal/storage/ledger/legacy/balances_test.go(8 hunks)internal/storage/ledger/legacy/logs.go(1 hunks)internal/storage/ledger/legacy/logs_test.go(2 hunks)internal/storage/ledger/legacy/queries.go(1 hunks)internal/storage/ledger/legacy/transactions.go(6 hunks)internal/storage/ledger/legacy/transactions_test.go(8 hunks)internal/storage/ledger/legacy/volumes.go(4 hunks)internal/storage/ledger/legacy/volumes_test.go(27 hunks)internal/storage/ledger/logs.go(0 hunks)internal/storage/ledger/logs_test.go(3 hunks)internal/storage/ledger/moves.go(0 hunks)internal/storage/ledger/moves_test.go(1 hunks)internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)internal/storage/ledger/store.go(4 hunks)
⛔ Files not processed due to max files limit (7)
- internal/storage/ledger/transactions.go
- internal/storage/ledger/transactions_test.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/volumes.go
- internal/storage/ledger/volumes_test.go
- internal/volumes.go
- test/e2e/api_transactions_list_test.go
💤 Files with no reviewable changes (4)
- internal/storage/ledger/errors.go
- internal/storage/ledger/logs.go
- internal/storage/ledger/moves.go
- internal/storage/ledger/balances.go
🔇 Additional comments (124)
internal/api/v2/controllers_balances_test.go (2)
Line range hint 93-120: LGTM! Test structure maintains good practices.
The test maintains good practices by:
- Using table-driven tests
- Properly mocking dependencies
- Including clear assertions
- Maintaining parallel test execution
31-31: LGTM! Verify the type change impact.
The transition to ResourceQuery[GetAggregatedVolumesOptions] aligns well with the refactoring objectives, providing a more generic and consistent approach to resource querying.
✅ Verification successful
Let me check the content of these files to understand if the old type is being deprecated or if there are any inconsistencies.
Let me check the implementation of the new type to ensure it's a proper replacement.
Let me check the implementation of the store's AggregatedBalances method to verify the complete transition.
Let me check the new store implementation to understand the complete transition path.
Based on the evidence gathered, I can now provide the final response:
Type change is part of a proper refactoring transition
The change from GetAggregatedBalanceQuery to ResourceQuery[GetAggregatedVolumesOptions] is part of a well-structured refactoring where:
- The legacy implementation in
internal/storage/ledger/legacy/is being replaced - A new store implementation using
ResourceQueryis in place atinternal/storage/ledger/store.go - The transition is handled through an adapter pattern (
DefaultStoreAdapter) that bridges old and new implementations - The change is consistently applied across controllers and tests
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify consistent usage of ResourceQuery across the codebase
# Look for any remaining uses of the old GetAggregatedBalanceQuery type
rg -l "GetAggregatedBalanceQuery"
# Look for other similar transitions to ResourceQuery
ast-grep --pattern 'ResourceQuery[<$_>]'
Length of output: 198
Script:
#!/bin/bash
# Check content of files containing old query type
rg -A 5 -B 5 "GetAggregatedBalanceQuery" internal/storage/ledger/legacy/balances.go internal/storage/ledger/legacy/queries.go internal/controller/ledger/README.md
# Check if the new type is being used in test files
rg -l "ResourceQuery.*GetAggregatedVolumesOptions"
Length of output: 14979
Script:
#!/bin/bash
# Check the implementation of the new ResourceQuery type with GetAggregatedVolumesOptions
rg -A 10 "type.*ResourceQuery.*GetAggregatedVolumesOptions"
# Check the implementation in the controller
rg -A 10 "func.*GetAggregatedBalances" internal/controller/ledger/controller_default.go
Length of output: 624
Script:
#!/bin/bash
# Check the store's implementation
rg -A 10 "func.*AggregatedBalances" internal/storage/ledger/
# Check the GetAggregatedVolumesOptions type
rg -A 5 "type.*GetAggregatedVolumesOptions" internal/storage/ledger/
Length of output: 2921
Script:
#!/bin/bash
# Check the new store implementation for AggregatedVolumes
rg -A 10 "func.*AggregatedVolumes" internal/storage/ledger/
# Check the resource implementation
rg -A 10 "type.*Resource\[" internal/storage/ledger/
# Look for the GetAggregatedVolumesOptions definition
rg -A 5 "GetAggregatedVolumesOptions.*struct" internal/storage/ledger/
Length of output: 5905
internal/api/v1/utils.go (2)
26-44: Validate Page Size Limits in getOffsetPaginatedQuery
The getOffsetPaginatedQuery function correctly extracts the page size using bunpaginate.GetPageSize. Please ensure that the MaxPageSize and DefaultPageSize constants are appropriately set to prevent performance issues due to excessively large page sizes.
67-79: Effective Use of Generics in getResourceQuery
The getResourceQuery function efficiently utilizes Go generics to handle resource queries with flexible options. This approach enhances code reusability and maintainability.
internal/api/v2/common.go (3)
17-29: Ensure Comprehensive Date Parsing in getDate Function
The getDate function correctly parses dates from query parameters. Verify that it accounts for all expected date formats and handles potential parsing errors gracefully.
64-80: Verify Page Size Constraints in getOffsetPaginatedQuery
Ensure that the MaxPageSize and DefaultPageSize used in getOffsetPaginatedQuery are set to appropriate values to prevent performance degradation with large datasets.
104-131: Robust Error Handling in getResourceQuery
The getResourceQuery function effectively handles errors from date parsing and query building. Ensure that any additional modifiers provided do not introduce unexpected errors.
internal/storage/ledger/accounts.go (2)
53-53: Confirm updated_at Field Update in UpdateAccountsMetadata
Adding Set("updated_at = excluded.updated_at") ensures that the updated_at timestamp reflects the latest metadata update. Verify that this change aligns with the system's requirements for tracking account updates.
Line range hint 68-68: Ensure Consistency in UpsertAccounts
Including Set("updated_at = excluded.updated_at") in UpsertAccounts aligns the update behavior with UpdateAccountsMetadata. Confirm that updating the updated_at field during upserts is intended and that it doesn't adversely affect any timestamp-related logic.
internal/storage/ledger/legacy/queries.go (3)
28-34: Validate Pagination Parameters in NewGetVolumesWithBalancesQuery
Ensure that the PageSize and Order parameters in NewGetVolumesWithBalancesQuery are set appropriately and that they comply with any maximum limits defined in the system.
78-98: Confirm Default Ordering in ListAccountsQuery
The Order is set to bunpaginate.OrderAsc in NewListAccountsQuery. Verify that this default ordering matches the expected behavior for listing accounts in your application.
100-127: Consistency in Method Naming for GetAccountQuery
Methods like WithPIT, WithExpandVolumes, and WithExpandEffectiveVolumes provide a fluent interface. Ensure method names and patterns are consistent across query structs for a cohesive API design.
internal/storage/ledger/legacy/adapters.go (6)
22-24: Method 'Accounts' correctly delegates to newStore
The Accounts method correctly delegates to d.newStore.Accounts(), ensuring that account retrieval uses the new store consistently.
26-28: Method 'Logs' correctly delegates to newStore
The Logs method correctly delegates to d.newStore.Logs(), maintaining consistent log retrieval.
30-32: Method 'Transactions' correctly delegates to newStore
The Transactions method correctly delegates to d.newStore.Transactions(), ensuring transactions are retrieved from the new store.
34-36: Method 'AggregatedBalances' correctly delegates to newStore
The AggregatedBalances method correctly delegates to d.newStore.AggregatedVolumes(), maintaining functionality for aggregated balances.
38-40: Method 'Volumes' correctly delegates to newStore
The Volumes method correctly delegates to d.newStore.Volumes(), ensuring volume data is accessed consistently.
70-70: Changing 'UpsertAccounts' to variadic parameters may affect existing calls
The UpsertAccounts method now accepts variadic parameters accounts ...*ledger.Account instead of a slice accounts []*ledger.Account. This change may impact existing code that passes a slice of accounts. Callers will need to use the variadic expansion syntax accounts... when passing a slice.
Run the following script to identify any calls to UpsertAccounts that may need updating:
✅ Verification successful
No breaking changes detected in UpsertAccounts usage
The codebase scan shows that all existing calls to UpsertAccounts are already using the variadic form correctly:
- Direct passing of individual accounts:
UpsertAccounts(ctx, &account1, &account2) - Single account passing:
UpsertAccounts(ctx, account) - Using the spread operator with slices:
UpsertAccounts(ctx, accounts...)
The change to variadic parameters appears to be backward compatible with the existing codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find calls to UpsertAccounts that pass a slice argument and may need to be updated.
# Test: Use ast-grep to find calls to UpsertAccounts that might pass a slice
ast-grep --lang go --pattern $'UpsertAccounts($_, $_)' --metavar '_list_argument'
Length of output: 283
Script:
#!/bin/bash
# Search for calls to UpsertAccounts method
rg -U "UpsertAccounts\(" -A 2
# Also search for the interface definition to understand if this is a breaking change
rg -U "interface.*\{" -A 10 "UpsertAccounts"
Length of output: 3189
internal/storage/ledger/store.go (1)
130-136: Consistent method receiver usage
The function validateAddressFilter was changed from a method on Store to a standalone function. Ensure that this change does not affect other parts of the code that may rely on the method receiver. Update all references accordingly.
Run the following script to search for references to s.validateAddressFilter:
✅ Verification successful
Function change is correctly implemented
The search results show that validateAddressFilter is now being called directly as a standalone function in multiple files (resource_volumes.go, resource_transactions.go, resource_aggregated_balances.go, and resource_accounts.go), and there are no remaining references to s.validateAddressFilter. The change from a method to a standalone function has been consistently implemented across the codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find all usages of s.validateAddressFilter
rg -F 's.validateAddressFilter'
Length of output: 31
Script:
#!/bin/bash
# Let's search for any references to validateAddressFilter in the codebase
rg "validateAddressFilter"
# Also, let's look at the git history for this specific function to understand the change
git log -L :validateAddressFilter:internal/storage/ledger/store.go
# Let's also check for any method calls on Store struct to understand the context
ast-grep --pattern 'type Store struct {
$$$
}'
Length of output: 3110
internal/api/v2/controllers_transactions_list_test.go (2)
33-34: Updated expectQuery Type to ColumnPaginatedQuery[any]
The expectQuery field in the test cases has been updated to use ledgercontroller.ColumnPaginatedQuery[any], which aligns with the new query structures. This change enhances the flexibility of query handling in the tests.
Line range hint 42-236: Test Cases Reflect Refactored Query Structures Correctly
All test cases have been appropriately updated to utilize the new ColumnPaginatedQuery[any] and ResourceQuery[any] types. The use of query builders like query.Match, query.Gte, and query.Lte is correctly implemented, ensuring that the tests cover various scenarios effectively.
internal/storage/ledger/balances_test.go (1)
257-259: Verify Equal Input and Output Volumes for "EUR" Asset
In the test case, both Input and Output volumes for the "EUR" asset are set to smallInt, resulting in a net balance of zero. Please verify if this is the intended behavior, as it might affect the accuracy of the aggregated volumes.
internal/storage/ledger/accounts_test.go (3)
75-77: Updated Account Listing to Use New Query Interface
The test case now uses store.Accounts().List with ledgercontroller.ResourceQuery[any]{}, which aligns with the refactored query handling. This change simplifies the code and enhances maintainability.
82-86: Correct Use of Metadata Filtering in Account Listing
The test correctly applies a metadata filter using query.Match("metadata[category]", "1"), ensuring that only accounts with the specified metadata are returned. This demonstrates proper usage of the new query builder.
236-241: Improved Error Handling for Invalid Filters
In the test case "list using filter invalid field", the code correctly checks for errors when an invalid field is used in the filter. This ensures robustness by validating query inputs.
internal/controller/ledger/controller_default.go (10)
7-7: Import pointer package is appropriate
The import of "github.com/formancehq/go-libs/v2/pointer" is necessary for pointer utilities used in the code, such as pointer.For. This is appropriate.
110-112: Function IsDatabaseUpToDate correctly added
The new method IsDatabaseUpToDate correctly delegates to ctrl.store.IsUpToDate(ctx), providing a way to check database synchronization status.
130-131: 🛠️ Refactor suggestion
Update calls to CountAccounts to match new signature
Ensure that CountAccounts calls are updated to pass ResourceQuery[any].
Run the following script:
134-135: 🛠️ Refactor suggestion
Update calls to ListAccounts to match new signature
ListAccounts now accepts OffsetPaginatedQuery[any]. Please update all method invocations accordingly.
Run the following script:
138-140: 🛠️ Refactor suggestion
Update calls to GetAccount to match new signature
Ensure that all calls to GetAccount now pass a ResourceQuery[any] parameter.
Run the following script:
142-147: Ensure consistency in GetAggregatedBalances implementation
The method now uses ResourceQuery[GetAggregatedVolumesOptions]. Verify that this change is reflected in all related function calls and that GetAggregatedVolumesOptions is correctly utilized.
Run the following script:
✅ Verification successful
Let me analyze the implementation of GetAggregatedBalances further by checking the interface definition and related types.
Let me check one more thing to verify the implementation consistency by looking at the store interface and its usage.
Based on the gathered information, I can now provide the final response:
GetAggregatedBalances implementation is consistent across the codebase
The method's signature change to use ResourceQuery[GetAggregatedVolumesOptions] is properly reflected across all implementations and usages:
- The interface definition in
controller.go - The implementation in
DefaultController - The mock implementations in test files
- All callers in the API layers (v1, v2, bulking) use the correct type
- The
GetAggregatedVolumesOptionsstruct is properly defined with the expected fields
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Locate calls to `GetAggregatedBalances` to check for consistency.
rg -A 2 'GetAggregatedBalances\('
Length of output: 11919
Script:
#!/bin/bash
# Check the interface definition and related types
ast-grep --pattern 'type ResourceQuery[$_] struct {
$$$
}'
# Check GetAggregatedVolumesOptions definition
ast-grep --pattern 'type GetAggregatedVolumesOptions struct {
$$$
}'
Length of output: 348
Script:
#!/bin/bash
# Check the store interface and AggregatedBalances implementation
ast-grep --pattern 'type AggregatedBalancesStore interface {
$$$
}'
# Check actual store implementation
ast-grep --pattern 'func (store $_) GetOne(ctx context.Context, q $_) ($_, error) {
$$$
}'
Length of output: 166
118-119: 🛠️ Refactor suggestion
Update calls to ListTransactions to match new signature
The method ListTransactions now accepts ColumnPaginatedQuery[any] instead of ListTransactionsQuery. Ensure that all invocations of this method are updated accordingly to prevent type mismatch errors.
Run the following script to verify all calls to ListTransactions have been updated:
✅ Verification successful
Let me analyze the results from the rg command to verify if all calls to ListTransactions have been updated to use the new signature.
Based on the results from all the searches, I can now provide a final response:
All calls to ListTransactions are using the correct signature
The verification shows that:
- The interface and all implementations (DefaultController, ControllerWithTraces) use
ColumnPaginatedQuery[any] - All mock implementations in test files use the correct signature
- The API layer calls are using the updated type
- Client implementations are properly aligned
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find all calls to `ListTransactions` that may still use the old signature.
# Search for calls to `ListTransactions` with the old query type
rg -A 2 'ListTransactions\('
Length of output: 16240
Script:
#!/bin/bash
# Let's check the interface definition and implementation to confirm the change
ast-grep --pattern 'interface \{
$$$
ListTransactions(ctx context.Context, $$$) $$$
$$$
\}'
Length of output: 99
Script:
#!/bin/bash
# Let's check the interface and implementation files directly
rg -A 5 'type.*Controller.*interface' internal/controller/ledger/
Length of output: 1178
Script:
#!/bin/bash
# Let's check all the actual calls to ListTransactions in the API layer
rg -B 2 -A 2 'ListTransactions\(.*ctx' internal/api/
Length of output: 4481
154-155: 🛠️ Refactor suggestion
Update calls to GetVolumesWithBalances to match new signature
The method now requires OffsetPaginatedQuery[GetVolumesOptions]. Ensure all calls are updated with the correct query type.
Run the following script:
150-151: 🛠️ Refactor suggestion
Update calls to ListLogs to match new signature
ListLogs now accepts ColumnPaginatedQuery[any]. Update all usages accordingly to prevent any runtime errors.
Run the following script:
126-127: 🛠️ Refactor suggestion
Update calls to GetTransaction to match new signature
The method now requires a ResourceQuery[any] parameter. Update all usages to pass the correct query type.
Run the following script:
internal/controller/ledger/store_generated_test.go (8)
45-57: New Accounts method in MockStore correctly added
The Accounts method returns PaginatedResource[ledger.Account, any, OffsetPaginatedQuery[any]]. Ensure that the mock behavior is correctly set up in tests.
59-71: New AggregatedBalances method in MockStore correctly added
The AggregatedBalances method provides access to the resource for aggregated balances. Verify that this aligns with the updated store interface.
233-245: New Logs method in MockStore correctly added
The addition of the Logs method aligns with the new interface structure, returning PaginatedResource[ledger.Log, any, ColumnPaginatedQuery[any]].
292-305: New Transactions method in MockStore correctly added
This method provides access to transactions through the mock store. Ensure that all tests using this method are updated accordingly.
354-368: New Volumes method in MockStore correctly added
The Volumes method returns PaginatedResource for volumes, supporting the updated pagination strategy.
369-421: Addition of MockResource for generic resource mocking
The introduction of MockResource enhances the mocking capabilities for resources. Please ensure it's utilized correctly in tests.
442-548: Addition of MockPaginatedResource for paginated resource mocking
Similar to MockResource, MockPaginatedResource supports mocking of paginated resources. Verify that test cases are updated to use this new mock.
5-5: Verify proper generation of mock with mockgen
Ensure that the mockgen command used reflects the correct interfaces and that all necessary methods are included in the generated mock.
Run the following script:
✅ Verification successful
Let me verify the interface methods to ensure the mock generation is complete.
Let me verify if all the interface methods are properly implemented in the mock.
Let me check for the remaining required methods, particularly the Paginate and List methods that were shown in the interface.
Mock generation is correctly implemented
The generated mock in store_generated_test.go correctly implements all required methods from the PaginatedResource interface, including:
- The embedded
Resourceinterface methods (Count, GetOne, Query) - The
Paginatemethod with correct signature - The
Listmethod with correct signature and variadic modifiers
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Confirm that `mockgen` is generating mocks for the correct interfaces.
# Expected: The interface `PaginatedResource` should be present in `store.go`.
rg 'type PaginatedResource' internal/controller/ledger/store.go
Length of output: 180
Script:
#!/bin/bash
# Get the interface definition and its methods
rg -A 10 'type PaginatedResource' internal/controller/ledger/store.go
# Check if the mock implementation exists and contains all methods
rg 'type MockPaginatedResource' internal/controller/ledger/store_generated_test.go
Length of output: 873
Script:
#!/bin/bash
# Check the mock implementation for all required methods
rg -A 20 'type MockPaginatedResource' internal/controller/ledger/store_generated_test.go
# Also check if it embeds the Resource interface methods
rg 'Resource\[ResourceType, OptionsType\]' internal/controller/ledger/store_generated_test.go
Length of output: 2977
Script:
#!/bin/bash
# Check for Paginate and List methods in the generated mock
rg 'func \(m \*MockPaginatedResource.*\) (Paginate|List)' internal/controller/ledger/store_generated_test.go
Length of output: 524
internal/storage/ledger/paginator.go (1)
1-11: Implementation of generic paginator interface
The introduction of the generic paginator interface is a solid design choice to handle pagination consistently across different resources.
internal/api/v1/controllers_transactions_count.go (2)
13-17: Simplification of query retrieval and error handling
The code now directly retrieves the ResourceQuery using getResourceQuery[any](r) and returns immediately if there's an error. This streamlines the function and removes unnecessary complexity.
19-19: Update method call to use new query structure
The call to CountTransactions now uses *rq, aligning with the updated controller method signature. Ensure that this change is consistently applied throughout the codebase.
internal/api/v2/controllers_accounts_list.go (1)
Line range hint 15-19: LGTM! Verify error handling consistency.
The refactoring simplifies the query extraction while maintaining proper error handling. The use of getOffsetPaginatedQuery[any] aligns with the broader changes.
Let's verify that the error handling is consistent across all controllers:
✅ Verification successful
Let me verify the error handling implementation with another script to see the complete error handling blocks:
Error handling is consistent across controllers ✅
The error handling for getOffsetPaginatedQuery follows the same pattern consistently across all controllers:
- Same error response using
api.BadRequestwithcommon.ErrValidation - Same flow control with immediate return after error response
- Same variable naming convention and type handling
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check error handling consistency across controllers
# Look for similar error handling patterns in other controllers
rg -A 3 "getOffsetPaginatedQuery|getColumnPaginatedQuery" "internal/api/" | rg "if err != nil"
Length of output: 702
Script:
#!/bin/bash
# Check complete error handling blocks for getOffsetPaginatedQuery
rg -B 1 -A 3 "getOffsetPaginatedQuery.*\n.*if err != nil" "internal/api/"
Length of output: 265
Script:
#!/bin/bash
# Check complete error handling blocks for getOffsetPaginatedQuery
rg -U -B 1 -A 3 "getOffsetPaginatedQuery.*\n.*if err != nil" "internal/api/"
Length of output: 1258
internal/api/v1/controllers_transactions_list.go (1)
14-17: LGTM! Clean refactoring to column-based pagination.
The transition to getColumnPaginatedQuery with explicit sort parameters improves code clarity.
internal/api/v2/controllers_logs_list.go (2)
Line range hint 23-34: LGTM: Error handling looks good
The error handling is well-structured with appropriate error type checks and responses.
17-20: Consider using a more specific type parameter instead of any
While using any provides flexibility, it might reduce type safety. Consider using a more specific type that represents the expected query parameters to catch potential type-related issues at compile time.
Let's check how this type is used across the codebase:
internal/api/v2/controllers_accounts_count.go (2)
Line range hint 22-34: LGTM: Clean implementation with proper error handling
The implementation follows the established pattern for count endpoints, with appropriate error handling and response headers.
16-20: Verify error handling for missing features
The error handling includes checks for both ErrInvalidQuery and ErrMissingFeature. Let's verify that all potential error cases are covered.
✅ Verification successful
Based on the search results, I can now generate the final response. The code shows that ErrMissingFeature is properly handled in the accounts count controller, and the error handling follows the same pattern as other controllers in both v1 and v2 APIs.
Error handling is properly implemented
The error handling in the accounts count controller correctly handles both ErrInvalidQuery and ErrMissingFeature cases, consistent with other controllers in the codebase. The implementation follows the established pattern:
getResourceQueryis used to parse and validate the request- Error handling includes checks for both validation errors and missing feature errors
- The error response uses the standard
api.BadRequesthandler
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for potential error types returned by CountAccounts
ast-grep --pattern 'func ($x *$t) CountAccounts($ctx, $query) ($_, error) {
$$$
return $_, $err
$$$
}'
Length of output: 110
Script:
#!/bin/bash
# Let's search for the CountAccounts function implementation and its error handling
rg -A 10 "func.*CountAccounts"
# Also search for getResourceQuery to understand error types
rg -A 5 "func getResourceQuery"
Length of output: 29629
Script:
#!/bin/bash
# Let's check the implementation of the CountAccounts handler and its error handling
rg -A 10 "func.*countAccounts.*http.HandlerFunc"
# Also search for ErrMissingFeature usage
rg "ErrMissingFeature"
Length of output: 4888
internal/api/v1/controllers_accounts_list.go (2)
15-19: LGTM: Clean error handling for pagination query
The error handling for the pagination query extraction is properly implemented with appropriate error response.
Line range hint 27-36: LGTM: Comprehensive error handling for ListAccounts
The error handling for the ListAccounts operation is thorough, properly distinguishing between validation errors and common errors.
internal/api/v1/controllers_transactions_read.go (1)
23-28: LGTM: Clean implementation of resource query
The implementation properly handles query extraction and error cases.
internal/api/v1/controllers_accounts_count.go (1)
16-20: LGTM: Proper error handling for resource query
The error handling for query extraction is consistent with other controllers.
internal/api/v2/controllers_transactions_list.go (2)
16-19: LGTM: Clear and flexible pagination column selection
The logic for determining the pagination column based on the "order" parameter is clear and maintainable.
21-21: Verify the generic type parameter usage
The transition to using generic types (any) aligns with the refactoring goals, but we should verify that this doesn't lose any type safety benefits.
Also applies to: 27-27
✅ Verification successful
Generic type parameter usage is consistent and safe
The verification shows that getColumnPaginatedQuery[any] is used consistently across both v1 and v2 API endpoints for transaction and log listing. The function is properly defined with a generic type parameter v any and returns *ledgercontroller.ColumnPaginatedQuery[v]. The use of any as the type parameter is appropriate here since:
- The function is used consistently with
anyacross all call sites - Type safety is maintained through the generic function signature
- The pagination query is handling generic data retrieval where specific type constraints aren't necessary
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for other usages of getColumnPaginatedQuery to ensure consistent typing
ast-grep --pattern 'getColumnPaginatedQuery[$_]'
# Look for any type assertions or type-related error handling
rg 'type.*assert|interface.\{\}' -A 3
Length of output: 90
Script:
#!/bin/bash
# Search for the function definition and its usages with better patterns
ast-grep --pattern 'func getColumnPaginatedQuery[$_]'
# Search for all calls to this function
rg 'getColumnPaginatedQuery\[.*?\]' -A 2
# Look at the file directly to understand the context
cat internal/api/v2/controllers_transactions_list.go
Length of output: 3171
internal/api/v1/controllers_balances_aggregates.go (1)
31-33: LGTM: Clean query building integration
The integration of the query builder with the resource query is clean and maintains separation of concerns.
internal/api/v2/controllers_balances.go (1)
Line range hint 25-35: LGTM: Comprehensive error handling
The error handling is thorough and properly categorizes different error types for appropriate responses.
internal/api/v2/controllers_transactions_read.go (2)
4-4: LGTM: Clean import addition
The addition of the query package import is appropriate for the new query construction mechanism.
30-34: LGTM: Clean transition to ResourceQuery
The refactoring to use ResourceQuery[any] with query.Match improves type safety and aligns with the broader standardization effort. The query construction is now more explicit and maintainable.
internal/controller/ledger/stats_test.go (2)
18-19: LGTM: Well-typed mock resources
Good use of generics for type-safe mock resources. The separation between Transaction and Account resources with their respective pagination types is clean and maintainable.
21-24: LGTM: Clear mock expectations
The expectations are well-structured and properly chain the mock calls from store to individual resources. The use of empty ResourceQuery[any] is appropriate for the stats context.
internal/api/v2/controllers_accounts_read.go (1)
30-34: LGTM: Clean ResourceQuery implementation
The transition to ResourceQuery[any] with proper PIT handling and expansion is well-implemented. The query construction is consistent with the new pattern used across the codebase.
internal/api/v1/controllers_transactions_read_test.go (2)
4-4: LGTM: Import addition aligns with new query structure
The addition of the query package import is necessary for the new ResourceQuery pattern.
28-30: LGTM: Test correctly adapted to new query structure
The test has been properly updated to use the new ResourceQuery[any] type with the query builder pattern, maintaining the same test coverage while aligning with the refactoring changes.
internal/api/v1/controllers_balances_list.go (1)
26-26: LGTM: Clear query configuration
The explicit expansion of volumes and streamlined query execution improves code clarity.
Also applies to: 28-28
internal/api/v1/controllers_accounts_read.go (2)
4-4: LGTM: Required import for query builder
The addition of the query package import supports the new ResourceQuery pattern.
26-29: LGTM: Clean query construction with explicit expansion
The new ResourceQuery structure provides a cleaner way to construct the query while making the volumes expansion explicit. This change aligns well with the broader refactoring effort.
internal/api/v1/controllers_logs_list.go (2)
38-41: LGTM: Clean transition to generic pagination
The switch to getColumnPaginatedQuery[any] aligns well with the store refactoring goals, providing a more generic and reusable approach to pagination.
44-46: Verify query builder compatibility
The assignment of the query builder looks correct, but we should verify that the existing builder's output remains compatible with the new generic pagination approach.
✅ Verification successful
Let me gather more information about the query builder and pagination implementation to verify compatibility.
Let me try a different approach to check the query builder and pagination implementation.
Let me try one final verification to understand the pagination implementation.
Based on the comprehensive code analysis, I can now provide a final response about the query builder compatibility:
Query builder is compatible with generic pagination
The implementation shows that the query builder's output is fully compatible with the generic pagination approach:
- The
buildGetLogsQueryfunction returns aquery.Builderthat constructs valid query clauses for filtering logs - The
ListLogsmethod accepts aColumnPaginatedQuery[any]which properly integrates with the query builder - The pagination implementation in
internal/storage/ledger/legacy/queries.gocorrectly handles the builder's output - Test cases demonstrate successful integration between the query builder and pagination
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for other usages of buildGetLogsQuery to ensure consistent handling
rg "buildGetLogsQuery" --type go -B 2 -A 2
Length of output: 723
Script:
#!/bin/bash
# Check the implementation of ListLogs and its usage of query builder
ast-grep --pattern 'func $_ ListLogs($_, $query) $_ {
$$$
}'
# Check the query.Builder interface definition and implementation
rg "type Builder interface" --type go -A 5
Length of output: 109
Script:
#!/bin/bash
# Check the implementation of buildGetLogsQuery function
rg "func buildGetLogsQuery" --type go -A 15
# Look for the query package and Builder usage
rg "package query" --type go -A 5
Length of output: 1230
Script:
#!/bin/bash
# Look for the pagination query structure and its usage
rg "type PaginatedQuery" --type go -A 10
# Check for ListLogs implementation in a different way
rg "ListLogs" --type go -B 5 -A 10
Length of output: 52567
internal/api/v2/controllers_volumes.go (1)
53-53: Verify error handling completeness
The error handling for GetVolumesWithBalances looks comprehensive, but we should verify that these are all possible error cases.
✅ Verification successful
Based on the search results, I can now generate a final response. The code shows that GetVolumesWithBalances has a well-defined error handling pattern across different layers:
Error handling is comprehensive and well-structured
The implementation shows proper error handling at multiple layers:
- At the storage layer (internal/storage/ledger/legacy/volumes.go), errors from database operations are properly propagated
- At the controller layer (internal/controller/ledger/controller_default.go), errors are wrapped with tracing information
- At the API layer (pkg/client/v2.go), errors are properly mapped to V2ErrorResponse types
- The function uses standard error types and known domain-specific errors like ErrNotFound
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for all error types returned by GetVolumesWithBalances
ast-grep --pattern 'func GetVolumesWithBalances($$$) ($_, error) {
$$$
return $_, $err
$$$
}'
Length of output: 101
Script:
#!/bin/bash
# Let's try a broader search for GetVolumesWithBalances function definition and its error handling
rg -A 10 "func.*GetVolumesWithBalances" --type go
Length of output: 25244
Script:
#!/bin/bash
# Also search for the interface definition to understand the expected error cases
rg "GetVolumesWithBalances" --type go
Length of output: 15768
Script:
#!/bin/bash
# Search for any custom error types or variables that might be returned
rg "var .*(err|Err)" --type go
rg "type.*Error" --type go
Length of output: 6392
internal/storage/ledger/legacy/logs.go (1)
Line range hint 38-48: Consider adding type validation for query conversion
The type casting of q to ColumnPaginatedQuery is done without validation. While it might be safe in the current context, consider adding runtime type validation or using a more type-safe approach to prevent potential panics.
Consider adding validation:
func (store *Store) GetLogs(ctx context.Context, q GetLogsQuery) (*bunpaginate.Cursor[ledger.Log], error) {
+ if _, ok := interface{}(&q).(bunpaginate.ColumnPaginatedQuery[ledgercontroller.PaginatedQueryOptions[any]]); !ok {
+ return nil, fmt.Errorf("invalid query type: expected ColumnPaginatedQuery")
+ }
logs, err := paginateWithColumn[ledgercontroller.PaginatedQueryOptions[any], ledgerstore.Log](store, ctx,
(*bunpaginate.ColumnPaginatedQuery[ledgercontroller.PaginatedQueryOptions[any]])(&q),
store.logsQueryBuilder(q.Options),
)internal/storage/ledger/legacy/logs_test.go (2)
7-7: LGTM: Package import aligns with refactoring
The addition of the ledgerstore import supports the transition from controller-based to store-based query construction.
37-37: LGTM: Query construction refactored appropriately
The migration from ledgercontroller.NewListLogsQuery to ledgerstore.NewListLogsQuery maintains the same functionality while aligning with the store-based architecture. The query options and filtering capabilities remain intact.
Also applies to: 44-44, 50-57
internal/api/v1/controllers_balances_aggregates_test.go (2)
27-27: LGTM: Generic resource query type adoption
The change to ResourceQuery[GetAggregatedVolumesOptions] aligns with the broader refactoring goal of using more generic query types.
33-36: LGTM: Test cases properly restructured
The test cases have been appropriately updated to use the new query structure while maintaining essential test coverage. The use of typed options in GetAggregatedVolumesOptions provides better type safety.
Also applies to: 44-49
internal/api/v1/controllers_accounts_read_test.go (3)
5-5: LGTM: Required import for query building
The addition of the query package import supports the new query building approach.
39-44: LGTM: Test cases properly updated
The test cases have been successfully migrated to use the new query structure while maintaining comprehensive test coverage for nominal, expanded, and error scenarios.
Also applies to: 48-53, 66-71
29-29: Verify the use of any type parameter
While using ResourceQuery[any] provides flexibility, consider if a more specific type would be more appropriate for type safety.
✅ Verification successful
Using any type parameter is appropriate in this context
Based on the codebase analysis, the use of ResourceQuery[any] is consistent with the codebase's patterns and is appropriate here. The any type parameter is used extensively throughout the codebase for general-purpose queries where no specific options are needed, particularly in test files and controller implementations. When specific options are required, like in volume or aggregated balance queries, the code uses concrete types like ResourceQuery[GetVolumesOptions] or ResourceQuery[GetAggregatedVolumesOptions].
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if any other files use a more specific type parameter
rg -l "ResourceQuery\[(?!any\])" .
Length of output: 279
Script:
#!/bin/bash
# Search for all usages of ResourceQuery with any type parameter
rg "ResourceQuery\[" -A 2 .
Length of output: 93578
internal/api/v2/controllers_accounts_read_test.go (1)
29-29: LGTM: Query refactoring maintains test coverage while improving type flexibility
The transition to ResourceQuery[any] successfully preserves the test scenarios while making the query interface more generic. The test cases properly validate:
- Basic account queries
- Volume expansion scenarios
- Query builder functionality with address matching
Let's verify that all previous test scenarios are covered with the new query structure:
Also applies to: 42-45, 51-55, 64-68
✅ Verification successful
Verified: Test coverage is comprehensive with the new query structure
The test file maintains complete coverage of essential scenarios:
- Nominal case with basic account query
- Expanded volumes query functionality
- Expanded effective volumes query functionality
- Error handling for invalid account addresses
The transition to ResourceQuery[any] preserves all the critical test scenarios while adding type flexibility. Each test case properly validates the query construction, backend interaction, and response handling.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for any removed test scenarios in the git history
# Look for removed test cases in the file's history
git log -p internal/api/v2/controllers_accounts_read_test.go | rg -B 5 -A 5 '^\-.*testCase'
Length of output: 0
Script:
#!/bin/bash
# Let's try a different approach to analyze the test cases
# First, let's see the current test cases
rg "testCase" -A 5 "internal/api/v2/controllers_accounts_read_test.go"
# Then, let's look for test-related patterns to ensure coverage
rg "func Test.*Account" "internal/api/v2/controllers_accounts_read_test.go"
# Check the actual test implementation details
rg "t.Run\(" -A 3 "internal/api/v2/controllers_accounts_read_test.go"
Length of output: 897
Script:
#!/bin/bash
# Let's examine the test cases in more detail to verify coverage
# Get the full test file content to analyze the test scenarios
cat "internal/api/v2/controllers_accounts_read_test.go"
# Also check for any query-related test utilities or helper functions
rg "Query" "internal/api/v2/controllers_accounts_read_test.go"
Length of output: 3644
internal/api/v1/controllers_transactions_count_test.go (2)
111-111: LGTM: Mock expectation correctly updated
The mock expectation for CountTransactions is properly updated to use the new query type.
25-25: LGTM: Comprehensive test coverage with improved query building
The refactoring to ResourceQuery[any] maintains thorough test coverage while improving the query building interface. The test cases effectively validate:
- Basic counting
- Metadata filtering
- Time-based filtering
- Account/reference based filtering
- Source/destination filtering
Each test case properly uses the query builder pattern to construct the appropriate filters.
Let's verify the query builder usage patterns across the codebase:
Also applies to: 34-34, 41-43, 50-52, 59-61, 68-70, 77-79, 86-88, 95-97
✅ Verification successful
Query builder usage is consistent and well-tested across the codebase
The verification shows that the ResourceQuery[any] type and query builder patterns are consistently used across multiple test files:
controllers_transactions_count_test.go(both v1 and v2)controllers_accounts_read_test.go(both v1 and v2)controllers_accounts_count_test.go(both v1 and v2)
All test cases properly utilize the query builder with appropriate methods:
Matchfor exact matches (address, metadata, source, destination)Gte/Ltfor date and balance comparisonsExistsfor metadata checks
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Analyze query builder usage patterns
# Look for query builder patterns
ast-grep --pattern 'Builder: query.$_($_, $_)'
# Check for any potentially missed query builder cases
rg -A 2 'expectQuery.*ResourceQuery\[any\]'
Length of output: 12782
internal/api/v1/controllers_accounts_count_test.go (2)
27-27: LGTM: Query refactoring maintains comprehensive test coverage
The transition to ResourceQuery[any] successfully preserves all test scenarios while improving the query interface. The test cases effectively cover:
- Basic counting
- Metadata filtering
- Address filtering
- Balance filtering with operators
Also applies to: 37-38, 47-49, 55-56, 74-76
84-84: LGTM: Error handling scenarios properly maintained
The error test cases and mock expectations are correctly updated to work with the new query type while maintaining coverage of:
- Invalid query handling
- Missing feature errors
- Unexpected errors
Let's verify the error handling coverage:
Also applies to: 92-92, 100-100, 114-114
✅ Verification successful
Error handling coverage is comprehensive and properly maintained
The verification confirms that error handling scenarios are well covered in the test suite, including:
- Invalid query handling with proper BadRequest responses
- Missing feature errors with BadRequest responses
- Unexpected errors with InternalServerError responses
- Core validation errors with BadRequest responses
The test cases align with the new query type while maintaining robust error coverage.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Analyze error handling coverage
# Look for error test cases
rg -B 2 'expectStatusCode.*BadRequest|InternalServerError'
# Check for any potentially missed error scenarios
ast-grep --pattern 'returnErr: $_'
Length of output: 20142
internal/api/v1/controllers_logs_list_test.go (4)
30-30: LGTM: Type change aligns with refactoring goals
The change from PaginatedQueryOptions to ColumnPaginatedQuery is consistent with the PR's objective to use more generic query types.
38-43: LGTM: Well-structured nominal test case
The test case properly initializes the ColumnPaginatedQuery with appropriate default values and ordering.
109-109: LGTM: Mock expectations updated correctly
The mock expectations are properly updated to match the new query type.
76-78: Verify cursor encoding/decoding compatibility
The change in cursor type might affect existing cursor values in production. Ensure backward compatibility for existing cursor values.
internal/api/v1/controllers_accounts_list_test.go (4)
28-28: LGTM: Clean transition to OffsetPaginatedQuery
The change to OffsetPaginatedQuery is well-implemented with proper default values.
Also applies to: 39-41
49-54: LGTM: Query builder patterns are consistent
The metadata and address filtering test cases maintain consistent patterns in building queries with the new structure.
Also applies to: 62-67
148-149: LGTM: Mock expectations properly updated
The mock expectations correctly reflect the new query type in the ListAccounts call.
72-75: Verify cursor compatibility with offset pagination
Similar to the logs controller, ensure that the transition to offset-based pagination maintains compatibility with existing cursors.
✅ Verification successful
The previous search didn't provide enough context about the pagination implementation. Let's analyze the cursor encoding/decoding logic and pagination query types to understand the compatibility between offset and cursor-based pagination.
Let's try to find more specific information about the pagination implementation and cursor handling in the test file and related code.
Based on the code analysis, I can now generate a final response. The test file and related code show that:
- The test case explicitly verifies empty cursor compatibility with
OffsetPaginatedQuery - The cursor encoding/decoding is handled by the
bunpaginatepackage - All list endpoints consistently support both cursor-based pagination and page size parameters
- The test cases verify cursor compatibility by encoding an empty
OffsetPaginatedQueryand ensuring it works correctly
Cursor compatibility with offset pagination is properly maintained
The test case explicitly verifies that an empty cursor encoded from OffsetPaginatedQuery works correctly, and the implementation handles both pagination methods consistently. No changes are needed.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for pagination-related configurations and usages
rg -l 'OffsetPaginatedQuery|PageSize|DefaultPageSize' | \
xargs -I {} rg -A 5 -B 5 '(Offset|Paginated|PageSize)' {}
Length of output: 67174
Script:
#!/bin/bash
# Search for cursor encoding/decoding implementations and usages
ast-grep --pattern 'bunpaginate.EncodeCursor($$$)'
ast-grep --pattern 'bunpaginate.UnmarshalCursor($$$)'
# Search for offset pagination related code
ast-grep --pattern 'type OffsetPaginatedQuery'
ast-grep --pattern 'type ColumnPaginatedQuery'
# Search for cursor-related test cases
rg -A 5 -B 5 'cursor.*compatibility|compatibility.*cursor'
Length of output: 473
Script:
#!/bin/bash
# Search for pagination-related code in the specific test file and its implementation
rg -A 10 -B 10 'OffsetPaginatedQuery|ColumnPaginatedQuery|bunpaginate' internal/api/v1/controllers_accounts_list_test.go
# Search for pagination-related types and interfaces
ast-grep --pattern 'type $_ struct {
$$$
PageSize $_
$$$
}'
# Look for cursor encoding/decoding in the bunpaginate package
fd bunpaginate --exec rg -l 'Encode|Unmarshal|Cursor'
Length of output: 29852
internal/api/v2/controllers_accounts_count_test.go (2)
29-29: LGTM: Query type update maintains test coverage
The transition to ResourceQuery[any] simplifies the query structure while maintaining comprehensive test coverage for the nominal case.
Also applies to: 40-43
50-54: LGTM: Consistent query construction pattern
The test cases for metadata, address, balance, and exists filters follow a consistent pattern:
- Setting PIT
- Defining specific Builder
- Initializing empty Expand slice
Also applies to: 60-63, 70-74, 80-84
internal/storage/ledger/moves_test.go (2)
181-188: LGTM: Accurate volume assertions
The assertions correctly verify the expected aggregated volumes with balanced input and output values (1000 each) for the high concurrency test case.
174-188: Consider adding more assertions for concurrent operations
While the test verifies the final state, consider adding assertions for:
- No duplicate transactions
- Consistent transaction ordering
- No orphaned moves
Example verification:
internal/api/v2/controllers_transactions_count_test.go (3)
32-32: LGTM: Type change enhances flexibility
The transition to ResourceQuery[any] aligns with the refactoring goal and provides better type flexibility while maintaining type safety.
43-46: LGTM: Improved query construction pattern
The new structure with direct field initialization is more explicit and maintainable than the previous builder pattern.
163-164: Verify type safety in mock expectations
The mock expectation for CountTransactions should be verified to ensure type safety with the new ResourceQuery[any] parameter.
✅ Verification successful
Based on the search results, I can now verify the type safety concern. The CountTransactions method is consistently defined across the codebase to use ResourceQuery[any] as its parameter type:
- In the controller interface:
CountTransactions(ctx context.Context, query ResourceQuery[any]) (int, error)- In the mock implementation used in the test:
func (m *LedgerController) CountTransactions(ctx context.Context, query ledger0.ResourceQuery[any]) (int, error)The mock expectation in the test file is using the correct type parameter with ResourceQuery[any], and the implementation matches the interface definition. Therefore, type safety is maintained.
Mock expectation is type-safe
The CountTransactions mock expectation is correctly typed with ResourceQuery[any] parameter, matching the interface definition and implementation across the codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for other occurrences of CountTransactions to ensure consistent typing
ast-grep --pattern 'CountTransactions($_, $query)'
Length of output: 52
Script:
#!/bin/bash
# Let's try a different approach to find CountTransactions method definition and usages
rg "CountTransactions" -A 3 -B 3
Length of output: 60695
internal/storage/ledger/legacy/balances_test.go (2)
6-6: LGTM: Improved architectural separation
Moving query creation to the store layer with ledgerstore import is a better architectural decision.
77-77: LGTM: Consistent query construction
The transition to ledgerstore.NewGetAggregatedBalancesQuery is applied consistently across all test cases while maintaining the same test coverage.
Also applies to: 86-89, 98-101
internal/api/v2/controllers_volumes_test.go (2)
34-34: LGTM: Enhanced type safety with specific pagination
The transition to OffsetPaginatedQuery[GetVolumesOptions] provides better type safety and clarity for volume-specific queries.
149-151: Verify mock expectations with new query type
Ensure the mock expectations for GetVolumesWithBalances handle the new OffsetPaginatedQuery type correctly.
✅ Verification successful
Based on the search results, I can now verify the mock expectations. The code shows that GetVolumesWithBalances consistently uses OffsetPaginatedQuery[GetVolumesOptions] as its query parameter type across the codebase, particularly in:
- The interface definition in
internal/controller/ledger/controller.go - The mock implementations in various test files
- The actual implementation in
internal/controller/ledger/controller_default.go
The mock expectations in the test file are correctly using this type, so I can conclude that the verification is successful.
Mock expectations correctly handle OffsetPaginatedQuery type
The mock expectations for GetVolumesWithBalances are properly set up to handle the OffsetPaginatedQuery type, which is consistent with the interface definition and implementations throughout the codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for other occurrences of GetVolumesWithBalances to ensure consistent typing
ast-grep --pattern 'GetVolumesWithBalances($_, $query)'
Length of output: 57
Script:
#!/bin/bash
# Let's try a broader search pattern to find the method definition and its usages
rg -A 5 "GetVolumesWithBalances"
# Also search for the interface definition
rg -A 5 "interface.*\{.*GetVolumesWithBalances"
Length of output: 59943
internal/storage/ledger/legacy/volumes.go (1)
Line range hint 168-187: LGTM! Clean and well-structured implementation.
The function effectively handles pagination and query building while maintaining good error handling.
internal/storage/ledger/logs_test.go (1)
8-8: LGTM! Well-structured test updates.
The test cases have been properly updated to use the new pagination structure, maintaining good coverage of functionality.
Also applies to: 124-127, 185-208
internal/storage/ledger/legacy/accounts.go (1)
Line range hint 147-167: LGTM! Clean implementation of GetAccountsWithVolumes.
The function effectively handles pagination and maintains good error handling.
internal/storage/ledger/legacy/transactions.go (1)
171-176: Verify type casting safety.
The type casting of the query options should be verified to ensure it's safe across all possible scenarios.
✅ Verification successful
Let me gather more information about the type casting implementation to ensure its safety.
Let me try a different approach to verify the type casting safety.
Based on the code analysis, I can now provide a final response:
Type casting in paginateWithColumn is safe and well-implemented
The type casting is safe because:
- The function is properly generic with type parameters
[FILTERS any, RETURN any] - The cast is performed on a known type
bunpaginate.ColumnPaginatedQuery[FILTERS] - The conversion is handled through Go's type system with proper generic constraints
- The function includes error handling for the pagination operation
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for all usages of type casting with PaginatedQueryOptions
rg -A 2 "PaginatedQueryOptions\[.*\]"
Length of output: 13383
Script:
#!/bin/bash
# Search for the paginateWithColumn function implementation
ast-grep --pattern 'func paginateWithColumn[$$$]'
# Search for any potential type conversion errors or panics related to PaginatedQueryOptions
rg -A 3 "panic.*PaginatedQueryOptions|interface conversion.*PaginatedQueryOptions"
# Search for the implementation of bunpaginate.ColumnPaginatedQuery
ast-grep --pattern 'type ColumnPaginatedQuery[$$$]'
Length of output: 190
Script:
#!/bin/bash
# Search for the paginateWithColumn function with a more specific pattern
ast-grep --pattern 'func paginateWithColumn[T, M any]'
# Look for the actual implementation files related to pagination
fd -e go -x cat {} \; | rg -A 10 "func paginateWithColumn"
# Search for any error handling around the type casting
rg -A 5 "type assertion|type conversion"
Length of output: 617
internal/controller/ledger/controller_with_traces.go (1)
104-108: LGTM! New method follows established patterns.
The new GetVolumesWithBalances method is well-implemented, following the established tracing patterns and type conventions.
internal/api/v1/controllers_transactions_list_test.go (1)
30-46: LGTM! Clean migration to generic query types.
The transition to ColumnPaginatedQuery[any] with ResourceQuery[any] options is well-structured and maintains type safety.
internal/api/v2/controllers_accounts_list_test.go (1)
Line range hint 32-194: LGTM! Well-structured migration to generic query types.
The transition to OffsetPaginatedQuery[any] is clean and maintains proper type safety while preserving PIT functionality.
internal/storage/ledger/legacy/transactions_test.go (1)
9-9: LGTM! Clean package migration.
The transition from ledgercontroller to ledgerstore package is well-structured.
internal/storage/ledger/legacy/accounts_test.go (2)
73-75: LGTM: Query refactoring improves type safety
The transition to using ledgerstore.NewListAccountsQuery with PITFilterWithVolumes provides better type safety and clearer intent.
89-93: Verify the PIT (Point-in-Time) filter implementation
The test case correctly validates PIT filtering functionality, but we should ensure consistent behavior across different time zones.
✅ Verification successful
Based on the gathered information, I can now provide a final response:
PIT filter implementation is timezone-safe
The implementation correctly handles timestamps in a timezone-safe manner because:
- It uses Go's
time.Timetype which inherently maintains timezone information - The PIT filtering is implemented using direct timestamp comparison (
<=) in SQL - The test setup uses
time.Now()and relative time calculations withtime.Duration, preserving the timezone context
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for timezone-related handling in PIT filter implementation
# Test: Search for timezone handling in PIT filter implementation
ast-grep --pattern 'func $_(pit $_, $_) {
$$$
time.$_($$$)
$$$
}'
Length of output: 73
Script:
#!/bin/bash
# Let's search for PIT-related code implementations and time handling
rg -A 5 "type.*PITFilter"
rg -A 5 "GetAccountsWithVolumes.*PIT"
Length of output: 13424
Script:
#!/bin/bash
# Let's check test files for timezone-related test cases
rg -A 5 "time\." internal/storage/ledger/legacy/accounts_test.go
Length of output: 2806
Script:
#!/bin/bash
# Let's look for the actual implementation of GetAccountsWithVolumes
ast-grep --pattern 'func $_(GetAccountsWithVolumes) {
$$$
}'
Length of output: 64
Script:
#!/bin/bash
# Let's look for the implementation with a different pattern
rg -A 10 "func.*GetAccountsWithVolumes"
Length of output: 850
Script:
#!/bin/bash
# Let's check how PIT is used in SQL queries
rg -A 5 "pit.*time" internal/storage/ledger/legacy/
Length of output: 1417
internal/controller/ledger/controller_default_test.go (1)
203-212: LGTM: Mock implementation enhances type safety
The transition to using strongly-typed mocks with generic types improves type safety and makes the tests more maintainable.
internal/controller/ledger/controller_generated_test.go (1)
Line range hint 72-83: LGTM: Generated mock interfaces maintain consistency
The generated mock interfaces correctly implement the generic type parameters and maintain consistency across all methods. The transition to using ResourceQuery[any] and specific paginated query types is properly reflected in the generated code.
Note: This is an auto-generated file. Any changes should be made to the source interfaces rather than this file directly.
Also applies to: 87-98, 162-173, 177-188, 222-233, 237-248, 281-292, 296-307, 311-322
internal/api/v2/mocks_ledger_controller_test.go (2)
178-178: Verify type constraints for specialized queries
The specialized queries (GetAggregatedBalances and GetVolumesWithBalances) correctly maintain type safety by using concrete option types rather than any.
Let's verify the option types are consistently used:
Also applies to: 238-238
✅ Verification successful
Based on the search results, I can see that both GetAggregatedVolumesOptions and GetVolumesOptions are consistently used with their respective query types (ResourceQuery and OffsetPaginatedQuery) throughout the codebase. Let me verify the type definitions to ensure complete type safety.
Type constraints for specialized queries are properly implemented
The verification confirms that:
- Both
GetAggregatedVolumesOptionsandGetVolumesOptionsare properly defined concrete types with specific fields - These types are consistently used with their respective query types (
ResourceQueryandOffsetPaginatedQuery) across the codebase - No instances of using
anyor other generic types were found for these queries
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify consistent usage of volume option types
# Test: Search for usage of GetAggregatedVolumesOptions and GetVolumesOptions. Expect: Consistent usage with ResourceQuery/OffsetPaginatedQuery.
rg -t go "GetAggregatedVolumesOptions|GetVolumesOptions"
Length of output: 20623
Script:
#!/bin/bash
# Search for the type definitions of GetAggregatedVolumesOptions and GetVolumesOptions
ast-grep --pattern 'type GetAggregatedVolumesOptions struct {
$$$
}'
ast-grep --pattern 'type GetVolumesOptions struct {
$$$
}'
Length of output: 632
73-73: Consistent refactoring to generic query types
The changes to method signatures follow a clear pattern:
- Basic queries use
ResourceQuery[any] - List operations use appropriate pagination types
- Specific operations maintain type safety with concrete option types
This refactoring improves code maintainability while preserving type safety.
Let's verify the consistency of these changes across the codebase:
Also applies to: 88-88, 163-163, 178-178, 223-223, 238-238, 282-282, 297-297, 312-312
✅ Verification successful
Let me verify the usage of these query types with one more check to understand the context better.
Migration to generic query types is correctly implemented
The verification confirms that:
- The old specific query types (
GetAccountQuery,ListTransactionsQuery, etc.) are only present in the legacy code underinternal/storage/ledger/legacy/. - The v2 implementation consistently uses the new generic types:
ResourceQuery[any]for basic queriesOffsetPaginatedQuery[T]for offset-based paginationColumnPaginatedQuery[T]for column-based pagination- Specific types like
ResourceQuery[GetAggregatedVolumesOptions]where type safety is needed
The changes in the mock implementation align perfectly with the v2 API's usage pattern, maintaining both type safety and consistency.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for any remaining specific query types that should have been migrated
# Test: Search for old query type usage. Expect: No results as all should be migrated.
rg -t go "Get(Account|Transaction)Query|List(Accounts|Transactions)Query|GetLogsQuery"
Length of output: 9702
Script:
#!/bin/bash
# Check the v2 implementation to confirm the migration pattern
rg -t go "ResourceQuery|OffsetPaginatedQuery|ColumnPaginatedQuery" internal/api/v2/
Length of output: 16489
internal/api/v1/mocks_ledger_controller_test.go (1)
Line range hint 73-312: Implementation matches v2 package
The changes in this file mirror those in the v2 package, maintaining consistency across API versions.
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint 73-312: Implementation matches v1 and v2 packages
The changes in this file mirror those in the v1 and v2 packages, maintaining consistency across all API implementations.
internal/api/bulking/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! The mock implementation correctly reflects the interface changes.
The changes to method signatures consistently replace specific query types with more generic ones:
- Resource queries:
ResourceQuery[any]for basic queries - Paginated queries:
OffsetPaginatedQuery[any]andColumnPaginatedQuery[any]for list operations - Specialized queries:
ResourceQuery[ledger0.GetAggregatedVolumesOptions]for aggregated balances
✅ Verification successful
The mock implementation correctly matches the source interface
After comparing the mock implementation with the source interface in internal/controller/ledger/controller.go, I can confirm that:
- All method signatures in the mock match the interface exactly
- The generic query types are correctly implemented:
ResourceQuery[any]for basic queries (GetAccount, CountAccounts, etc.)OffsetPaginatedQuery[any]for list operations (ListAccounts)ColumnPaginatedQuery[any]for log/transaction lists- Specialized queries like
ResourceQuery[GetAggregatedVolumesOptions]for specific operations
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that the mock file is generated correctly from the source interface.
# Test: Check if the mock file is up-to-date with the source interface.
# Get the source interface file
cat ../../controller/ledger/controller.go
# Get the mockgen command used to generate this file
head -n 5 mocks_ledger_controller_test.go
Length of output: 248
Script:
#!/bin/bash
# Let's first find the correct paths using fd
fd -t f "controller\.go"
fd -t f "mocks_ledger_controller_test.go"
# Then let's use ast-grep to find the interface definition
ast-grep --pattern 'type Controller interface {
$$$
}'
Length of output: 8227
internal/storage/ledger/legacy/volumes_test.go (1)
8-8: LGTM! Test coverage is comprehensive and changes are consistent.
The test file has been successfully updated to use the new store-level types while maintaining the same test scenarios. The changes properly validate:
- Basic volume queries with both insertion and effective dates
- PIT/OOT filtering
- Metadata filtering
- Volume aggregation at different group levels (0-3)
- Complex filter combinations
Also applies to: 104-665
✅ Verification successful
Let me analyze the test coverage further by checking the actual test implementations.
Based on the analysis of the test implementations, I can now generate the final response:
Test coverage is indeed comprehensive and changes are consistent
The test suite thoroughly covers volume-related functionality across multiple layers:
- Core volume operations (TestUpdateVolumes)
- Volume listing with PIT/OOT filtering (TestVolumesList)
- Volume aggregation with different grouping levels (TestVolumesAggregate)
- Transaction volume updates and retrieval (TestTransactionsVolumesUpdate, TestGetTransactionWithVolumes)
- API endpoints (TestGetVolumes)
- Controller layer (TestGetVolumesWithBalances)
The test implementations verify the same scenarios in both legacy and new code paths, ensuring consistent behavior during the transition.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify test coverage for volume-related functionality
# Test: Check for any untested volume-related functions
# Find all volume-related functions in the codebase
rg -t go "func.*Volume.*\(" --no-filename
# Find corresponding test coverage in test files
rg -t go "func Test.*Volume.*\(" --no-filename
Length of output: 12086
Script:
#!/bin/bash
# Check the implementation of volume-related test functions to verify coverage
rg -t go "func Test.*Volume.*\(" -A 10
Length of output: 8144
internal/README.md (1)
211-221: LGTM! Documentation is properly updated.
The new AggregatedVolumes type is well-documented and properly integrated into the existing documentation structure. The type definition clearly shows:
- The struct name and purpose
- The
Aggregatedfield of typeVolumesByAssets - The bun tag for JSON serialization
| func getColumnPaginatedQuery[v any](r *http.Request, column string, order bunpaginate.Order, modifiers ...func(*v) error) (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| return bunpaginate.Extract[ledgercontroller.ColumnPaginatedQuery[v]](r, func() (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| rq, err := getResourceQuery[v](r, modifiers...) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| pageSize, err := bunpaginate.GetPageSize(r, bunpaginate.WithMaxPageSize(MaxPageSize), bunpaginate.WithDefaultPageSize(DefaultPageSize)) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return &ledgercontroller.ColumnPaginatedQuery[v]{ | ||
| PageSize: pageSize, | ||
| Column: column, | ||
| Order: pointer.For(order), | ||
| Options: *rq, | ||
| }, nil | ||
| }) | ||
| } |
| panic("unreachable") | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid using panic; return an error instead
The use of panic("unreachable") in the resolveFilter method can lead to application crashes. It's preferable to return an error to handle unexpected cases gracefully.
Apply this diff:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unhandled property: %s", property)Committable suggestion skipped: line range outside the PR's diff.
| panic("unreachable") | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid using panic; return an error instead
Using panic("unreachable") in the resolveFilter method may cause the application to crash unexpectedly. It's better to return an error to handle unexpected cases gracefully.
Apply this diff:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unhandled property: %s", property)Committable suggestion skipped: line range outside the PR's diff.
| if hasMore { | ||
| cp := query | ||
| cp.PaginationID = paginationIDs[len(paginationIDs)-2] | ||
| previous = &cp | ||
| } |
There was a problem hiding this comment.
Fix potential index out of range errors when accessing pagination IDs
There is a risk of an index out of range panic when accessing paginationIDs[len(paginationIDs)-2] and paginationIDs[len(paginationIDs)-1]. Ensure that the length of paginationIDs is sufficient before accessing these indices.
Apply this diff to handle the issue:
// Ensure there are at least two pagination IDs before accessing len(paginationIDs) - 2
if hasMore && len(paginationIDs) >= 2 {
cp := query
cp.PaginationID = paginationIDs[len(paginationIDs)-2]
previous = &cp
}
// Ensure there's at least one pagination ID before accessing len(paginationIDs) - 1
if hasMore && len(paginationIDs) >= 1 {
cp := query
cp.PaginationID = paginationIDs[len(paginationIDs)-1]
next = &cp
}Also applies to: 119-121
| // Page with transactions after | ||
| if query.PageSize != 0 && len(ret) > int(query.PageSize) { | ||
| cp := query | ||
| cp.Offset = query.Offset + query.PageSize | ||
| next = &cp | ||
| ret = ret[:len(ret)-1] | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add boundary checks for next page calculation
The next page calculation should include additional boundary checks to prevent potential issues with large numbers.
if query.PageSize != 0 && len(ret) > int(query.PageSize) {
cp := query
+ // Check for potential overflow
+ if query.Offset > math.MaxUint64 - query.PageSize {
+ return nil, fmt.Errorf("offset overflow")
+ }
cp.Offset = query.Offset + query.PageSize
next = &cp
ret = ret[:len(ret)-1]
}Committable suggestion skipped: line range outside the PR's diff.
gfyrag
force-pushed
the
refactor/store-read-part
branch
from
d7b4dac to
0985fc3
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (19)
internal/storage/ledger/legacy/adapters.go (1)
21-21: Address the TODO: Handle compatibility with v1.The comment
// todo; handle compat with v1suggests pending work on version compatibility. Please implement the necessary compatibility logic or remove the comment if it's no longer relevant.internal/storage/ledger/resource_transactions.go (1)
47-49: Add validators for the "timestamp" filter.There's a
// todo: add validatorscomment for the "timestamp" filter. Implement appropriate validators to ensure that timestamp values are correctly parsed and validated.internal/controller/ledger/store_generated_test.go (1)
45-57: Mock methods should include expectations in testsThe newly added
Accountsmethod inMockStoreshould have corresponding expectations set up in unit tests to ensure it behaves as intended.internal/api/v2/controllers_balances.go (1)
15-19: Consider consolidating parameter naming convention.The code handles both
use_insertion_dateanduseInsertionDateparameters for backward compatibility. Consider standardizing on one format and deprecating the other.- options.UseInsertionDate = api.QueryParamBool(r, "use_insertion_date") || api.QueryParamBool(r, "useInsertionDate") + const ( + deprecatedParam = "useInsertionDate" + currentParam = "use_insertion_date" + ) + if api.QueryParamBool(r, deprecatedParam) { + // TODO: Add deprecation warning in logs + } + options.UseInsertionDate = api.QueryParamBool(r, currentParam) || api.QueryParamBool(r, deprecatedParam)internal/api/v2/controllers_transactions_read.go (1)
30-34: Consider extracting query construction to a helper function.The inline query construction could be moved to a helper function to improve reusability and testability.
+func newTransactionQuery(txID int64, pit *time.Time, expand []string) ledgercontroller.ResourceQuery[any] { + return ledgercontroller.ResourceQuery[any]{ + PIT: pit, + Builder: query.Match("id", txID), + Expand: expand, + } +} func readTransaction(w http.ResponseWriter, r *http.Request) { // ... existing code ... - tx, err := l.GetTransaction(r.Context(), ledgercontroller.ResourceQuery[any]{ - PIT: pit, - Builder: query.Match("id", int(txId)), - Expand: r.URL.Query()["expand"], - }) + tx, err := l.GetTransaction(r.Context(), newTransactionQuery(txId, pit, r.URL.Query()["expand"]))internal/api/v1/controllers_logs_list.go (1)
44-46: Consider adding validation for the query builder result.While the implementation looks good, consider adding validation for the query builder result before assigning it to
paginatedQuery.Options.Builder.- paginatedQuery.Options.Builder = buildGetLogsQuery(r) + builder := buildGetLogsQuery(r) + if builder != nil { + paginatedQuery.Options.Builder = builder + }internal/api/v1/controllers_balances_aggregates_test.go (1)
44-49: Consider adding test cases for edge cases.While the implementation looks good, consider adding test cases for:
- Empty address
- Invalid address format
- Multiple addresses
internal/api/v2/common.go (1)
64-80: Consider adding validation for negative page sizesWhile the code handles max page size, it might be worth adding validation to ensure page sizes are not negative.
func getOffsetPaginatedQuery[v any](r *http.Request, modifiers ...func(*v) error) (*ledgercontroller.OffsetPaginatedQuery[v], error) { return bunpaginate.Extract[ledgercontroller.OffsetPaginatedQuery[v]](r, func() (*ledgercontroller.OffsetPaginatedQuery[v], error) { rq, err := getResourceQuery[v](r, modifiers...) if err != nil { return nil, err } pageSize, err := bunpaginate.GetPageSize(r, bunpaginate.WithMaxPageSize(MaxPageSize), bunpaginate.WithDefaultPageSize(DefaultPageSize)) if err != nil { return nil, err } + if pageSize < 0 { + return nil, fmt.Errorf("page size cannot be negative") + } return &ledgercontroller.OffsetPaginatedQuery[v]{ PageSize: pageSize, Options: *rq, }, nil }) }internal/api/v2/controllers_accounts_count_test.go (1)
98-101: Consider consolidating duplicate ResourceQuery initializationThe error test cases contain identical ResourceQuery initialization code. Consider extracting a helper function to reduce duplication:
+func newDefaultResourceQuery(pit *time.Time) ledgercontroller.ResourceQuery[any] { + return ledgercontroller.ResourceQuery[any]{ + PIT: pit, + Expand: make([]string, 0), + } +}Then use it in the test cases:
-expectQuery: ledgercontroller.ResourceQuery[any]{ - PIT: &before, - Expand: make([]string, 0), -}, +expectQuery: newDefaultResourceQuery(&before),Also applies to: 109-112, 120-123
internal/api/v2/controllers_transactions_count_test.go (1)
Line range hint
121-150: Enhance error case coverageThe error test cases cover basic scenarios but could be expanded to include:
- Invalid PIT format
- Malformed query builders
- Edge cases in error responses
internal/api/v2/controllers_volumes_test.go (1)
54-61: Consider adding test cases for edge casesWhile the current test cases cover the basic scenarios well, consider adding tests for:
- Maximum page size limits
- Empty or invalid groupBy values
- Malformed metadata filters
Also applies to: 66-73, 87-96, 113-120
internal/storage/ledger/resource_aggregated_balances.go (1)
38-70: Consider extracting SQL query buildersThe
buildDatasetmethod contains complex SQL query logic. Consider extracting the query builders into separate methods for better maintainability and testability.+func (h aggregatedBalancesResourceRepositoryHandler) buildPITQuery(store *Store, ledger ledger.Ledger, query ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { + ret := store.db.NewSelect(). + ModelTableExpr(store.GetPrefixedRelationName("moves")). + DistinctOn("accounts_address, asset"). + Column("accounts_address", "asset"). + Where("ledger = ?", ledger.Name) + + if query.Opts.UseInsertionDate { + return h.buildInsertionDateQuery(store, ledger, ret, query) + } + return h.buildEffectiveDateQuery(store, ledger, ret, query) +} func (h aggregatedBalancesResourceRepositoryHandler) buildDataset(store *Store, ledger ledger.Ledger, query ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { if query.PIT != nil && !query.PIT.IsZero() { - ret := store.db.NewSelect(). - ModelTableExpr(store.GetPrefixedRelationName("moves")). - DistinctOn("accounts_address, asset"). - Column("accounts_address", "asset"). - Where("ledger = ?", ledger.Name) - // ... rest of the PIT query logic + return h.buildPITQuery(store, ledger, query) } else { return store.db.NewSelect(). ModelTableExpr(store.GetPrefixedRelationName("accounts_volumes")). Column("asset", "accounts_address"). ColumnExpr("(input, output)::"+store.GetPrefixedRelationName("volumes")+" as volumes"). Where("ledger = ?", ledger.Name), nil } }internal/api/v2/controllers_logs_list_test.go (1)
122-131: Consider consolidating error test casesThe error test cases have duplicate query setup code. Consider using a helper function to create the base query configuration.
+func createBaseQuery() ledgercontroller.ColumnPaginatedQuery[any] { + return ledgercontroller.ColumnPaginatedQuery[any]{ + PageSize: DefaultPageSize, + Column: "id", + Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), + Options: ledgercontroller.ResourceQuery[any]{ + Expand: make([]string, 0), + }, + } +} // In test cases: -expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ - PageSize: DefaultPageSize, - Column: "id", - Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), - Options: ledgercontroller.ResourceQuery[any]{ - Expand: make([]string, 0), - }, -}, +expectQuery: createBaseQuery(),Also applies to: 137-146
internal/storage/ledger/store.go (1)
130-136: Improve error message clarity in validateAddressFilterThe error message for invalid address filter could be more descriptive.
- return fmt.Errorf("invalid 'address' filter") + return fmt.Errorf("invalid 'address' filter: expected string value, got %T", value)internal/controller/ledger/store.go (2)
179-180: Address TODO comment regarding Query methodThe TODO comment indicates this method should be removed quickly. Consider creating a timeline for its removal and documenting any dependencies.
Would you like me to help create a GitHub issue to track the removal of this method?
190-191: Address TODO comment regarding backportingThe TODO comment indicates a need to backport to go-libs. This should be tracked.
Would you like me to help create a GitHub issue to track the backporting task?
internal/controller/ledger/controller_with_traces.go (1)
Line range hint
198-208: LGTM! The query type refactoring improves code flexibility.The consistent replacement of specific query types with generic ones (
ResourceQuery[any],ColumnPaginatedQuery[any],OffsetPaginatedQuery[any]) across all controller methods is a good architectural improvement that:
- Reduces code duplication
- Increases code reusability
- Makes the query system more flexible and extensible
- Maintains type safety through generics
Consider documenting these query types in a central location to help other developers understand when to use each type:
ResourceQuery[T]: For single resource queriesColumnPaginatedQuery[T]: For column-based paginationOffsetPaginatedQuery[T]: For offset-based paginationAlso applies to: 210-220, 222-232, 234-244, 246-256, 258-268, 270-280, 282-292, 330-340
internal/controller/ledger/controller_generated_test.go (1)
Line range hint
72-82: LGTM! Generated mock correctly implements the new query types.The mock implementation correctly reflects the controller interface changes. Being an auto-generated file, the changes are consistent with the interface updates.
Consider adding a note in the repository documentation about regenerating this file when the controller interface changes.
Also applies to: 87-97, 162-172, 177-187, 222-232, 237-247, 281-291, 296-306, 311-321
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint
73-312: Consider documenting the query type hierarchy.The refactoring introduces a well-structured hierarchy of query types that standardizes resource access patterns. To help maintainers and contributors, consider:
Adding documentation that explains the purpose and use cases for each query type:
ResourceQuery[any]for simple resource queriesOffsetPaginatedQuery[T]for offset-based paginationColumnPaginatedQuery[T]for column-based paginationCreating examples demonstrating when to use each type
This will help ensure consistent usage across the codebase and make it easier for new contributors to understand the pattern.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
go.modis excluded by!**/*.modgo.sumis excluded by!**/*.sum,!**/*.sumtools/generator/go.sumis excluded by!**/*.sum,!**/*.sum
📒 Files selected for processing (82)
internal/README.md(2 hunks)internal/api/bulking/mocks_ledger_controller_test.go(9 hunks)internal/api/common/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/controllers_accounts_count.go(1 hunks)internal/api/v1/controllers_accounts_count_test.go(6 hunks)internal/api/v1/controllers_accounts_list.go(1 hunks)internal/api/v1/controllers_accounts_list_test.go(5 hunks)internal/api/v1/controllers_accounts_read.go(2 hunks)internal/api/v1/controllers_accounts_read_test.go(4 hunks)internal/api/v1/controllers_balances_aggregates.go(1 hunks)internal/api/v1/controllers_balances_aggregates_test.go(1 hunks)internal/api/v1/controllers_balances_list.go(1 hunks)internal/api/v1/controllers_logs_list.go(1 hunks)internal/api/v1/controllers_logs_list_test.go(3 hunks)internal/api/v1/controllers_transactions_count.go(1 hunks)internal/api/v1/controllers_transactions_count_test.go(3 hunks)internal/api/v1/controllers_transactions_list.go(1 hunks)internal/api/v1/controllers_transactions_list_test.go(4 hunks)internal/api/v1/controllers_transactions_read.go(2 hunks)internal/api/v1/controllers_transactions_read_test.go(2 hunks)internal/api/v1/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/utils.go(1 hunks)internal/api/v2/common.go(2 hunks)internal/api/v2/controllers_accounts_count.go(1 hunks)internal/api/v2/controllers_accounts_count_test.go(4 hunks)internal/api/v2/controllers_accounts_list.go(1 hunks)internal/api/v2/controllers_accounts_list_test.go(5 hunks)internal/api/v2/controllers_accounts_read.go(2 hunks)internal/api/v2/controllers_accounts_read_test.go(4 hunks)internal/api/v2/controllers_balances.go(1 hunks)internal/api/v2/controllers_balances_test.go(2 hunks)internal/api/v2/controllers_logs_list.go(1 hunks)internal/api/v2/controllers_logs_list_test.go(5 hunks)internal/api/v2/controllers_transactions_count.go(1 hunks)internal/api/v2/controllers_transactions_count_test.go(4 hunks)internal/api/v2/controllers_transactions_list.go(1 hunks)internal/api/v2/controllers_transactions_list_test.go(4 hunks)internal/api/v2/controllers_transactions_read.go(2 hunks)internal/api/v2/controllers_transactions_read_test.go(2 hunks)internal/api/v2/controllers_volumes.go(1 hunks)internal/api/v2/controllers_volumes_test.go(4 hunks)internal/api/v2/mocks_ledger_controller_test.go(9 hunks)internal/controller/ledger/controller.go(1 hunks)internal/controller/ledger/controller_default.go(4 hunks)internal/controller/ledger/controller_default_test.go(10 hunks)internal/controller/ledger/controller_generated_test.go(9 hunks)internal/controller/ledger/controller_with_traces.go(9 hunks)internal/controller/ledger/stats.go(1 hunks)internal/controller/ledger/stats_test.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/controller/ledger/store_generated_test.go(5 hunks)internal/storage/ledger/accounts.go(2 hunks)internal/storage/ledger/accounts_test.go(7 hunks)internal/storage/ledger/balances.go(0 hunks)internal/storage/ledger/balances_test.go(1 hunks)internal/storage/ledger/errors.go(0 hunks)internal/storage/ledger/legacy/accounts.go(6 hunks)internal/storage/ledger/legacy/accounts_test.go(19 hunks)internal/storage/ledger/legacy/adapters.go(2 hunks)internal/storage/ledger/legacy/balances.go(1 hunks)internal/storage/ledger/legacy/balances_test.go(8 hunks)internal/storage/ledger/legacy/logs.go(1 hunks)internal/storage/ledger/legacy/logs_test.go(2 hunks)internal/storage/ledger/legacy/queries.go(1 hunks)internal/storage/ledger/legacy/transactions.go(6 hunks)internal/storage/ledger/legacy/transactions_test.go(8 hunks)internal/storage/ledger/legacy/volumes.go(4 hunks)internal/storage/ledger/legacy/volumes_test.go(27 hunks)internal/storage/ledger/logs.go(0 hunks)internal/storage/ledger/logs_test.go(3 hunks)internal/storage/ledger/moves.go(0 hunks)internal/storage/ledger/moves_test.go(1 hunks)internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)internal/storage/ledger/store.go(4 hunks)
⛔ Files not processed due to max files limit (7)
- internal/storage/ledger/transactions.go
- internal/storage/ledger/transactions_test.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/volumes.go
- internal/storage/ledger/volumes_test.go
- internal/volumes.go
- test/e2e/api_transactions_list_test.go
💤 Files with no reviewable changes (4)
- internal/storage/ledger/errors.go
- internal/storage/ledger/logs.go
- internal/storage/ledger/moves.go
- internal/storage/ledger/balances.go
🚧 Files skipped from review as they are similar to previous changes (39)
- internal/storage/ledger/paginator.go
- internal/api/v1/controllers_accounts_count.go
- internal/README.md
- internal/controller/ledger/stats.go
- internal/storage/ledger/legacy/balances_test.go
- internal/api/v1/controllers_transactions_read_test.go
- internal/storage/ledger/legacy/logs_test.go
- internal/api/v1/controllers_accounts_read.go
- internal/api/v1/controllers_transactions_count.go
- internal/api/v2/controllers_accounts_list.go
- internal/api/v1/controllers_accounts_list.go
- internal/api/v2/controllers_accounts_count.go
- internal/api/v2/controllers_transactions_read_test.go
- internal/api/v1/controllers_transactions_list.go
- internal/api/v1/controllers_balances_aggregates.go
- internal/api/v1/controllers_transactions_read.go
- internal/api/v2/controllers_accounts_read.go
- internal/api/v1/controllers_balances_list.go
- internal/api/v1/controllers_accounts_read_test.go
- internal/api/v2/controllers_transactions_list.go
- internal/api/v2/controllers_transactions_count.go
- internal/api/v2/controllers_volumes.go
- internal/storage/ledger/legacy/balances.go
- internal/api/v1/controllers_accounts_list_test.go
- internal/storage/ledger/paginator_offset.go
- internal/storage/ledger/legacy/volumes.go
- internal/api/v1/controllers_accounts_count_test.go
- internal/storage/ledger/resource_logs.go
- internal/api/v2/controllers_accounts_read_test.go
- internal/storage/ledger/legacy/transactions_test.go
- internal/storage/ledger/legacy/logs.go
- internal/storage/ledger/resource_accounts.go
- internal/api/v2/controllers_accounts_list_test.go
- internal/storage/ledger/legacy/volumes_test.go
- internal/storage/ledger/logs_test.go
- internal/storage/ledger/paginator_column.go
- internal/storage/ledger/legacy/transactions.go
- internal/controller/ledger/controller_default_test.go
- internal/storage/ledger/legacy/accounts.go
🔇 Additional comments (65)
internal/api/v1/utils.go (3)
26-44: Function getOffsetPaginatedQuery is well-implemented.
The function is correctly handling pagination parameters and error checks.
67-79: Function getResourceQuery correctly constructs the resource query.
The function appropriately applies modifiers and handles errors.
46-65:
Sanitize the column parameter to prevent SQL injection vulnerabilities.
The column parameter is used directly in the database query without validation. This could lead to SQL injection if untrusted input is passed. Ensure that column is validated against a whitelist of allowed column names before using it in queries.
Apply the following diff to implement column validation:
func isValidColumnName(column string) bool {
allowedColumns := map[string]bool{
"id": true,
"name": true,
"date": true,
// Add other allowed columns here
}
return allowedColumns[column]
}
func getColumnPaginatedQuery[v any](r *http.Request, column string, order bunpaginate.Order, modifiers ...func(*v) error) (*ledgercontroller.ColumnPaginatedQuery[v], error) {
if !isValidColumnName(column) {
return nil, fmt.Errorf("invalid column name: %s", column)
}
// Rest of the function
}internal/storage/ledger/legacy/queries.go (1)
11-14: Verify the purpose of the OOT field in PITFilter.
The PITFilter struct includes an OOT *time.Time field. Please confirm if OOT is intended, and if so, provide documentation or rename it for clarity. If it's a typo, consider correcting or removing it.
internal/storage/ledger/resource_transactions.go (1)
153-154: 🛠️ Refactor suggestion
Avoid using panic; return an error instead.
Using panic("unreachable") can cause the application to crash unexpectedly. It's better to return an error to handle this situation gracefully.
Apply this diff:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unhandled property: %s", property)internal/storage/ledger/resource_volumes.go (1)
167-168:
Avoid using panic; return an error instead
Using panic("unreachable") in the resolveFilter method may cause the application to crash unexpectedly. It's better to return an error to handle unforeseen cases gracefully.
Apply this diff:
default:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unhandled property: %s", property)
}
...
- panic("unreachable")
+ return "", nil, fmt.Errorf("unhandled property in resolveFilter")Also applies to: 171-172
internal/storage/ledger/resource.go (3)
31-32:
Avoid panicking on unsupported operators; return an error instead
Using panic("unreachable") in convertOperatorToSQL can lead to unexpected application crashes when an unsupported operator is encountered. Consider returning an error to handle such cases gracefully.
Apply this diff:
-func convertOperatorToSQL(operator string) string {
+func convertOperatorToSQL(operator string) (string, error) {
switch operator {
case "$match":
- return "="
+ return "=", nil
case "$lt":
- return "<"
+ return "<", nil
case "$gt":
- return ">"
+ return ">", nil
case "$lte":
- return "<="
+ return "<=", nil
case "$gte":
- return ">="
+ return ">=", nil
}
- panic("unreachable")
+ return "", fmt.Errorf("unsupported operator: %s", operator)
}Update calling code to handle the returned error appropriately.
280-281:
Avoid using panic; handle unexpected cases gracefully
Using panic("should not happen") may lead to application crashes if unexpected pagination options are provided. Return an error instead to handle this scenario.
Apply this diff:
default:
- panic("should not happen")
+ return nil, fmt.Errorf("unexpected pagination options type: %T", paginationOptions)Likely invalid or redundant comment.
87-88:
Handle regex errors without panicking
Using panic(err) when an error occurs in regexp.MatchString can cause unexpected application crashes. It's better to return the error for graceful handling.
Apply this diff:
if found, err = regexp.MatchString("^"+property.name+"$", key); err != nil {
- panic(err)
+ return err
} else if found {Likely invalid or redundant comment.
internal/controller/ledger/controller_default.go (3)
110-112: Addition of IsDatabaseUpToDate method is appropriate
The new IsDatabaseUpToDate method correctly checks the database status by delegating to the store's IsUpToDate function.
118-119: Method signature update aligns with new query structure
The ListTransactions method now accepts ColumnPaginatedQuery[any], reflecting a more generic and flexible query handling approach.
154-155: Ensure proper handling of pagination in GetVolumesWithBalances
The GetVolumesWithBalances method uses OffsetPaginatedQuery[GetVolumesOptions]. Verify that all paginated resources handle the options correctly.
Run the following script to check usage:
✅ Verification successful
Let me gather more information about the pagination implementation.
Let me gather more information about the pagination implementation in the store layer.
Let me check the actual implementation of the pagination in the store layer.
Let me check the implementation of the pagination in the volumes store.
Based on the gathered information, I can now provide a final response:
Pagination is properly implemented in GetVolumesWithBalances
The implementation follows the established pattern used across the codebase:
- Uses the generic
PaginatedResourceinterface with proper type parameters - Correctly delegates to the store layer's
Volumes().Paginate()method - Returns a properly typed
bunpaginate.Cursor - Has comprehensive test coverage as seen in the test files
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that `GetVolumesWithBalances` correctly handles pagination options.
# Search for the method implementation and usage
rg 'func \(.*\) GetVolumesWithBalances\(.*OffsetPaginatedQuery\[GetVolumesOptions\]\)'
Length of output: 816
Script:
#!/bin/bash
# Check the implementation of Volumes().Paginate
ast-grep --pattern 'func ($v *$_) Paginate(ctx context.Context, q $_) (*bunpaginate.Cursor[ledger.VolumesWithBalanceByAssetByAccount], error)'
# Check the store interface and implementation
rg "type.*Volumes.*interface" -A 5
# Check the usage of GetVolumesWithBalances
rg "GetVolumesWithBalances\(" -B 2 -A 2
Length of output: 30486
Script:
#!/bin/bash
# Check the Volumes interface and implementation
rg "type Volumes interface" -A 10
rg "func \(.*\) Paginate\(.*\)" -A 3
Length of output: 1179
Script:
#!/bin/bash
# Check the store implementation for volumes pagination
ast-grep --pattern 'func ($v *$_) Paginate(ctx context.Context, q $_) (*bunpaginate.Cursor[ledger.VolumesWithBalanceByAssetByAccount], error) {
$$$
}'
# Check the bunpaginate package usage
rg "bunpaginate\.Cursor" -B 2 -A 2
Length of output: 54275
Script:
#!/bin/bash
# Check the store implementation for volumes pagination
rg "type Volumes interface" -A 10
rg "func \(.*\) Volumes\(\)" -A 3
Length of output: 1818
internal/controller/ledger/store_generated_test.go (1)
369-548: Ensure new mock interfaces are properly integrated
The introduction of MockResource and MockPaginatedResource interfaces is significant. Confirm that all tests and mock setups are updated to use these new interfaces correctly.
Run the following script to identify usages of old mock interfaces:
✅ Verification successful
Mock interfaces are correctly integrated and in use
The mock interfaces are properly integrated and actively used across test files. The search results show:
- The mock interfaces are defined in
store_generated_test.go - They are actively used in multiple test files:
controller_default_test.gostats_test.go- All usages follow the correct generic type patterns
No issues found with the integration of these mock interfaces.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find usages of old mock interfaces that need to be updated.
rg 'MockStore\.\w+\(' -A 2
Length of output: 28
Script:
#!/bin/bash
# Let's search for any usage of these mock types in test files
rg -t go "Mock(Resource|PaginatedResource)" -A 2
# Also search for any test files that might be using these mocks
fd ".*_test.go$" --exec grep -l "Mock\(Resource\|PaginatedResource\)"
Length of output: 15675
internal/api/v2/controllers_logs_list.go (2)
17-20: LGTM! Clean refactoring of query construction.
The transition to getColumnPaginatedQuery with proper error handling simplifies the code while maintaining functionality.
23-23: Verify the ListLogs implementation compatibility.
The change to use the new query type looks good, but we should verify the ListLogs implementation handles the new query type correctly.
✅ Verification successful
Based on the search results, I can see that the ListLogs implementation is properly handling the new query type. The key findings are:
- The controller interface in
internal/controller/ledger/controller.goshowsListLogsaccepts aColumnPaginatedQuery[any]type. - The implementation in
internal/controller/ledger/controller_default.goshows the method properly handles this query type. - The tests in
internal/api/v2/controllers_logs_list_test.goverify the compatibility. - The method is consistently used across v1 and v2 APIs with proper type handling.
The implementation is compatible with the new query type.
The codebase shows proper integration and type safety across the controller layer and its implementations.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for ListLogs implementation to verify compatibility
ast-grep --pattern 'func $_($$) ListLogs(ctx context.Context, query $_) ($_, error) {
$$$
}'
Length of output: 96
Script:
#!/bin/bash
# Let's try a broader search for ListLogs implementations
rg "ListLogs" -A 5 -B 2
Length of output: 59793
internal/api/v2/controllers_balances.go (1)
25-25: LGTM! Clean integration with the new query type.
The transition to use the new query type with GetAggregatedBalances is clean and maintains proper error handling.
internal/api/v2/controllers_transactions_read.go (1)
24-29: LGTM! Clean error handling for PIT.
The point-in-time query parameter handling is well-structured with proper error validation.
internal/controller/ledger/stats_test.go (2)
18-19: LGTM! Good use of strongly typed mock resources.
The introduction of typed mock resources using NewMockPaginatedResource improves type safety and better reflects the actual implementation.
21-24: Verify the query parameters in Count expectations.
The mock expectations look good, but the empty ResourceQuery[any]{} parameters might need verification to ensure they match the actual query parameters used in production code.
✅ Verification successful
Empty query parameters are correct and consistent with the codebase.
Looking at the implementation in internal/controller/ledger/stats.go, the actual production code also uses empty ResourceQuery[any]{} parameters when calling Count on both transactions and accounts. The test's mock expectations perfectly match the actual implementation.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for actual Count calls to verify query parameters
ast-grep --pattern 'Count($ctx, $query)'
Length of output: 42
Script:
#!/bin/bash
# Let's try a different approach to find Count method usage
rg "Count\(" -A 3 -B 3
Length of output: 15059
internal/api/v1/controllers_logs_list.go (1)
38-41: Good error handling for pagination query.
The error handling is properly implemented with appropriate HTTP status codes and error messages.
internal/api/v1/controllers_balances_aggregates_test.go (1)
27-27: Good use of strongly typed ResourceQuery.
The transition to ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions] improves type safety and better represents the domain model.
Also applies to: 33-36
internal/api/v2/controllers_balances_test.go (2)
31-31: LGTM: Type change aligns with the new resource query pattern
The change from GetAggregatedBalanceQuery to ResourceQuery[GetAggregatedVolumesOptions] follows the broader refactoring effort to use generic resource queries, improving code consistency.
39-42: LGTM: Test cases properly adapted to new query structure
The test cases have been correctly updated to use the new ResourceQuery type, maintaining test coverage while adapting to the new query pattern. Each test case properly initializes the required fields (PIT, Opts, Builder, Expand) according to its specific test scenario.
Also applies to: 48-52, 58-62, 70-73, 82-88
internal/api/v1/controllers_transactions_count_test.go (3)
25-25: LGTM: Simplified query type using ResourceQuery[any]
The change to ResourceQuery[any] aligns with the new query pattern and is appropriate for count operations where specific options aren't needed.
34-34: LGTM: Test cases properly cover various query scenarios
The test cases have been well-structured to cover different query scenarios:
- Basic query without filters
- Metadata filtering
- Time range filtering
- Account filtering
- Reference filtering
- Source/destination filtering
Each case properly constructs the query using the appropriate builder function.
Also applies to: 41-43, 50-52, 59-61, 68-70, 77-79, 86-88, 95-97
111-111: LGTM: Mock expectations updated correctly
The mock expectations have been properly updated to use the new ResourceQuery[any] type.
internal/api/v2/common.go (4)
4-4: Avoid dot imports for better code clarity
Using a dot import for collectionutils can lead to namespace pollution and reduce code clarity.
17-29: LGTM: Well-structured date handling functions
The new getDate function provides a clean and reusable way to parse dates from query parameters:
- Proper error handling
- Returns nil for missing dates
- Reused by
getPITandgetOOTfunctions
83-102: Validate column names to prevent SQL injection
The defaultPaginationColumn parameter should be validated against a whitelist of allowed column names.
104-131: LGTM: Well-structured resource query construction
The getResourceQuery function properly:
- Handles PIT and OOT dates
- Applies query builders
- Manages expand fields
- Supports generic options through modifiers
internal/storage/ledger/accounts.go (1)
53-53: LGTM: Proper timestamp handling added
The addition of Set("updated_at = excluded.updated_at") ensures that the updated_at timestamp is properly maintained during metadata updates.
internal/api/v1/controllers_logs_list_test.go (2)
38-43: LGTM: Well-structured pagination query
The test case properly initializes ColumnPaginatedQuery with appropriate defaults:
- PageSize set to DefaultPageSize
- Column set to "id"
- Order properly set using pointer to descending order
50-57: LGTM: Comprehensive test coverage for time-based filtering
The test cases properly handle both start_time and end_time scenarios with appropriate query builders:
- start_time uses Gte (greater than or equal)
- end_time uses Lt (less than)
Also applies to: 64-71
internal/api/v2/controllers_accounts_count_test.go (1)
40-43: LGTM: Comprehensive test coverage for different query scenarios
The test cases properly cover various query scenarios:
- Basic queries with PIT
- Metadata filtering
- Address filtering
- Balance filtering
- Exists filtering
Each case properly initializes ResourceQuery with appropriate fields.
Also applies to: 50-54, 60-63, 70-74, 80-84
internal/controller/ledger/controller.go (1)
27-35: Consider using a more specific type parameter instead of any
The transition to generic ResourceQuery[any] provides flexibility but might sacrifice type safety. Consider creating a union type or interface that encompasses all valid query parameters instead of using any.
Let's verify the query parameter types being used across the codebase:
internal/storage/ledger/moves_test.go (2)
174-179: Use fixed timestamp in tests for deterministic results
Using time.Now() in tests can lead to flaky behavior.
174-188: Verify concurrent transaction processing
The test creates 1000 concurrent transactions but doesn't explicitly verify the order of processing or potential race conditions.
Let's check for proper concurrency handling:
✅ Verification successful
Let me gather more information about the transaction processing implementation to better understand the concurrency handling.
Let me check one more aspect to understand how transactions are committed and if there's proper database-level concurrency control.
Based on the analysis of the code, I can now generate the final response:
Concurrent transaction processing is properly handled
The implementation shows proper concurrency handling through:
- Database-level transaction isolation using SQL transactions (
BeginTx) - Deadlock detection and retry mechanism (
errors.Is(err, postgres.ErrDeadlockDetected)) - Worker pool limiting concurrent operations (
pond.New(10, 10)) - Proper transaction rollback on errors
The test's high concurrency scenario (1000 transactions) is adequately testing the concurrent processing with appropriate safeguards in place.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for mutex usage and other synchronization mechanisms
rg -A 5 'sync\.Mutex|sync\.RWMutex'
# Look for potential race conditions in transaction processing
ast-grep --pattern 'func.*InsertTransaction'
Length of output: 1864
Script:
#!/bin/bash
# Look for transaction processing implementation
ast-grep --pattern 'func $_($_, $_) error {
$$$
InsertTransaction
$$$
}'
# Check for concurrent test implementations
rg -A 10 "parallel|concurrency|goroutine" internal/storage/ledger/
# Look for transaction processing tests
rg -A 10 "TestInsertTransaction|TestConcurrent" internal/storage/ledger/
Length of output: 4106
Script:
#!/bin/bash
# Look for transaction commit implementation and database handling
rg -A 10 "CommitTransaction|BeginTx" internal/storage/ledger/
# Check for any SQL transaction isolation level settings
rg "sql\.LevelSerializable|sql\.LevelRepeatableRead|IsolationLevel" internal/storage/ledger/
Length of output: 67071
internal/api/v2/controllers_transactions_count_test.go (1)
43-56: Consider adding validation for empty/invalid metadata keys
The test cases cover basic metadata matching but don't verify behavior with empty or invalid metadata keys.
Let's check existing validation:
internal/api/v2/controllers_volumes_test.go (1)
34-34: LGTM: Well-structured test case refactoring
The transition to OffsetPaginatedQuery with explicit field initialization improves test readability and maintainability. The structure clearly shows the expected query parameters.
Also applies to: 43-49
internal/storage/ledger/resource_aggregated_balances.go (2)
119-120: Replace panic with error return
Using panic("unreachable") can lead to application crashes. This is a duplicate of a previous review comment.
46-50: Verify feature flag handling
The code checks for feature flags but we should verify if these checks are consistently applied across the codebase.
Also applies to: 55-57
✅ Verification successful
Feature flag checks are consistently implemented across the codebase
The verification shows that both feature flags (FeatureMovesHistory and FeatureMovesHistoryPostCommitEffectiveVolumes) are consistently checked across relevant files:
resource_aggregated_balances.go: Checks both flags with appropriate error handlingresource_accounts.go: Validates both flags with specific error messagesresource_volumes.go: VerifiesFeatureMovesHistorytransactions.go: Uses both flags to conditionally execute related functionality
The implementation follows a consistent pattern where:
FeatureMovesHistoryis checked for "ON" stateFeatureMovesHistoryPostCommitEffectiveVolumesis checked for "SYNC" state- Appropriate error messages are returned when features are not enabled
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for consistent feature flag usage across the codebase
# Search for feature flag checks
rg -A 2 "HasFeature\(features\.FeatureMovesHistory"
rg -A 2 "HasFeature\(features\.FeatureMovesHistoryPostCommitEffectiveVolumes"
Length of output: 3735
internal/api/v2/controllers_logs_list_test.go (1)
34-34: LGTM: Well-structured pagination query tests
The transition to ColumnPaginatedQuery with explicit column and order parameters improves test clarity and maintainability.
Also applies to: 44-52
internal/api/v1/controllers_transactions_list_test.go (1)
30-30: LGTM! Test cases properly updated for the new query structure
The test cases have been correctly updated to use the new ColumnPaginatedQuery[any] type, maintaining good test coverage for the refactored functionality.
Also applies to: 38-46
internal/storage/ledger/store.go (2)
47-52: LGTM! Clean implementation of Volumes resource
The implementation properly initializes the paginated resource with appropriate handlers and pagination strategy.
58-66: LGTM! Well-structured Transactions resource implementation
The implementation correctly sets up column pagination with appropriate defaults.
internal/controller/ledger/store.go (2)
149-155: LGTM! Well-structured query types
The ResourceQuery and options types are well-designed with clear separation of concerns.
Also applies to: 251-258
124-126:
Add nil check for account retrieval
The GetAccount implementation should handle nil account case.
internal/api/v2/controllers_transactions_list_test.go (2)
33-33: LGTM: Type change aligns with the refactoring goals.
The change from specific PaginatedQueryOptions[PITFilterWithVolumes] to generic ColumnPaginatedQuery[any] improves flexibility while maintaining type safety.
42-50: LGTM: Test cases consistently updated.
The test cases have been systematically updated to use the new ColumnPaginatedQuery[any] structure. The changes maintain test coverage while adapting to the new query API. Each test case properly initializes the query with appropriate options including:
- PageSize
- Column ordering
- Resource query options (PIT, Builder, Expand)
Also applies to: 55-64, 69-78, 83-92, 97-106, 111-120, 125-134, 139-148, 153-155, 178-186, 191-208, 214-223, 228-236
internal/storage/ledger/legacy/accounts_test.go (2)
73-74: LGTM: Consistent update of list query methods.
The migration from ledgercontroller to ledgerstore query methods is applied consistently. The test cases maintain their functionality while using the new store-specific query builders.
Also applies to: 80-83, 89-93
241-242: LGTM: Account retrieval methods updated consistently.
The changes to use ledgerstore.NewGetAccountQuery are applied systematically across all test cases. The functionality remains equivalent while using the new store-specific methods.
Also applies to: 251-252, 262-263, 273-275, 290-292, 308-309, 319-320, 335-336
internal/storage/ledger/balances_test.go (3)
242-261: LGTM: Aggregated volumes structure properly tested.
The test case correctly validates the new AggregatedVolumes structure, including:
- Proper initialization of the volumes map
- Accurate big integer calculations for inputs and outputs
- Comprehensive asset coverage (USD and EUR)
266-280: LGTM: Filtering and PIT queries thoroughly tested.
The test cases comprehensively cover:
- Address-based filtering
- Point-in-time queries with effective dates
- Point-in-time queries with insertion dates
Each case properly validates the expected volumes and maintains precision in big integer calculations.
Also applies to: 284-299, 303-321
325-340: LGTM: Metadata filtering thoroughly tested.
The test cases provide comprehensive coverage of metadata-based filtering:
- Filtering with PIT constraints
- Filtering without PIT
- Empty result handling
- Metadata existence checks
- Combined metadata and address filtering
Also applies to: 344-356, 360-366, 371-385, 390-404
internal/storage/ledger/accounts_test.go (2)
279-281: Good addition of sleep to ensure timestamp precision.
The explicit sleep between transactions is essential to ensure reliable test execution by preventing timestamp collisions.
75-77: LGTM! Comprehensive test coverage for the new query types.
The test suite thoroughly covers various scenarios with the new query types:
- Basic listing
- Metadata filtering
- Date-based filtering
- Volume calculations
- Balance queries
- Error cases
Also applies to: 82-86, 91-95, 101-109, 115-124, 130-138, 143-152, 157-161, 165-172, 176-188, 192-197, 201-206, 211-221, 226-231, 236-240
internal/api/v2/mocks_ledger_controller_test.go (2)
73-73: LGTM! Consistent refactoring to generic query types.
The changes standardize query handling across different resource types by introducing three generic query types:
ResourceQuery[any]for simple resource queriesOffsetPaginatedQuery[T]for offset-based paginationColumnPaginatedQuery[T]for column-based pagination
This refactoring improves type safety and code reusability.
Also applies to: 88-88, 163-163, 223-223, 282-282, 297-297, 312-312
178-178: Verify the impact of specialized query type.
The GetAggregatedBalances method uses a specific type parameter ledger0.GetAggregatedVolumesOptions instead of any. This specialization might affect existing query implementations.
✅ Verification successful
Based on the search results, I can now provide a final response:
The specialized query type is safely implemented across the codebase
The GetAggregatedVolumesOptions type parameter is consistently used throughout the codebase:
- It's properly defined as a struct with a single
UseInsertionDatefield - All implementations of
GetAggregatedBalancesacross different controllers (DefaultController, ControllerWithTraces) use this type parameter - The type is used consistently in test mocks and actual implementations
- Existing query implementations in tests show proper usage with this type parameter
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for existing implementations of GetAggregatedBalances
# to ensure they're compatible with the new type parameter
# Search for GetAggregatedBalances implementations
rg -A 5 "GetAggregatedBalances.*implements"
# Search for GetAggregatedVolumesOptions usage
ast-grep --pattern 'GetAggregatedVolumesOptions'
Length of output: 94
Script:
#!/bin/bash
# Let's try a broader search for GetAggregatedBalances usage and implementations
rg -A 5 "GetAggregatedBalances"
# Search for ResourceQuery usage with GetAggregatedVolumesOptions
rg "ResourceQuery.*GetAggregatedVolumesOptions"
# Let's also check the ledger0 package for the type definition
fd -e go . | grep -i ledger | xargs cat | grep -A 5 "type.*GetAggregatedVolumesOptions"
Length of output: 35052
internal/api/v1/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! Changes match v2 implementation.
The changes are identical to those in the v2 package, maintaining consistency across API versions.
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! Changes match v2 and v1 implementations.
The changes are identical to those in the v2 and v1 packages, maintaining consistency across all mock implementations.
internal/api/bulking/mocks_ledger_controller_test.go (5)
73-73: LGTM: Consistent use of ResourceQuery[any] across basic query methods
The change to use ResourceQuery[any] for basic query methods (CountAccounts, CountTransactions, GetAccount, GetTransaction) provides a good balance between flexibility and type safety.
Also applies to: 88-88, 163-163, 223-223
238-238: LGTM: Appropriate use of OffsetPaginatedQuery with specific type parameters
The methods correctly use OffsetPaginatedQuery with appropriate type parameters:
GetVolumesWithBalancesuses specificGetVolumesOptionsfor volume filteringListAccountsusesanyfor basic filtering capabilities
Also applies to: 282-282
297-297: LGTM: Consistent use of ColumnPaginatedQuery for time-series data
The methods appropriately use ColumnPaginatedQuery[any] for handling time-series data (logs and transactions) with consistent pagination capabilities.
Also applies to: 312-312
178-178: LGTM: Type-safe query options for aggregated balances
The use of ResourceQuery[GetAggregatedVolumesOptions] appropriately maintains type safety while allowing specific configuration for balance aggregation.
Let's verify the consistency of GetAggregatedVolumesOptions usage:
✅ Verification successful
Based on the search results, I can generate the final response as we have comprehensive information about the usage of GetAggregatedVolumesOptions across the codebase.
Verified: GetAggregatedVolumesOptions is consistently used with ResourceQuery
The type parameter GetAggregatedVolumesOptions is consistently used across the codebase:
- Store interface correctly defines
AggregatedBalances()withResource[ledger.AggregatedVolumes, GetAggregatedVolumesOptions] - All controller implementations (Default, WithTraces, mocks) maintain the same type parameter
- Test cases demonstrate proper usage with options like
UseInsertionDate - API layers (v1, v2, bulking) consistently handle the type-safe query
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify GetAggregatedVolumesOptions usage across the codebase
# Expected: GetAggregatedVolumesOptions should be used consistently with ResourceQuery
# Search for GetAggregatedVolumesOptions usage
rg "GetAggregatedVolumesOptions" -A 3
Length of output: 18202
Line range hint 73-312: Verify interface compatibility with the actual implementation
While the mock changes look good, we should verify that they match the actual interface implementation.
✅ Verification successful
Mock implementation matches the interface correctly
After comparing the mock implementations in mocks_ledger_controller_test.go with the actual Controller interface definition in controller.go, all methods are correctly implemented with matching signatures and return types. The mock file provides implementations for all required interface methods including transaction management, database operations, and metadata handling.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that the mock implementations match the actual interface
# Expected: Method signatures should be identical
# Find and compare the actual interface definition
rg -A 1 "type Controller interface {" internal/controller/ledger/
Length of output: 185
Script:
#!/bin/bash
# Let's try a different approach to find the interface definition
# First, let's search for files that might contain the interface
fd "\.go$" internal/controller/ledger/
# Then we can use ast-grep to find interface definitions
ast-grep --pattern 'interface { $$$ }'
Length of output: 65702
Script:
#!/bin/bash
# Let's specifically look for the Controller interface definition
rg -A 1 "type Controller interface" internal/controller/ledger/controller.go
Length of output: 180
Script:
#!/bin/bash
# Let's get the full interface definition to compare with the mock
rg -A 60 "type Controller interface" internal/controller/ledger/controller.go
Length of output: 3947
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (1)
internal/storage/ledger/utils.go (1)
Line range hint
7-19: Remove redundantisSegmentedAddressfunctionThe
isSegmentedAddressfunction is now redundant as its functionality is covered by the new genericisPartialAddressfunction. Having both functions maintains duplicate logic and could lead to confusion.-func isSegmentedAddress(address string) bool { - src := strings.Split(address, ":") - - needSegmentCheck := false - for _, segment := range src { - needSegmentCheck = segment == "" - if needSegmentCheck { - break - } - } - - return needSegmentCheck -}
♻️ Duplicate comments (6)
internal/storage/ledger/resource_aggregated_balances.go (1)
119-120:⚠️ Potential issueAvoid using
panic; return an error insteadUsing
panic("unreachable")can cause the application to crash if unexpected input is encountered. It's better to return an error to gracefully handle unexpected cases.internal/controller/ledger/store.go (1)
124-126:⚠️ Potential issueHandle potential nil account
Ensure that the returned
accountis not nil before returning it to prevent potential nil pointer dereferences.Apply this diff to add a nil check:
account, err := v.Store.Accounts().GetOne(ctx, ResourceQuery[any]{ Builder: query.Match("address", address), }) if err != nil { return nil, err } +if account == nil { + return nil, fmt.Errorf("account not found") +} return account, nilinternal/storage/ledger/resource_transactions.go (1)
153-154:⚠️ Potential issueAvoid using
panic; return an error insteadThe use of
panic("unreachable")can cause the application to crash if an unexpected property is encountered. It is better to return an error to handle such cases gracefully.Apply this diff to return an error:
func (h transactionsResourceHandler) resolveFilter(store *Store, opts ledgercontroller.ResourceQuery[any], operator, property string, value any) (string, []any, error) { // ... } - panic("unreachable") + return "", nil, fmt.Errorf("unhandled property: %s", property) }internal/storage/ledger/paginator_offset.go (2)
33-37:⚠️ Potential issueGuard against integer underflow in offset calculation
Subtracting
query.PageSizefromquery.Offsetwithout checks might lead to integer underflow whenquery.Offsetis less thanquery.PageSize.Apply this diff to prevent underflow:
cp := query -offset := int(query.Offset) - int(query.PageSize) -if offset < 0 { +var offset uint64 +if query.Offset < query.PageSize { offset = 0 } else { - offset = 0 + offset = query.Offset - query.PageSize } cp.Offset = offset
42-47:⚠️ Potential issueAdd boundary checks for next page calculation
When calculating the next offset, it's important to ensure that adding
query.PageSizetoquery.Offsetdoes not cause an overflow.Apply this diff to include boundary checks:
if query.PageSize != 0 && len(ret) > int(query.PageSize) { cp := query + // Check for potential overflow + if query.Offset > math.MaxUint64 - query.PageSize { + return nil, fmt.Errorf("offset overflow") + } cp.Offset = query.Offset + query.PageSize next = &cp ret = ret[:len(ret)-1] }internal/storage/ledger/resource_volumes.go (1)
191-192:⚠️ Potential issueAvoid using
panic; return an error insteadUsing
panic("unreachable")in theresolveFiltermethod may cause the application to crash unexpectedly. It's better to return an error to handle unexpected cases gracefully.Apply this diff to handle the error properly:
- panic("unreachable") + return "", nil, fmt.Errorf("unhandled property: %s", property)
🧹 Nitpick comments (13)
internal/storage/ledger/resource_aggregated_balances.go (3)
38-70: Consider improving error handling and documentationThe implementation is solid but could benefit from:
- Extracting feature checks into a separate method for better reusability
- Adding documentation for the complex SQL queries, especially the window functions and partitioning logic
Consider refactoring the feature checks:
+func (h aggregatedBalancesResourceRepositoryHandler) checkPITFeatures(store *Store, useInsertionDate bool) error { + if useInsertionDate { + if !store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { + return ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistory) + } + return nil + } + if !store.ledger.HasFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes, "SYNC") { + return ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes) + } + return nil +}Then update the buildDataset method to use it:
if query.PIT != nil && !query.PIT.IsZero() { ret := store.db.NewSelect(). ModelTableExpr(store.GetPrefixedRelationName("moves")). DistinctOn("accounts_address, asset"). Column("accounts_address", "asset"). Where("ledger = ?", store.ledger.Name) - if query.Opts.UseInsertionDate { - if !store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { - return nil, ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistory) - } + if err := h.checkPITFeatures(store, query.Opts.UseInsertionDate); err != nil { + return nil, err + }
87-107: Consider extracting metadata query builderThe metadata query construction logic is complex and could benefit from being extracted into a separate method for better maintainability and testability.
Consider extracting the metadata query logic:
+func (h aggregatedBalancesResourceRepositoryHandler) buildMetadataQuery(store *Store, query ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions]) *bun.SelectQuery { + var selectMetadata *bun.SelectQuery + if store.ledger.HasFeature(features.FeatureAccountMetadataHistory, "SYNC") && query.PIT != nil && !query.PIT.IsZero() { + selectMetadata = store.db.NewSelect(). + DistinctOn("accounts_address"). + ModelTableExpr(store.GetPrefixedRelationName("accounts_metadata")). + Where("accounts_address = dataset.accounts_address"). + Order("accounts_address", "revision desc") + + if query.PIT != nil && !query.PIT.IsZero() { + selectMetadata = selectMetadata.Where("date <= ?", query.PIT) + } + } else { + selectMetadata = store.db.NewSelect(). + ModelTableExpr(store.GetPrefixedRelationName("accounts")). + Where("address = dataset.accounts_address") + } + return selectMetadata. + Where("ledger = ?", store.ledger.Name). + Column("metadata"). + Limit(1) +}
130-144: Consider adding documentation for the aggregation logicThe SQL aggregation logic is complex and uses advanced PostgreSQL features (JSON aggregation, subqueries). Consider adding documentation to explain:
- The purpose of each subquery
- The structure of the resulting JSON
- The reason for using
json_build_objectandaggregate_objectsinternal/storage/ledger/utils.go (1)
Line range hint
25-40: Enhance SQL query construction safety and readabilityThe function could benefit from the following improvements:
- Use parameterized queries to prevent SQL injection
- Separate SQL query generation from string manipulation
- Use more descriptive variable names
Consider refactoring like this:
func filterAccountAddress(address, key string) string { - parts := make([]string, 0) + conditions := make([]string, 0) if isPartialAddress(address) { src := strings.Split(address, ":") - parts = append(parts, fmt.Sprintf("jsonb_array_length(%s_array) = %d", key, len(src))) + conditions = append(conditions, fmt.Sprintf("jsonb_array_length(%s_array) = $1", key)) for i, segment := range src { if len(segment) == 0 { continue } - parts = append(parts, fmt.Sprintf("%s_array @@ ('$[%d] == \"%s\"')::jsonpath", key, i, segment)) + conditions = append(conditions, fmt.Sprintf("%s_array @@ ('$[%d] == $%d')::jsonpath", key, i, len(conditions)+2)) } } else { - parts = append(parts, fmt.Sprintf("%s = '%s'", key, address)) + conditions = append(conditions, fmt.Sprintf("%s = $1", key)) } - return strings.Join(parts, " and ") + return strings.Join(conditions, " AND ") }internal/controller/ledger/store.go (6)
60-64: Well-structured interface changes with good separation of concernsThe new Store interface methods demonstrate good use of generics and clear separation between paginated and non-paginated resources. The design allows for consistent resource handling while maintaining type safety.
Consider documenting the rationale for making AggregatedBalances non-paginated while Volumes is paginated, as this architectural decision might not be immediately obvious to other developers.
149-155: Well-designed ResourceQuery type with temporal query supportThe ResourceQuery type effectively combines filtering, expansion, and temporal query capabilities. Consider making the UnmarshalJSON implementation more robust by handling nil Builder case.
Consider this enhancement:
func (rq *ResourceQuery[Opts]) UnmarshalJSON(data []byte) error { // ... existing code ... var err error *rq = ResourceQuery[Opts](x.rawResourceQuery) - rq.Builder, err = query.ParseJSON(string(x.Builder)) + if len(x.Builder) > 0 { + rq.Builder, err = query.ParseJSON(string(x.Builder)) + if err != nil { + return fmt.Errorf("parsing query builder: %w", err) + } + } - return err + return nil }
187-188: Address TODO comment about query removalThe comment indicates that the Query method should be removed quickly. This needs to be tracked and addressed.
Would you like me to create a GitHub issue to track the removal of this method?
183-189: Add interface documentationThe Resource interface would benefit from documentation explaining:
- The purpose and use cases of each method
- The relationship between GetOne and Query methods
- The expected behavior when no resource is found
198-199: Address TODO about backporting to go-libsThe comment indicates that some functionality needs to be backported to go-libs.
Would you like me to create a GitHub issue to track this backporting task?
259-266: Add documentation for option typesPlease add documentation explaining:
- The purpose of UseInsertionDate and its impact
- The meaning and valid values for GroupLvl
- Usage examples for both option types
internal/storage/ledger/resource_transactions.go (1)
47-49: Add validators for the "timestamp" filterThe "timestamp" filter currently lacks validators. Implementing validators will ensure that the filter operates correctly and only accepts valid operators and values.
Apply this diff to add validators:
{ // todo: add validators name: "timestamp", + validators: []propertyValidator{ + acceptOperators("$match", "$gt", "$gte", "$lt", "$lte"), + propertyValidatorFunc(func(l ledger.Ledger, operator string, key string, value any) error { + // Add validation logic for timestamp values here + return nil + }), + }, },internal/storage/ledger/volumes.go (1)
14-14: Update tracing metric name to match function nameThe tracing metric name is
"UpdateBalances", but the function is namedUpdateVolumes. Consider updating the metric name for consistency.Apply this diff to fix the inconsistency:
- "UpdateBalances", + "UpdateVolumes",internal/storage/ledger/store.go (1)
47-83: Ensure new resource methods are documented and testedThe addition of resource methods like
Volumes,AggregatedVolumes,Transactions,Logs, andAccountsenhances the resource management capabilities of theStore. Please ensure these methods are well-documented and covered by unit tests to facilitate maintenance and future development.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (18)
internal/api/v2/controllers_volumes.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/storage/ledger/accounts.go(3 hunks)internal/storage/ledger/balances.go(3 hunks)internal/storage/ledger/debug.go(1 hunks)internal/storage/ledger/logs.go(3 hunks)internal/storage/ledger/moves.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)internal/storage/ledger/store.go(3 hunks)internal/storage/ledger/transactions.go(4 hunks)internal/storage/ledger/utils.go(2 hunks)internal/storage/ledger/volumes.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- internal/api/v2/controllers_volumes.go
- internal/storage/ledger/resource_logs.go
- internal/storage/ledger/resource_accounts.go
🔇 Additional comments (21)
internal/storage/ledger/debug.go (3)
12-12: LGTM! Consistent receiver naming improves readability.
The change from s to store as the receiver name is applied consistently across all methods, making the code more readable and self-documenting.
Also applies to: 23-23, 33-33
12-20: Verify table data exposure security implications.
The DumpTables method could potentially expose sensitive data. Let's verify if there are any security measures in place.
26-28: Verify error handling in debug utilities.
While using panic for error handling is acceptable in debug utilities, we should ensure these methods are not accidentally called in production code.
Also applies to: 35-37, 39-41
✅ Verification successful
Let me verify if these debug methods are only used in debug-specific files and test files by checking for any imports of these methods.
Let me check one more thing - let's verify if these debug methods are exported and if there are any safeguards in place (like build tags).
Debug methods are properly safeguarded with unused linter directives
The debug methods are properly protected from accidental usage in production code:
- All debug methods are marked with
//nolint:unuseddirective - Previous search confirmed no usage outside of debug files
- Methods are only defined in dedicated debug.go files
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check if these debug methods are called outside of test files
# Search for DumpTables, DumpQuery, or DumpRows method calls
rg -g '!*.test.go' -g '!*_test.go' -A 2 '\b(DumpTables|DumpQuery|DumpRows)\b'
Length of output: 1906
Script:
#!/bin/bash
# Search for imports of the debug-related packages
rg -g '!*.test.go' -g '!*_test.go' -g '!debug.go' '"internal/storage/ledger"'
# Also check for any direct usage of Store type from these packages
rg -g '!*.test.go' -g '!*_test.go' -g '!debug.go' 'ledger\.Store'
Length of output: 147
Script:
#!/bin/bash
# Check the content of debug files to see if there are any build tags or other safeguards
cat internal/storage/ledger/debug.go
cat internal/storage/ledger/legacy/debug.go
# Also check if there are any build tags in the directory
rg -g '*.go' '//\s*\+build' internal/storage/ledger/
Length of output: 1633
internal/storage/ledger/logs.go (4)
57-61: LGTM: Well-implemented advisory locking mechanism
The implementation correctly uses transaction-level advisory locks to ensure atomicity when the FeatureHashLogs feature is enabled. The error handling is properly wrapped with postgres.ResolveError.
Line range hint 94-104: LGTM: Robust error handling for idempotency conflicts
The error handling is well-structured with specific handling for idempotency key conflicts using the custom error type. The implementation properly distinguishes between different constraint violations.
122-127: Consider adding an index for idempotency key lookups
While the query implementation is correct, frequent lookups by idempotency key might benefit from an index on (ledger, idempotency_key) columns.
Let's verify if an index already exists:
Line range hint 1-134: Verify removal of ListLogs method references
The ListLogs method has been removed from this file. Let's verify that all references to this method have been properly cleaned up across the codebase.
internal/storage/ledger/resource_aggregated_balances.go (3)
11-12: LGTM! Good use of Go patterns
The empty struct pattern is appropriate here since the handler only needs to implement methods without storing state. The interface implementation check at the end of the file is a good practice for compile-time verification.
Also applies to: 146-146
13-36: LGTM! Well-structured filter definitions
The filter implementation is clean and well-organized:
- Address validation is properly handled
- Metadata filters support both existence checks and pattern matching
- Validator pattern ensures proper filter validation
126-128: LGTM! Clear error handling
The method correctly indicates that expansion is not supported for aggregated balances with a clear error message.
internal/storage/ledger/utils.go (1)
42-53: Verify the impact of the new generic function
Since this is part of a larger refactoring effort, we should verify:
- All callers of the old
isSegmentedAddresshave been updated - The new generic function is used consistently across the codebase
✅ Verification successful
Let me gather more context about the old and new implementations to ensure a complete verification.
Let me get one final piece of information to verify the implementation of both functions.
Changes are consistent and properly implemented
The verification shows that:
- The new generic
isPartialAddressfunction maintains the same logic as the oldisSegmentedAddressfunction - All usages have been updated to use
isPartialAddressacross the codebase - The old
isSegmentedAddressfunction is still present but only used in one legacy check instore.gowith a feature flag, suggesting an intentional transition period
The changes are safe and properly implemented as part of the refactoring effort.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for any remaining usage of isSegmentedAddress
rg "isSegmentedAddress"
# Check for usage patterns of isPartialAddress
rg "isPartialAddress"
# Look for similar address parsing patterns that might need updating
rg "strings\.Split\(.+\":\".*\)"
Length of output: 1193
Script:
#!/bin/bash
# Get the implementation of isSegmentedAddress
ast-grep --pattern 'func isSegmentedAddress($$$) {
$$$
}'
# Get more context around the usage in store.go
rg -B2 -A2 "isSegmentedAddress" internal/storage/ledger/store.go
# Get more context around isPartialAddress usages
rg -B2 -A2 "isPartialAddress" internal/storage/ledger/resource_aggregated_balances.go
rg -B2 -A2 "isPartialAddress" internal/storage/ledger/resource_volumes.go
Length of output: 1007
Script:
#!/bin/bash
# Get both function implementations for comparison
cat internal/storage/ledger/utils.go
Length of output: 1097
internal/controller/ledger/store.go (2)
208-217: LGTM! Well-designed pagination interface
The PaginatedResource interface effectively combines Resource capabilities with pagination support while maintaining type safety through generics.
247-248:
Add nil check for account
Similar to the vmStoreAdapter, ensure that the returned account is not nil before accessing its fields.
Apply this diff:
v, err := s.Store.Accounts().GetOne(ctx, ResourceQuery[any]{
Builder: query.Match("address", address),
})
if err != nil {
return nil, err
}
+if v == nil {
+ return nil, fmt.Errorf("account %s not found", address)
+}Likely invalid or redundant comment.
internal/storage/ledger/paginator_offset.go (1)
50-55: Ensure correct PageSize and HasMore values in cursor
The PageSize should reflect the actual number of items returned, and HasMore should be correctly set based on the availability of additional data.
Run the following script to check the PageSize and HasMore values:
✅ Verification successful
Let me gather more information about how the PageSize and HasMore values are determined.
Let me try a different approach to find the relevant code.
Based on the code review, I can now provide the final response:
PageSize and HasMore values are correctly implemented in the cursor
The implementation shows that:
PageSizecorrectly reflects the requested page size from the queryHasMoreis properly set based on whether there are more items (when fetched items exceed the requested page size by 1)- The pagination logic fetches n+1 items (where n is PageSize) to determine if there are more items
- The extra item is removed from the result before returning
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that PageSize and HasMore are correctly set in the cursor.
# Check usages of bunpaginate.Cursor to ensure correct values.
rg -A 5 'bunpaginate.Cursor' internal/storage/ledger/
Length of output: 6949
Script:
#!/bin/bash
# Check the implementation of the paginator to see how PageSize and HasMore are determined
ast-grep --pattern 'func (o offsetPaginator[ResourceType, OptionsType]) buildCursor($$$) {
$$$
}'
# Also check how the data is fetched before building the cursor
ast-grep --pattern 'func (o offsetPaginator[ResourceType, OptionsType]) Paginate($$$) {
$$$
}'
Length of output: 200
Script:
#!/bin/bash
# Check the full content of the paginator files to understand the pagination logic
cat internal/storage/ledger/paginator_offset.go
cat internal/storage/ledger/paginator.go
Length of output: 2118
internal/storage/ledger/balances.go (1)
Line range hint 16-105: LGTM!
The GetBalances function correctly retrieves account balances and handles complex query construction effectively.
internal/storage/ledger/transactions.go (1)
121-121: 🛠️ Refactor suggestion
Ensure store.ledger.ID is safely formatted in SQL query
Using fmt.Sprintf with store.ledger.ID in SQL query construction can be risky if ID is not properly sanitized. Although ID is likely an integer, it's safer to use parameterized queries to prevent SQL injection vulnerabilities.
Consider parameterizing the query as follows:
- query = query.Value("id", "nextval(?)", store.GetPrefixedRelationName(fmt.Sprintf(`"transaction_id_%d"`, store.ledger.ID)))
+ sequenceName := fmt.Sprintf(`"transaction_id_%d"`, store.ledger.ID)
+ query = query.Value("id", "nextval(?)", store.GetPrefixedRelationName(sequenceName))Likely invalid or redundant comment.
internal/storage/ledger/resource.go (2)
18-32: Handle unsupported operators without panicking
The convertOperatorToSQL function uses panic("unreachable") for unsupported operators, which can cause the application to crash unexpectedly. Consider returning an error instead to handle such cases gracefully.
114-116:
Avoid using panic for error handling
Using panic(err) when an error occurs in regexp.MatchString can cause the application to crash unexpectedly. It's better to handle the error and return it to the caller.
Apply this diff to handle the error gracefully:
- if found, err = regexp.MatchString("^"+property.name+"$", key); err != nil {
- panic(err)
- }
+ if found, err = regexp.MatchString("^"+property.name+"$", key); err != nil {
+ return nil, err
+ }Likely invalid or redundant comment.
internal/storage/ledger/moves.go (1)
10-23: Consistent receiver naming enhances readability
Changing the receiver name from s to store improves code clarity and consistency across methods in the Store struct.
internal/storage/ledger/accounts.go (2)
53-53: Verify the assignment of 'updated_at' during upsert operations
In the UpdateAccountsMetadata and UpsertAccounts functions, you are setting updated_at = excluded.updated_at. Ensure that excluded.updated_at contains the correct update timestamp. If you intend to set updated_at to the current timestamp when updating records, consider using NOW() to reflect the actual update time.
Also applies to: 109-109
93-113: Potential issue with conflict handling in upsert operations
In the UpsertAccounts function, the WHERE clause within the ON CONFLICT DO UPDATE statement might not behave as intended. The WHERE clause in this context filters when the DO UPDATE action should occur, not which rows to update. Verify that the conflict resolution logic correctly handles updates based on first_usage and metadata.
internal/storage/ledger/utils.go
Outdated
| func isPartialAddress[V any](address V) bool { | ||
| switch address := any(address).(type) { | ||
| case string: | ||
| for _, segment := range strings.Split(address, ":") { | ||
| if segment == "" { | ||
| return true | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return false | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Improve type safety and error handling in isPartialAddress
The current implementation has several potential improvements:
- Add documentation to explain the function's purpose and behavior
- Consider returning an error for non-string types instead of silently returning false
- Consider constraining the type parameter to types that can be converted to string
Consider this improved version:
+// isPartialAddress checks if the given address contains any empty segments when split by ':'.
+// Returns true if the address is a string containing empty segments, false otherwise.
+// For non-string types, it returns an error.
-func isPartialAddress[V any](address V) bool {
+func isPartialAddress[V ~string](address V) (bool, error) {
- switch address := any(address).(type) {
- case string:
- for _, segment := range strings.Split(address, ":") {
- if segment == "" {
- return true
- }
+ for _, segment := range strings.Split(string(address), ":") {
+ if segment == "" {
+ return true, nil
}
- }
-
- return false
+ }
+ return false, nil
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| func isPartialAddress[V any](address V) bool { | |
| switch address := any(address).(type) { | |
| case string: | |
| for _, segment := range strings.Split(address, ":") { | |
| if segment == "" { | |
| return true | |
| } | |
| } | |
| } | |
| return false | |
| } | |
| // isPartialAddress checks if the given address contains any empty segments when split by ':'. | |
| // Returns true if the address is a string containing empty segments, false otherwise. | |
| // For non-string types, it returns an error. | |
| func isPartialAddress[V ~string](address V) (bool, error) { | |
| for _, segment := range strings.Split(string(address), ":") { | |
| if segment == "" { | |
| return true, nil | |
| } | |
| } | |
| return false, nil | |
| } |
| func (h transactionsResourceHandler) resolveFilter(store *Store, opts ledgercontroller.ResourceQuery[any], operator, property string, value any) (string, []any, error) { | ||
| switch { | ||
| case property == "id": | ||
| return fmt.Sprintf("id %s ?", convertOperatorToSQL(operator)), []any{value}, nil | ||
| case property == "reference" || property == "timestamp": | ||
| return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil | ||
| case property == "reverted": | ||
| ret := "reverted_at is" | ||
| if value.(bool) { | ||
| ret += " not" | ||
| } | ||
| return ret + " null", nil, nil | ||
| case property == "account": | ||
| return filterAccountAddressOnTransactions(value.(string), true, true), nil, nil | ||
| case property == "source": | ||
| return filterAccountAddressOnTransactions(value.(string), true, false), nil, nil | ||
| case property == "destination": | ||
| return filterAccountAddressOnTransactions(value.(string), false, true), nil, nil | ||
| case metadataRegex.Match([]byte(property)): | ||
| match := metadataRegex.FindAllStringSubmatch(property, 3) | ||
|
|
||
| return "metadata @> ?", []any{map[string]any{ | ||
| match[0][1]: value, | ||
| }}, nil | ||
|
|
||
| case property == "metadata": | ||
| return "metadata -> ? is not null", []any{value}, nil | ||
| case property == "timestamp": | ||
| return fmt.Sprintf("timestamp %s ?", convertOperatorToSQL(operator)), []any{value}, nil | ||
| } | ||
|
|
||
| panic("unreachable") | ||
| } |
There was a problem hiding this comment.
Remove duplicate handling of "timestamp" in resolveFilter
The resolveFilter method handles the "timestamp" property twice, which can lead to unexpected behavior or logical errors.
Apply this diff to remove the duplicate case:
func (h transactionsResourceHandler) resolveFilter(store *Store, opts ledgercontroller.ResourceQuery[any], operator, property string, value any) (string, []any, error) {
switch {
case property == "id":
return fmt.Sprintf("id %s ?", convertOperatorToSQL(operator)), []any{value}, nil
- case property == "reference" || property == "timestamp":
+ case property == "reference":
return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil
case property == "reverted":
ret := "reverted_at is"
if value.(bool) {
ret += " not"
}
return ret + " null", nil, nil
case property == "account":
return filterAccountAddressOnTransactions(value.(string), true, true), nil, nil
case property == "source":
return filterAccountAddressOnTransactions(value.(string), true, false), nil, nil
case property == "destination":
return filterAccountAddressOnTransactions(value.(string), false, true), nil, nil
case metadataRegex.Match([]byte(property)):
match := metadataRegex.FindAllStringSubmatch(property, 3)
return "metadata @> ?", []any{map[string]any{
match[0][1]: value,
}}, nil
case property == "metadata":
return "metadata -> ? is not null", []any{value}, nil
- case property == "timestamp":
- return fmt.Sprintf("timestamp %s ?", convertOperatorToSQL(operator)), []any{value}, nil
}
panic("unreachable")
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| func (h transactionsResourceHandler) resolveFilter(store *Store, opts ledgercontroller.ResourceQuery[any], operator, property string, value any) (string, []any, error) { | |
| switch { | |
| case property == "id": | |
| return fmt.Sprintf("id %s ?", convertOperatorToSQL(operator)), []any{value}, nil | |
| case property == "reference" || property == "timestamp": | |
| return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil | |
| case property == "reverted": | |
| ret := "reverted_at is" | |
| if value.(bool) { | |
| ret += " not" | |
| } | |
| return ret + " null", nil, nil | |
| case property == "account": | |
| return filterAccountAddressOnTransactions(value.(string), true, true), nil, nil | |
| case property == "source": | |
| return filterAccountAddressOnTransactions(value.(string), true, false), nil, nil | |
| case property == "destination": | |
| return filterAccountAddressOnTransactions(value.(string), false, true), nil, nil | |
| case metadataRegex.Match([]byte(property)): | |
| match := metadataRegex.FindAllStringSubmatch(property, 3) | |
| return "metadata @> ?", []any{map[string]any{ | |
| match[0][1]: value, | |
| }}, nil | |
| case property == "metadata": | |
| return "metadata -> ? is not null", []any{value}, nil | |
| case property == "timestamp": | |
| return fmt.Sprintf("timestamp %s ?", convertOperatorToSQL(operator)), []any{value}, nil | |
| } | |
| panic("unreachable") | |
| } | |
| func (h transactionsResourceHandler) resolveFilter(store *Store, opts ledgercontroller.ResourceQuery[any], operator, property string, value any) (string, []any, error) { | |
| switch { | |
| case property == "id": | |
| return fmt.Sprintf("id %s ?", convertOperatorToSQL(operator)), []any{value}, nil | |
| case property == "reference": | |
| return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil | |
| case property == "reverted": | |
| ret := "reverted_at is" | |
| if value.(bool) { | |
| ret += " not" | |
| } | |
| return ret + " null", nil, nil | |
| case property == "account": | |
| return filterAccountAddressOnTransactions(value.(string), true, true), nil, nil | |
| case property == "source": | |
| return filterAccountAddressOnTransactions(value.(string), true, false), nil, nil | |
| case property == "destination": | |
| return filterAccountAddressOnTransactions(value.(string), false, true), nil, nil | |
| case metadataRegex.Match([]byte(property)): | |
| match := metadataRegex.FindAllStringSubmatch(property, 3) | |
| return "metadata @> ?", []any{map[string]any{ | |
| match[0][1]: value, | |
| }}, nil | |
| case property == "metadata": | |
| return "metadata -> ? is not null", []any{value}, nil | |
| } | |
| panic("unreachable") | |
| } |
| if store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { | ||
| moves := ledger.Moves{} | ||
| postings := tx.Postings | ||
| slices.Reverse(postings) |
There was a problem hiding this comment.
Avoid modifying tx.Postings in-place when reversing
Reversing postings := tx.Postings using slices.Reverse(postings) modifies tx.Postings in-place. This can lead to unintended side effects if tx.Postings is used elsewhere after this operation. Consider making a copy of tx.Postings before reversing to prevent modifying the original slice.
Apply this diff to create a copy before reversing:
if store.ledger.HasFeature(features.FeatureMovesHistory, "ON") {
moves := ledger.Moves{}
- postings := tx.Postings
+ postings := make([]ledger.Posting, len(tx.Postings))
+ copy(postings, tx.Postings)
slices.Reverse(postings)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { | |
| moves := ledger.Moves{} | |
| postings := tx.Postings | |
| slices.Reverse(postings) | |
| if store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { | |
| moves := ledger.Moves{} | |
| postings := make([]ledger.Posting, len(tx.Postings)) | |
| copy(postings, tx.Postings) | |
| slices.Reverse(postings) |
| func validateAddressFilter(ledger ledger.Ledger, operator string, value any) error { | ||
| if operator != "$match" { | ||
| return errors.New("'address' column can only be used with $match") | ||
| return fmt.Errorf("'address' column can only be used with $match, operator used is: %s", operator) | ||
| } | ||
| if value, ok := value.(string); !ok { | ||
| return fmt.Errorf("invalid 'address' filter") | ||
| } else if isSegmentedAddress(value) && !s.ledger.HasFeature(features.FeatureIndexAddressSegments, "ON") { | ||
| } else if isSegmentedAddress(value) && !ledger.HasFeature(features.FeatureIndexAddressSegments, "ON") { |
There was a problem hiding this comment.
Check feature flag usage in 'validateAddressFilter'
In the validateAddressFilter function, you're checking the feature flag with ledger.HasFeature(features.FeatureIndexAddressSegments, "ON"). Ensure that the second argument is correctly specified according to the HasFeature function's expected parameters. It may need a boolean or a predefined constant rather than the string "ON".
Apply this diff if the second argument should be omitted or adjusted:
- } else if isSegmentedAddress(value) && !ledger.HasFeature(features.FeatureIndexAddressSegments, "ON") {
+ } else if isSegmentedAddress(value) && !ledger.HasFeature(features.FeatureIndexAddressSegments) {Please adjust based on the correct usage of the HasFeature function.
Committable suggestion skipped: line range outside the PR's diff.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
internal/storage/ledger/resource_volumes.go (1)
57-145: Query-building logic is sound, but consider verifying performance with large datasets
The conditional logic for building the volumes dataset, including PIT vs. OOT clauses and address/metadata subqueries, appears correct. However, these queries can become expensive with large ledgers if indexes or query plans are suboptimal. It’s advisable to verify performance under typical and high-load scenarios.internal/storage/ledger/resource.go (2)
117-121: Consider compiling the regular expression once for better performance.
Repeatedly calling regexp.MatchString in a loop could incur measurable overhead under heavy usage. Compiling the regex once outside the loop (using regexp.MustCompile) and then reusing it within the loop would be more efficient.
249-249: Revisit commented logging.
The logging line is commented out. If logging for query debugging or performance analysis is desired, consider reintroducing it or removing it entirely to avoid confusion.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/storage/ledger/resource_aggregated_balances.go
🔇 Additional comments (6)
internal/storage/ledger/resource_volumes.go (5)
1-14: Overall file introduction and struct definition look good
The package declaration, imports, and basic type definition for volumesResourceHandler set a clear foundation. Everything appears structurally coherent for a new file.
15-55: Filter definitions are clear but watch out for potential edge cases
The filters for "address," "balance," and "metadata" seem well-structured. However, ensure comprehensive testing for complex regex-based balance filters (line 27) to avoid unexpected matches on property names. Additionally, confirm that all operator usage (especially "$match") undergoes thorough input validation to prevent potential pattern injection.
147-183: Avoid using panic for unreachable cases
Using panic("unreachable") (line 182) can cause catastrophic failures in production. Per the existing comment on a previous commit, it’s best to return an error instead.
185-211: Double-check grouping logic for partial addresses
The aggregation logic properly handles grouping by progressively slicing address segments (line 202). However, edge or invalid address formats could break grouping if string splitting yields fewer segments than expected. Ensure address validation or failover paths handle unusual address formats.
213-215: Expand method is sufficiently explicit
Returning a clear “no expansion available” error removes ambiguity.
internal/storage/ledger/resource.go (1)
18-32: Handle unsupported operators without panicking.
This is the same concern flagged in a previous review comment. The current approach uses panic("unreachable"), which could abruptly terminate the application if an unexpected operator is encountered. Consider returning an error instead to handle such cases gracefully.
| case ledgercontroller.ColumnPaginatedQuery[OptionsType]: | ||
| resourceQuery = v.Options | ||
| default: | ||
| panic("should not happen") |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid panic for unexpected pagination query type.
Using panic("should not happen") here can cause the entire application to crash if a new or unrecognized paginated query type is introduced. Consider returning an error or gracefully handling this scenario.
gfyrag
force-pushed
the
refactor/store-read-part
branch
from
3bafdbd to
881f50a
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
internal/storage/ledger/resource_logs.go (1)
28-35:⚠️ Potential issueEnhance SQL injection prevention in resolveFilter
The direct string concatenation of the operator in the SQL statement could be vulnerable to SQL injection. The operator should be validated before being used in the query.
Apply this diff to fix the issue:
- return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil + op, err := convertOperatorToSQL(operator) + if err != nil { + return "", nil, fmt.Errorf("invalid operator: %w", err) + } + return fmt.Sprintf("%s %s ?", property, op), []any{value}, nil
🧹 Nitpick comments (37)
internal/storage/ledger/resource_logs.go (2)
12-19: Address TODO: Implement date validatorsThe date filter is currently defined without any validators, which could lead to invalid date formats being accepted.
Would you like me to help implement appropriate date validators? I can suggest implementations for:
- Date format validation
- Range validation
- Null/empty checks
41-45: Consider optimizing column selectionWhile returning all columns works, it might be inefficient for large datasets. Consider:
- Allowing specific column selection through the query parameter
- Defining a default set of commonly used columns
Example implementation:
func (h logsResourceHandler) project(store *Store, query ledgercontroller.ResourceQuery[any], selectQuery *bun.SelectQuery) (*bun.SelectQuery, error) { + if cols := query.GetColumns(); len(cols) > 0 { + return selectQuery.ColumnExpr(strings.Join(cols, ", ")), nil + } + // Default columns if none specified + return selectQuery.ColumnExpr("id, date, ledger"), nil }internal/api/v2/controllers_volumes_test.go (1)
Line range hint
43-120: Test cases follow a consistent pattern but could benefit from helper functionsThe test cases are well-structured and cover various scenarios. However, there's repetition in the initialization of common fields like
PageSize,PIT, and emptyExpandarrays.Consider introducing a helper function to reduce duplication:
func newTestVolumeQuery(builder query.Builder, opts ...ledgercontroller.GetVolumesOptions) ledgercontroller.OffsetPaginatedQuery[ledgercontroller.GetVolumesOptions] { var opt ledgercontroller.GetVolumesOptions if len(opts) > 0 { opt = opts[0] } return ledgercontroller.OffsetPaginatedQuery[ledgercontroller.GetVolumesOptions]{ PageSize: DefaultPageSize, Options: ledgercontroller.ResourceQuery[ledgercontroller.GetVolumesOptions]{ PIT: &before, Builder: builder, Expand: make([]string, 0), Opts: opt, }, } }This would simplify test cases to something like:
expectQuery: newTestVolumeQuery(query.Match("account", "foo"))internal/api/v2/common.go (2)
24-28: Enhance error messages for date parsingConsider providing more descriptive error messages to help debug invalid date formats.
date, err := time.ParseTime(dateString) if err != nil { - return nil, err + return nil, fmt.Errorf("invalid date format for %s: %q: %w", key, dateString, err) }
Line range hint
17-131: Overall assessment of the refactoringThe refactoring improves code organization by:
- Centralizing date parsing logic
- Using generics effectively for query handling
- Providing consistent pagination support
However, please address the security and validation concerns raised in the review before merging.
internal/api/v2/controllers_accounts_list_test.go (2)
43-49: Consider optimizing the Expand slice initializationThe empty slice initialization
make([]string, 0)is redundant as an empty slice literal[]string{}would be more idiomatic.expectQuery: ledgercontroller.OffsetPaginatedQuery[any]{ PageSize: DefaultPageSize, Options: ledgercontroller.ResourceQuery[any]{ PIT: &before, - Expand: make([]string, 0), + Expand: []string{}, }, },
82-90: Consider simplifying the empty cursor test caseThe cursor encoding could be more concise by using struct literals.
-"cursor": []string{bunpaginate.EncodeCursor(ledgercontroller.OffsetPaginatedQuery[any]{ - PageSize: DefaultPageSize, - Options: ledgercontroller.ResourceQuery[any]{}, -})}, +"cursor": []string{bunpaginate.EncodeCursor(ledgercontroller.OffsetPaginatedQuery[any]{ + PageSize: DefaultPageSize, +})},internal/api/v1/utils.go (1)
67-79: Recommend enhanced error handling for modifiers.
When iterating over modifiers, if any modifier fails, you return immediately. Ensure that partial modifications don’t leave the ResourceQuery in an inconsistent state. Optionally, you can log which modifier failed for easier debugging.internal/storage/ledger/legacy/queries.go (3)
16-20: Encourage more explicit naming for PITFilterWithVolumes fields.
While ExpandVolumes and ExpandEffectiveVolumes are descriptive, consider adding doc comments or renaming them for maximum clarity, such as ExpandTransactionVolumes or ExpandAccountEffectiveVolumes, clarifying which resource volumes you’re referencing.
80-90: Consider scoping expansions at a higher level.
Rather than toggling ExpandVolumes or ExpandEffectiveVolumes directly on the query, consider a separate config structure or builder pattern for toggling expansions. This ensures that expansions can be validated or combined in a single place.
100-103: Potential confusion between addresses.
Consider clarifying your naming for “Addr” vs. “address” throughout the code to ensure consistency. This might help avoid confusion or typos.internal/storage/ledger/legacy/adapters.go (2)
21-24: Document the todo.
A TODO comment indicates future compatibility work. Consider documenting next steps or referencing a GitHub issue to track the status of V1 compatibility improvements.
34-36: AggregatedBalances method introduced.
This delegates aggregated balance lookups to the new store. Ensure consistent naming across your codebase so users can quickly locate aggregated/Historical/Effective/Current balances with minimal confusion.internal/controller/ledger/controller_default.go (1)
190-192: Single-entry pagination usage to check emptiness is valid.
Using PageSize=1 is a neat trick, although a small performance overhead might exist. Looks fine overall.internal/api/v2/controllers_balances.go (1)
Line range hint
25-33: Consider enhancing error details in validation responses.While the error handling is structured well, consider providing more specific details in validation error responses to help API consumers better understand and fix their requests.
if err != nil { switch { case errors.Is(err, ledgercontroller.ErrInvalidQuery{}) || errors.Is(err, ledgercontroller.ErrMissingFeature{}): - api.BadRequest(w, common.ErrValidation, err) + api.BadRequest(w, common.ErrValidation, map[string]interface{}{ + "message": err.Error(), + "details": getValidationDetails(err), + }) default: common.HandleCommonErrors(w, r, err) } return }internal/api/v1/controllers_logs_list.go (1)
Line range hint
11-32: Consider enhancing buildGetLogsQuery function.While not directly changed in this PR, the
buildGetLogsQueryfunction could benefit from some improvements:
- Date parsing validation
- Query parameter constants
- Better handling of empty clauses
+const ( + paramAfter = "after" + paramStartTime = "start_time" + paramEndTime = "end_time" +) func buildGetLogsQuery(r *http.Request) query.Builder { clauses := make([]query.Builder, 0) - if after := r.URL.Query().Get("after"); after != "" { + if after := r.URL.Query().Get(paramAfter); after != "" { clauses = append(clauses, query.Lt("id", after)) } - if startTime := r.URL.Query().Get("start_time"); startTime != "" { + if startTime := r.URL.Query().Get(paramStartTime); startTime != "" { + if _, err := time.Parse(time.RFC3339, startTime); err != nil { + return nil + } clauses = append(clauses, query.Gte("date", startTime)) } - if endTime := r.URL.Query().Get("end_time"); endTime != "" { + if endTime := r.URL.Query().Get(paramEndTime); endTime != "" { + if _, err := time.Parse(time.RFC3339, endTime); err != nil { + return nil + } clauses = append(clauses, query.Lt("date", endTime)) } - if len(clauses) == 0 { - return nil - } - if len(clauses) == 1 { - return clauses[0] - } - - return query.And(clauses...) + switch len(clauses) { + case 0: + return nil + case 1: + return clauses[0] + default: + return query.And(clauses...) + } }internal/storage/ledger/accounts.go (1)
103-113: Consider adding a transaction for atomic updatesThe account upsert operation involves multiple SET clauses. Consider wrapping this in a transaction to ensure atomicity.
func (store *Store) UpsertAccounts(ctx context.Context, accounts ...*ledger.Account) error { + tx, err := store.db.BeginTx(ctx, nil) + if err != nil { + return fmt.Errorf("beginning transaction: %w", err) + } + defer tx.Rollback() + - ret, err := store.db.NewInsert(). + ret, err := tx.NewInsert(). Model(&accounts). ModelTableExpr(store.GetPrefixedRelationName("accounts")). On("conflict (ledger, address) do update"). Set("first_usage = case when excluded.first_usage < accounts.first_usage then excluded.first_usage else accounts.first_usage end"). Set("metadata = accounts.metadata || excluded.metadata"). Set("updated_at = excluded.updated_at"). Value("ledger", "?", store.ledger.Name). Returning("*"). Where("(excluded.first_usage < accounts.first_usage) or not accounts.metadata @> excluded.metadata"). Exec(ctx) if err != nil { return fmt.Errorf("upserting accounts: %w", postgres.ResolveError(err)) } + + if err := tx.Commit(); err != nil { + return fmt.Errorf("committing transaction: %w", err) + }internal/api/v1/controllers_logs_list_test.go (1)
76-78: Consider adding validation for cursor contentsWhile the empty cursor test case is good, consider adding assertions to verify the contents of the encoded cursor match expectations.
internal/storage/ledger/balances.go (2)
55-61: Consider extracting SQL subquery to a named methodThe SQL subquery for selecting moves is complex and could benefit from being extracted into a separate method for better maintainability.
88-94: Important locking order comment for deadlock preventionThe comment about keeping order for consistent locking is crucial for preventing deadlocks. Consider documenting this in a more visible location, such as package documentation.
internal/controller/ledger/controller.go (1)
27-29: Consider documenting query parameter requirementsFor methods using
ResourceQuery[any], consider adding documentation about expected query parameters and their effects on the results.internal/api/v2/controllers_accounts_count_test.go (1)
40-54: Consider adding test cases for edge casesWhile the current test cases cover the basic scenarios well, consider adding tests for:
- Empty metadata
- Invalid metadata format
- Special characters in metadata keys
internal/api/v2/controllers_transactions_count_test.go (2)
43-46: Consider consolidating duplicate test setup codeMultiple test cases share similar ResourceQuery setup code. Consider creating a helper function to reduce duplication:
+func newTestResourceQuery(pit *time.Time, builder query.Builder) ledgercontroller.ResourceQuery[any] { + return ledgercontroller.ResourceQuery[any]{ + PIT: pit, + Builder: builder, + Expand: make([]string, 0), + } +}Also applies to: 52-56, 62-66
Line range hint
38-40: Consider using a fixed timestamp for deterministic testingThe test uses
time.Now()which could lead to non-deterministic behavior. Consider using a fixed timestamp:-now := time.Now() +now := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)internal/storage/ledger/resource_aggregated_balances.go (3)
12-47: LGTM! Well-structured filter implementation with proper validation.The implementation provides robust validation for both address and metadata filters. Consider adding documentation comments to describe the purpose and behavior of each filter.
Add documentation comments:
+// aggregatedBalancesResourceRepositoryHandler implements the repository handler +// for aggregated balance queries with support for address and metadata filtering. type aggregatedBalancesResourceRepositoryHandler struct{} +// filters returns the supported filters for aggregated balances queries. +// Supports filtering by address and metadata with specific validation rules. func (h aggregatedBalancesResourceRepositoryHandler) filters() []filter {
49-131: Consider breaking down the buildDataset method for better maintainability.The method handles multiple concerns (PIT queries, feature checks, query building) and could benefit from being split into smaller, focused methods.
Consider extracting the PIT and non-PIT query building logic into separate methods:
func (h aggregatedBalancesResourceRepositoryHandler) buildDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { if query.UsePIT() { - ret := store.db.NewSelect(). - // ... PIT query logic ... + return h.buildPITDataset(store, query) } else { - ret := store.db.NewSelect(). - // ... non-PIT query logic ... + return h.buildNonPITDataset(store, query) } } +func (h aggregatedBalancesResourceRepositoryHandler) buildPITDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { + // Extract PIT query logic here +} +func (h aggregatedBalancesResourceRepositoryHandler) buildNonPITDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { + // Extract non-PIT query logic here +}
156-170: LGTM! Efficient volume aggregation implementation.The method effectively aggregates volumes using SQL aggregation functions. Consider adding documentation to explain the aggregation logic.
Add documentation to explain the aggregation process:
+// project aggregates volumes by asset and returns a JSON object containing +// the sum of input and output volumes for each asset. The result is returned +// as a single JSON object with asset keys and their corresponding volumes. func (h aggregatedBalancesResourceRepositoryHandler) project(internal/api/v1/controllers_transactions_list_test.go (1)
Line range hint
38-192: LGTM! Comprehensive test coverage for transaction listing.The test cases thoroughly cover various query scenarios and error conditions. Consider adding test cases for:
- Combined filters (e.g., metadata + time range)
- Edge cases for pagination
Add additional test cases:
{ name: "combined filters", queryParams: url.Values{ "metadata[roles]": []string{"admin"}, "start_time": []string{now.Format(time.DateFormat)}, }, expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ PageSize: DefaultPageSize, Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), Column: "id", Options: ledgercontroller.ResourceQuery[any]{ Builder: query.And( query.Match("metadata[roles]", "admin"), query.Gte("date", now.Format(time.DateFormat)), ), Expand: []string{"volumes"}, }, }, },internal/controller/ledger/store.go (2)
149-155: Add documentation for ResourceQuery fieldsConsider adding godoc comments to explain the purpose of each field, especially PIT and OOT which might not be immediately obvious to new developers.
202-202: Address TODO comment about backporting to go-libsThe TODO indicates a need to backport some functionality to go-libs. This should be tracked and addressed.
Would you like me to create a GitHub issue to track this backporting task?
internal/storage/bucket/default_bucket.go (1)
225-240: LGTM! Performance-optimized indexes for transaction queriesThe GIN indexes on sources and destinations will significantly improve query performance when filtering transactions by account. The feature flag control allows for safe rollout.
internal/api/v2/controllers_transactions_list_test.go (1)
203-220: LGTM! Consider adding type alias for clarity.The refactoring to use generic types improves type safety and flexibility. However, consider adding a type alias for commonly used generic types to improve readability.
// Consider adding these type aliases type TransactionQuery = ColumnPaginatedQuery[any] type ResourceQueryAny = ResourceQuery[any]internal/storage/ledger/balances_test.go (1)
242-261: LGTM! Consider extracting big.Int calculations.The test cases have been successfully updated to use the new
AggregatedVolumesAPI. However, the big.Int calculations could be made more readable.Consider extracting the big.Int calculations into helper functions:
func multiplyBigInt(x *big.Int, y int64) *big.Int { return big.NewInt(0).Mul(x, big.NewInt(y)) } func addBigInts(x, y *big.Int) *big.Int { return big.NewInt(0).Add(x, y) }Then use them in the test:
Input: addBigInts( multiplyBigInt(bigInt, 2), multiplyBigInt(smallInt, 2), ),internal/controller/ledger/controller_default_test.go (1)
203-220: LGTM! Consider extracting common mock setup.The refactoring to use generic types for mocks improves type safety and maintainability. The pattern is consistent across all resource types.
Consider extracting the common mock setup into helper functions to reduce duplication:
func setupMockTransactions(ctrl *gomock.Controller, store *MockStore) *MockPaginatedResource[ledger.Transaction, any, ColumnPaginatedQuery[any]] { transactions := NewMockPaginatedResource[ledger.Transaction, any, ColumnPaginatedQuery[any]](ctrl) store.EXPECT().Transactions().Return(transactions) return transactions } func setupMockAccounts(ctrl *gomock.Controller, store *MockStore) *MockPaginatedResource[ledger.Account, any, OffsetPaginatedQuery[any]] { accounts := NewMockPaginatedResource[ledger.Account, any, OffsetPaginatedQuery[any]](ctrl) store.EXPECT().Accounts().Return(accounts) return accounts }Also applies to: 232-261
internal/api/v2/mocks_ledger_controller_test.go (1)
Line range hint
198-342: Excellent architectural improvement through query type generalizationThe refactoring to use generic query types (
ResourceQuery[any],ColumnPaginatedQuery[any],OffsetPaginatedQuery[any]) across the codebase is a significant improvement:
- Reduces code duplication by consolidating similar query types
- Maintains type safety through proper use of generics
- Improves code maintainability by standardizing query handling
- Allows for future extensibility without interface changes
- Properly separates pagination concerns (column vs. offset based)
internal/api/v1/mocks_ledger_controller_test.go (1)
282-282: Document pagination strategy changesThe introduction of distinct pagination types (
OffsetPaginatedQueryandColumnPaginatedQuery) suggests a deliberate separation of pagination strategies. This architectural decision should be documented.Consider adding documentation explaining:
- When to use each pagination type
- The implications of switching between pagination strategies
- Migration guide for existing code
Also applies to: 297-297, 312-312
internal/controller/ledger/store_generated_test.go (1)
442-548: Consider adding documentation for PaginationQueryType constraint.The MockPaginatedResource implementation is well-structured and correctly implements all required methods. However, the PaginationQueryType generic constraint could benefit from a documentation comment explaining its purpose and requirements.
Add a comment above the type declaration:
+// PaginationQueryType must implement PaginatedQuery[OptionsType] interface +// to ensure proper pagination functionality type MockPaginatedResource[ResourceType any, OptionsType any, PaginationQueryType PaginatedQuery[OptionsType]] struct {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
go.modis excluded by!**/*.modgo.sumis excluded by!**/*.sum,!**/*.sumtools/generator/go.sumis excluded by!**/*.sum,!**/*.sum
📒 Files selected for processing (82)
internal/README.md(2 hunks)internal/api/bulking/mocks_ledger_controller_test.go(9 hunks)internal/api/common/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/controllers_accounts_count.go(1 hunks)internal/api/v1/controllers_accounts_count_test.go(6 hunks)internal/api/v1/controllers_accounts_list.go(1 hunks)internal/api/v1/controllers_accounts_list_test.go(5 hunks)internal/api/v1/controllers_accounts_read.go(2 hunks)internal/api/v1/controllers_accounts_read_test.go(4 hunks)internal/api/v1/controllers_balances_aggregates.go(1 hunks)internal/api/v1/controllers_balances_aggregates_test.go(1 hunks)internal/api/v1/controllers_balances_list.go(1 hunks)internal/api/v1/controllers_logs_list.go(1 hunks)internal/api/v1/controllers_logs_list_test.go(3 hunks)internal/api/v1/controllers_transactions_count.go(1 hunks)internal/api/v1/controllers_transactions_count_test.go(3 hunks)internal/api/v1/controllers_transactions_list.go(1 hunks)internal/api/v1/controllers_transactions_list_test.go(4 hunks)internal/api/v1/controllers_transactions_read.go(2 hunks)internal/api/v1/controllers_transactions_read_test.go(2 hunks)internal/api/v1/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/utils.go(1 hunks)internal/api/v2/common.go(2 hunks)internal/api/v2/controllers_accounts_count.go(1 hunks)internal/api/v2/controllers_accounts_count_test.go(4 hunks)internal/api/v2/controllers_accounts_list.go(1 hunks)internal/api/v2/controllers_accounts_list_test.go(5 hunks)internal/api/v2/controllers_accounts_read.go(2 hunks)internal/api/v2/controllers_accounts_read_test.go(4 hunks)internal/api/v2/controllers_balances.go(1 hunks)internal/api/v2/controllers_balances_test.go(2 hunks)internal/api/v2/controllers_logs_list.go(1 hunks)internal/api/v2/controllers_logs_list_test.go(5 hunks)internal/api/v2/controllers_transactions_count.go(1 hunks)internal/api/v2/controllers_transactions_count_test.go(4 hunks)internal/api/v2/controllers_transactions_list.go(1 hunks)internal/api/v2/controllers_transactions_list_test.go(4 hunks)internal/api/v2/controllers_transactions_read.go(2 hunks)internal/api/v2/controllers_transactions_read_test.go(2 hunks)internal/api/v2/controllers_volumes.go(1 hunks)internal/api/v2/controllers_volumes_test.go(4 hunks)internal/api/v2/mocks_ledger_controller_test.go(9 hunks)internal/controller/ledger/controller.go(1 hunks)internal/controller/ledger/controller_default.go(4 hunks)internal/controller/ledger/controller_default_test.go(10 hunks)internal/controller/ledger/controller_generated_test.go(9 hunks)internal/controller/ledger/controller_with_traces.go(9 hunks)internal/controller/ledger/stats.go(1 hunks)internal/controller/ledger/stats_test.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/controller/ledger/store_generated_test.go(5 hunks)internal/storage/bucket/default_bucket.go(3 hunks)internal/storage/ledger/accounts.go(3 hunks)internal/storage/ledger/accounts_test.go(11 hunks)internal/storage/ledger/balances.go(3 hunks)internal/storage/ledger/balances_test.go(1 hunks)internal/storage/ledger/debug.go(1 hunks)internal/storage/ledger/errors.go(0 hunks)internal/storage/ledger/legacy/accounts.go(6 hunks)internal/storage/ledger/legacy/accounts_test.go(19 hunks)internal/storage/ledger/legacy/adapters.go(2 hunks)internal/storage/ledger/legacy/balances.go(1 hunks)internal/storage/ledger/legacy/balances_test.go(8 hunks)internal/storage/ledger/legacy/logs.go(1 hunks)internal/storage/ledger/legacy/logs_test.go(2 hunks)internal/storage/ledger/legacy/queries.go(1 hunks)internal/storage/ledger/legacy/transactions.go(6 hunks)internal/storage/ledger/legacy/transactions_test.go(8 hunks)internal/storage/ledger/legacy/volumes.go(4 hunks)internal/storage/ledger/legacy/volumes_test.go(27 hunks)internal/storage/ledger/logs.go(3 hunks)internal/storage/ledger/logs_test.go(3 hunks)internal/storage/ledger/moves.go(1 hunks)internal/storage/ledger/moves_test.go(1 hunks)internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)
⛔ Files not processed due to max files limit (9)
- internal/storage/ledger/resource_volumes.go
- internal/storage/ledger/store.go
- internal/storage/ledger/transactions.go
- internal/storage/ledger/transactions_test.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/volumes.go
- internal/storage/ledger/volumes_test.go
- internal/volumes.go
- test/e2e/api_transactions_list_test.go
💤 Files with no reviewable changes (1)
- internal/storage/ledger/errors.go
🚧 Files skipped from review as they are similar to previous changes (41)
- internal/storage/ledger/paginator.go
- internal/api/v1/controllers_transactions_list.go
- internal/api/v1/controllers_transactions_read_test.go
- internal/api/v1/controllers_accounts_read.go
- internal/api/v2/controllers_transactions_read_test.go
- internal/controller/ledger/stats.go
- internal/README.md
- internal/api/v2/controllers_logs_list.go
- internal/api/v1/controllers_accounts_list_test.go
- internal/api/v1/controllers_accounts_count_test.go
- internal/storage/ledger/legacy/balances_test.go
- internal/storage/ledger/legacy/logs_test.go
- internal/api/v2/controllers_accounts_read.go
- internal/storage/ledger/debug.go
- internal/storage/ledger/legacy/logs.go
- internal/api/v2/controllers_transactions_read.go
- internal/api/v1/controllers_accounts_count.go
- internal/api/v2/controllers_transactions_list.go
- internal/api/v1/controllers_balances_aggregates_test.go
- internal/api/v1/controllers_balances_list.go
- internal/api/v2/controllers_transactions_count.go
- internal/api/v1/controllers_accounts_list.go
- internal/storage/ledger/legacy/balances.go
- internal/storage/ledger/legacy/accounts_test.go
- internal/api/v2/controllers_balances_test.go
- internal/api/v1/controllers_transactions_read.go
- internal/storage/ledger/paginator_column.go
- internal/api/v1/controllers_transactions_count.go
- internal/api/v1/controllers_accounts_read_test.go
- internal/controller/ledger/stats_test.go
- internal/api/v1/controllers_balances_aggregates.go
- internal/api/v2/controllers_accounts_list.go
- internal/api/v2/controllers_accounts_read_test.go
- internal/storage/ledger/legacy/volumes_test.go
- internal/api/v2/controllers_accounts_count.go
- internal/storage/ledger/legacy/volumes.go
- internal/storage/ledger/accounts_test.go
- internal/api/v2/controllers_volumes.go
- internal/storage/ledger/legacy/accounts.go
- internal/storage/ledger/legacy/transactions.go
- internal/api/v2/controllers_logs_list_test.go
🔇 Additional comments (71)
internal/storage/ledger/resource_logs.go (2)
21-26: LGTM! Query construction looks secure and well-structured.
The implementation correctly uses parameterized queries and proper table name prefixing.
37-39: LGTM! Clear error handling for unsupported operation.
The method appropriately indicates that expansion is not supported with a clear error message.
internal/api/v2/controllers_volumes_test.go (2)
149-150: LGTM! Mock expectations are correctly updated
The mock expectations are properly aligned with the new query type, maintaining the test's integrity.
34-34: Verify type consistency across the codebase
The change from PaginatedQueryOptions[FiltersForVolumes] to OffsetPaginatedQuery[GetVolumesOptions] appears to be part of a broader refactoring effort.
✅ Verification successful
Type change appears inconsistent across the codebase
The search results reveal that both types are currently present in the codebase, indicating an incomplete migration:
- The old type
PaginatedQueryOptions[FiltersForVolumes]is still used ininternal/storage/ledger/legacy/and referenced in documentation - The new type
OffsetPaginatedQuery[GetVolumesOptions]is used in most other locations including tests and controllers
Let's verify if this is an intentional split between legacy and new implementations:
The type change is part of a legacy transition
The search results reveal that the old type PaginatedQueryOptions[FiltersForVolumes] is intentionally maintained in the legacy package, while the new type OffsetPaginatedQuery[GetVolumesOptions] is used in the main implementation. This is a common pattern when maintaining backward compatibility during a transition period.
- The old type exists in
internal/storage/ledger/legacy/ - The new type is used consistently in all non-legacy code
- Documentation in
internal/controller/ledger/README.mdstill references the old type, suggesting it needs updating
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for consistent usage of the new type across the codebase
# Search for any remaining usage of the old type
rg "PaginatedQueryOptions\[.*FiltersForVolumes\]"
# Search for consistent usage of the new type
rg "OffsetPaginatedQuery\[.*GetVolumesOptions\]"
Length of output: 9792
Script:
#!/bin/bash
# Check if there's a migration or transition plan documented
rg -A 5 "FiltersForVolumes|GetVolumesOptions" internal/storage/ledger/legacy/README.md internal/controller/ledger/README.md
# Check imports to understand the relationship between old and new types
ast-grep --pattern 'import $$$' internal/storage/ledger/legacy/volumes.go
Length of output: 3484
internal/api/v2/common.go (3)
4-4: Avoid dot imports for better code clarity
Using dot imports can lead to namespace pollution and reduce code readability.
83-102: Validate column names to prevent SQL injection
The defaultPaginationColumn parameter should be validated against a whitelist of allowed column names.
57-61: 🛠️ Refactor suggestion
Qualify collectionutils function calls
Following up on the dot import issue, these function calls should be qualified with the package name.
- return Flatten(
- Map(r.URL.Query()["expand"], func(from string) []string {
+ return collectionutils.Flatten(
+ collectionutils.Map(r.URL.Query()["expand"], func(from string) []string {Likely invalid or redundant comment.
internal/api/v2/controllers_accounts_list_test.go (6)
56-63: LGTM! Query builder implementation is consistent
The metadata and address query structures correctly implement the new pagination approach with appropriate query builders.
Also applies to: 69-76
114-120: LGTM! Page size limit handling is correct
The implementation properly enforces the maximum page size limit while maintaining consistency with the new pagination structure.
126-133: LGTM! Filter implementations are well structured
The balance and exists filter queries correctly implement their respective filters while maintaining consistency with the new pagination approach.
Also applies to: 139-146
160-194: LGTM! Error handling is comprehensive
The test cases thoroughly cover various error scenarios with appropriate error types and status codes.
217-217: LGTM! Mock expectation is correctly updated
The mock expectation properly reflects the new query type structure.
32-32: Verify the type change impact on other controllers
The change from PaginatedQueryOptions[ledgercontroller.PITFilterWithVolumes] to OffsetPaginatedQuery[any] suggests a significant shift in pagination strategy.
✅ Verification successful
Type change is part of a broader pagination refactoring
The change from PaginatedQueryOptions[ledgercontroller.PITFilterWithVolumes] to OffsetPaginatedQuery[any] is consistent with the codebase. The evidence shows:
PITFilterWithVolumesis only used in legacy code (internal/storage/ledger/legacy/*)- The new
OffsetPaginatedQuerytype is widely adopted across the codebase - The controller implementation in
controllers_accounts_list.goalready uses the new pagination approach withgetOffsetPaginatedQuery[any] - All test cases in the file consistently use the new type
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for other controllers that might need similar updates
rg -l "PaginatedQueryOptions\[ledgercontroller\.PITFilterWithVolumes\]" --type go
# Search for existing usage of OffsetPaginatedQuery
rg -l "OffsetPaginatedQuery" --type go
Length of output: 1510
Script:
#!/bin/bash
# Let's check the actual implementation of OffsetPaginatedQuery to understand the change better
ast-grep --pattern 'type OffsetPaginatedQuery[$_] struct {
$$$
}'
# Also check for PaginatedQueryOptions to compare
ast-grep --pattern 'type PaginatedQueryOptions[$_] struct {
$$$
}'
# Let's also check the imports in the test file to understand the type resolution
rg -p "^import \(" -A 20 internal/api/v2/controllers_accounts_list_test.go
Length of output: 868
Script:
#!/bin/bash
# Let's check the actual usage of these types in the test file
rg -p "OffsetPaginatedQuery" internal/api/v2/controllers_accounts_list_test.go -B 2 -A 2
# And check the implementation in the controller itself
rg -p "OffsetPaginatedQuery" internal/api/v2/controllers_accounts_list.go -B 2 -A 2
# Check if there are any other test files using PITFilterWithVolumes
rg "PITFilterWithVolumes" --type go -l
Length of output: 3364
internal/storage/ledger/paginator_offset.go (3)
71-71: Interface assertion is correct.
The assertion ensures that offsetPaginator implements paginator, which is valuable for compile-time checks.
17-36:
Consider adding boundary checks for large offset and page size to avoid overflow.
While the logic for applying an offset and limit here appears correct, be aware that extremely large values for Offset and PageSize might cause issues, particularly when casting to int. Consider enforcing upper bounds or using 64-bit integer operations carefully to avoid potential overflow.
func (o offsetPaginator[ResourceType, OptionsType]) paginate(sb *bun.SelectQuery, query ledgercontroller.OffsetPaginatedQuery[OptionsType]) *bun.SelectQuery {
paginationColumn := o.defaultPaginationColumn
originalOrder := o.defaultOrder
if query.Order != nil {
originalOrder = *query.Order
}
orderExpression := fmt.Sprintf("%s %s", paginationColumn, originalOrder)
sb = sb.ColumnExpr("row_number() OVER (ORDER BY " + orderExpression + ")")
- if query.Offset > 0 {
- sb = sb.Offset(int(query.Offset))
- }
+ // Optional boundary check
+ if query.Offset > math.MaxInt32 {
+ return sb // Or handle the overflow scenario as needed
+ } else if query.Offset > 0 {
+ sb = sb.Offset(int(query.Offset))
+ }
if query.PageSize > 0 {
sb = sb.Limit(int(query.PageSize) + 1)
}
return sb
}Likely invalid or redundant comment.
39-69: 🛠️ Refactor suggestion
Verify the offset arithmetic in buildCursor for potential overflow.
When adding or subtracting PageSize from Offset, ensure 64-bit operations remain safe for large numeric combinations to avoid negative or overflow conditions.
func (o offsetPaginator[ResourceType, OptionsType]) buildCursor(ret []ResourceType, query ledgercontroller.OffsetPaginatedQuery[OptionsType]) (*bunpaginate.Cursor[ResourceType], error) {
var previous, next *ledgercontroller.OffsetPaginatedQuery[OptionsType]
// Page with transactions before
if query.Offset > 0 {
cp := query
- offset := int(query.Offset) - int(query.PageSize)
+ // Use 64-bit arithmetic
+ offset64 := int64(query.Offset) - int64(query.PageSize)
+ if offset64 < 0 {
+ offset64 = 0
+ }
+ offset := uint64(offset64)
cp.Offset = offset
previous = &cp
}
// Page with transactions after
if query.PageSize != 0 && len(ret) > int(query.PageSize) {
cp := query
- cp.Offset = query.Offset + query.PageSize
+ // Safe addition to avoid overflow
+ if query.Offset > math.MaxUint64 - query.PageSize {
+ return nil, fmt.Errorf("offset overflow while computing next page")
+ }
+ cp.Offset = query.Offset + query.PageSize
next = &cp
ret = ret[:len(ret)-1]
}
return &bunpaginate.Cursor[ResourceType]{
PageSize: int(query.PageSize),
HasMore: next != nil,
Previous: encodeCursor[OptionsType, ledgercontroller.OffsetPaginatedQuery[OptionsType]](previous),
Next: encodeCursor[OptionsType, ledgercontroller.OffsetPaginatedQuery[OptionsType]](next),
Data: ret,
}, nil
}Likely invalid or redundant comment.
internal/api/v1/utils.go (2)
46-65: Validate column against SQL injection.
If the column parameter is derived from user input, consider validating it against known allowed column names. Storing a whitelist in code or configuration helps mitigate injection risk.
Because a similar suggestion was made in a previous review, I will mark this accordingly.
27-44: Ensure thorough testing for getOffsetPaginatedQuery.
This function constructs an OffsetPaginatedQuery by combining ResourceQuery with extracted page size. Please verify in tests that edge cases (e.g., zero or excessively large pageSize) produce the expected behavior.
✅ Verification successful
Edge cases for getOffsetPaginatedQuery are already well tested
The function's edge cases are effectively covered through integration tests in controllers_accounts_list_test.go. The test suite includes cases for:
- Invalid page size ("nan")
- Page size exceeding maximum (1000000 -> MaxPageSize)
- Empty cursor
- Invalid cursor format
- Default page size behavior
- Various query parameters and filters
The implementation also uses the bunpaginate package's utility functions with proper configuration:
WithMaxPageSize(MaxPageSize)WithDefaultPageSize(DefaultPageSize)
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Script to identify unit tests for getOffsetPaginatedQuery
rg -A 10 "func Test.*getOffsetPaginatedQuery"
Length of output: 47
Script:
#!/bin/bash
# Let's first check if there are any test files in the same directory
fd ".*_test.go" internal/api/v1/
# Then let's search for any tests that might be using this function
rg -l "getOffsetPaginatedQuery" internal/api/v1/
# Also check if there are any tests for the related bunpaginate package
fd ".*_test.go" --full-path ".*bunpaginate.*"
# Look for any tests that might be testing pagination functionality
rg -l "TestPagination|TestOffset"
Length of output: 1415
Script:
#!/bin/bash
# Let's check the content of api_utils_test.go since it's in the same directory as utils.go
cat internal/api/v1/api_utils_test.go
# Also check the content of controllers that use this function to see if they have related tests
cat internal/api/v1/controllers_accounts_list_test.go
cat internal/api/v1/controllers_balances_list.go
Length of output: 6781
internal/storage/ledger/legacy/queries.go (2)
36-43: Check column mutation side effects.
When calling WithColumn, the ret pointer is updated and returned. Confirm that it doesn’t inadvertently mutate the original query in a shared context if the original query object is intended to be immutable.
28-34: Confirm that PageSize behavior matches your standard.
The constructor sets PageSize = opts.PageSize, and order to ascending. Ensure that this ordering is consistent with your typical usage, or consider providing an override.
✅ Verification successful
Let me check the usage of NewPaginatedQueryOptions to understand how the page size is set.
Let me try a different approach to find the implementation.
Let me check the implementation of NewPaginatedQueryOptions and WithPageSize methods.
Let me check the default page size and any constants related to pagination.
Based on the search results, I can now generate the final response:
The ascending order and page size behavior is consistent with the codebase standards.
The default page size is handled by the system (15 by default, max 1000) and can be overridden using WithPageSize(). The ascending order is appropriate for volumes listing as it's consistently used across volume-related queries in the test cases. No changes needed.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Searching for references of NewGetVolumesWithBalancesQuery to see usage patterns
rg -A 5 "NewGetVolumesWithBalancesQuery"
Length of output: 17690
Script:
#!/bin/bash
# Search for NewPaginatedQueryOptions implementation
ast-grep --pattern 'func NewPaginatedQueryOptions'
Length of output: 52
Script:
#!/bin/bash
# Search for PaginatedQueryOptions using ripgrep
rg -A 5 "type PaginatedQueryOptions"
Length of output: 1710
Script:
#!/bin/bash
# Search for NewPaginatedQueryOptions and WithPageSize implementations
rg -A 5 "func (.*PaginatedQueryOptions.*WithPageSize|func NewPaginatedQueryOptions)"
Length of output: 1784
Script:
#!/bin/bash
# Search for pagination-related constants and default values
rg -A 5 "const.*Page|DefaultPage|PageSize"
Length of output: 90376
internal/storage/ledger/legacy/adapters.go (4)
26-29: Logs method is straightforward.
Returning d.newStore.Logs() is consistent with the approach for other resources. The logic is simple and correct.
30-33: Transactions method is straightforward.
Same approach as Logs. Looks good.
38-40: Volumes method is aligned with the new store design.
By returning the new store’s volumes resource directly, you maintain a consistent resource-based approach.
70-70: UpsertAccounts signature is correct.
The function uses a variadic parameter for performance or convenience. The approach looks solid.
internal/storage/ledger/resource_transactions.go (2)
153-154: Avoid panicking in production code.
Returning an error instead of using panic gives you more control and prevents unexpected crashes.
126-130: 🛠️ Refactor suggestion
Redundant timestamp case leads to unreachable code.
You have “timestamp” handled both at line 126 (part of "reference" or "timestamp") and at line 149. If the property is "timestamp", the first case will match, rendering the second case block unreachable. Removing one of them is recommended to avoid confusion and unreachable code.
case property == "reference" || property == "timestamp":
return fmt.Sprintf("%s %s ?", property, convertOperatorToSQL(operator)), []any{value}, nil
-case property == "timestamp":
- return fmt.Sprintf("timestamp %s ?", convertOperatorToSQL(operator)), []any{value}, nilLikely invalid or redundant comment.
internal/storage/ledger/resource_accounts.go (2)
113-116: Guard regex submatches to prevent out-of-range indexing.
Without checking the length of the slice from FindAllStringSubmatch, code accessing [0][1] risks a panic when no match is found.
180-180: Safely handle string concatenation to prevent injection.
Instead of concatenating the column alias, consider using parameterized queries or Bun’s identifier-escaping APIs.
internal/storage/ledger/resource.go (2)
17-31: Return an error instead of panicking on unsupported operators.
Relying on panic can crash the application unexpectedly. Graceful error handling is recommended.
283-283: Avoid panic for unexpected pagination query types.
Return an error or handle the case explicitly rather than panicking with “should not happen.”
internal/controller/ledger/controller_default.go (3)
110-112: Check database migrations with caution.
Implementation for IsDatabaseUpToDate is straightforward. The usage looks correct.
118-119: New resource-based query signatures appear consistent.
Your refactored methods (ListTransactions, CountTransactions, GetTransaction, CountAccounts, ListAccounts, GetAccount, GetAggregatedBalances, ListLogs, GetVolumesWithBalances) cleanly align with the new ResourceQuery interface.
Also applies to: 122-123, 126-127, 130-131, 134-135, 138-139, 142-147, 150-151, 154-155
292-295: Reasonable pagination settings for export.
PageSize=100 should be performant for most use cases, though a dynamic or configurable value might be beneficial for large data sets.
internal/storage/ledger/moves.go (1)
10-10: LGTM! The receiver rename improves readability.
The changes to InsertMoves are well-structured:
- Consistent renaming of receiver from
stostoreimproves code clarity - Core functionality preserved while maintaining proper error handling
Also applies to: 14-15, 17-17, 19-20
internal/api/v2/controllers_balances.go (1)
15-19: LGTM! Clean refactoring of query handling.
The consolidation of query building logic improves code maintainability.
internal/api/v1/controllers_logs_list.go (2)
38-41: LGTM! Clean pagination implementation.
The use of getColumnPaginatedQuery with proper type parameters and error handling improves the code structure.
44-46: LGTM! Clean integration of query building.
The assignment of the query builder and the subsequent list operation are well-structured.
internal/api/v1/controllers_transactions_count_test.go (1)
25-25: LGTM: Test cases properly updated for new generic query type
The transition from PaginatedQueryOptions[ledgercontroller.PITFilterWithVolumes] to ResourceQuery[any] is well implemented across all test cases. The test coverage remains comprehensive, testing various query scenarios including metadata, time ranges, and specific field matches.
Also applies to: 34-97
internal/storage/ledger/logs.go (2)
Line range hint 98-104: Good error handling for idempotency key conflicts
The error handling for idempotency key conflicts is well implemented, providing clear error messages and proper error type conversion.
57-61: Verify the advisory lock implementation
The advisory lock implementation for FeatureHashLogs looks correct, but let's verify its usage across the codebase to ensure consistent locking patterns.
✅ Verification successful
Let me gather more context about the feature flag usage to ensure the locking is properly guarded.
Based on the gathered context, I can now provide a final response:
Advisory lock implementation is correctly guarded and consistent
The advisory lock implementation is properly guarded by the FeatureHashLogs feature flag and is used consistently in the codebase. The feature flag is well-defined with "SYNC" and "DISABLED" as valid states, and the locking mechanism is only activated when the feature is in "SYNC" mode. The lock uses the ledger's ID as the lock key, ensuring proper isolation between different ledgers.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for other advisory lock implementations to verify consistency
rg -A 3 "pg_advisory_xact_lock" --type go
Length of output: 921
Script:
#!/bin/bash
# Search for FeatureHashLogs usage
rg -A 3 "FeatureHashLogs" --type go
# Check the feature flag definition
ast-grep --pattern 'const FeatureHashLogs = $_'
Length of output: 32990
internal/storage/ledger/accounts.go (2)
7-9: Verify tracing implementation consistency
The tracing implementation looks good, but let's verify it's consistent across the codebase.
✅ Verification successful
Tracing implementation is consistent across the codebase
The tracing implementation shows a consistent pattern throughout the codebase:
- All storage operations use
tracing.TraceWithMetric - Controller operations consistently use tracing with proper span naming
- OpenTelemetry imports are consistently used across all relevant files
- Proper attribute setting and context propagation is maintained
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for tracing patterns to ensure consistency
ast-grep --pattern 'tracing.TraceWithMetric(
$_,
$_,
$tracer,
$_histogram,
$_
)'
Length of output: 92
Script:
#!/bin/bash
# Let's search for tracing usage patterns more broadly
rg -A 3 "tracing\." --type go
# Also check for OpenTelemetry imports
rg "go\.opentelemetry\.io/otel" --type go
Length of output: 19790
Line range hint 21-71: Verify the impact of metadata update changes
The addition of updated_at in the conflict resolution is good, but we should verify that this change doesn't affect any existing metadata update patterns.
✅ Verification successful
Let me gather more information about the metadata update patterns and the schema to ensure we have a complete picture.
Let me try a different approach to gather information about the schema and metadata handling.
Let me check the metadata update patterns in the codebase more specifically.
Based on the gathered information, I can now provide a final response. The SQL migrations and tests show that the metadata update behavior is consistent and well-defined:
Metadata update behavior is safe and working as intended
The implementation correctly handles metadata updates by:
- Using
||operator for JSONB concatenation with proper conflict resolution - Including
updated_attimestamp updates - Checking for existing metadata with
@>containment operator to prevent unnecessary updates - Maintaining metadata history through triggers
The change is verified through:
- SQL migrations showing consistent metadata merge behavior
- Multiple test cases covering metadata updates
- Proper database constraints and indexes for metadata operations
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for metadata update patterns in tests and implementation
rg -A 5 "metadata.*update" --type go
# Search for updated_at field usage
rg -A 3 "updated_at.*excluded" --type go
Length of output: 1677
Script:
#!/bin/bash
# Check schema definition for accounts table
rg -A 10 "CREATE TABLE.*accounts" --type sql
# Look for metadata related tests
ast-grep --pattern 'func Test$_($_) {
$$$
metadata $$$
$$$
}'
# Check for other metadata update implementations
ast-grep --pattern 'func $_($_ *Store) {
$$$
metadata $$$
$$$
}'
Length of output: 189
Script:
#!/bin/bash
# Look for SQL migrations related to accounts
fd -e sql | xargs cat
# Check for metadata related test files
fd -e go test | xargs rg "metadata" -B 2 -A 2
# Look for Store method implementations with metadata
rg "func.*Store.*metadata" -B 1 -A 5 --type go
Length of output: 70180
Script:
#!/bin/bash
# Look for metadata update patterns in accounts table
rg -A 5 "metadata.*accounts" --type sql
# Check for metadata update tests
fd test -e go | xargs rg "UpdateAccountsMetadata" -B 2 -A 2
# Look for metadata merge behavior
rg "metadata.*\|\|" --type sql -A 2
Length of output: 17352
internal/api/v1/controllers_logs_list_test.go (2)
38-43: LGTM: Clear and well-structured test case for nominal scenario
The nominal test case correctly validates the new ColumnPaginatedQuery structure with appropriate defaults:
- PageSize set to DefaultPageSize
- Column set to "id"
- Order set to descending
50-57: Verify time-based filtering implementation
The test cases for start_time and end_time filtering look correct, but we should verify the actual implementation handles these filters consistently.
Also applies to: 64-71
✅ Verification successful
Let me check the actual implementation of the query builders to ensure complete verification.
Let me try a different approach to find the query builder implementation.
Let me try one final search to locate the query builder implementation.
Based on the code analysis, I can now provide the final response:
Time-based filtering is consistently implemented
The implementation shows consistent handling of time-based filters across logs and transactions:
- Both controllers use the same query building pattern with
Gte("date", startTime)for start_time andLt("date", endTime)for end_time - The filtering is properly integrated into the query builder system
- Test cases correctly validate the expected query construction
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if the date filtering is consistently implemented across the codebase
rg -A 5 "start_time|end_time" --type go
Length of output: 6877
Script:
#!/bin/bash
# Check the query builder implementation
ast-grep --pattern 'func Gte($_, $_) $_ {
$$$
}'
# Also check Lt implementation
ast-grep --pattern 'func Lt($_, $_) $_ {
$$$
}'
Length of output: 104
Script:
#!/bin/bash
# Search for the query package and its implementation
fd query.go --type f --exec cat {}
Length of output: 1027
Script:
#!/bin/bash
# Search for query builder related files and implementations
rg -g '!*.test.go' -A 5 'type.*Builder.*struct|func.*Builder' --type go
Length of output: 5570
internal/storage/ledger/balances.go (1)
96-103: Verify query performance with large datasets
The complex query with multiple CTEs (Common Table Expressions) might impact performance with large datasets.
✅ Verification successful
The previous search might have been too specific. Let's broaden our search to find any related tests, benchmarks, or similar query patterns that could help us understand the performance implications better.
Let's examine the actual test files to understand if there's any performance testing coverage for these database operations.
Performance considerations validated through test coverage
The codebase has comprehensive test coverage including performance benchmarks and integration tests that validate the query behavior:
- The
balances_test.goincludes concurrent access tests and various balance query scenarios with different data patterns - The project has dedicated performance testing infrastructure in
test/performance/with configurable parallelism and metrics collection - The test suite validates the query behavior with transactions, multiple CTEs, and balance aggregations
The complex query structure is intentional and well-tested. The performance impact has been considered in the test infrastructure.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Look for existing performance tests or benchmarks
rg -l "Benchmark.*Balance" --type go
Length of output: 38
Script:
#!/bin/bash
# Look for any existing tests related to this query
rg -l "TestBalance|test.*balance" --type go
# Check for any existing benchmarks in the codebase
rg -l "Benchmark" --type go
# Look for similar query patterns to understand if this is a common pattern
ast-grep --pattern 'NewSelect().With($$$)'
Length of output: 362
Script:
#!/bin/bash
# Let's examine the test files to see if they cover performance scenarios
cat internal/storage/ledger/balances_test.go
# Also check the performance test files
cat test/performance/write_test.go test/performance/benchmark_test.go
Length of output: 19391
internal/controller/ledger/controller.go (1)
27-35: Well-structured transition to generic query types
The change to use generic query types (ResourceQuery[any], ColumnPaginatedQuery[any], etc.) improves flexibility while maintaining type safety. This is a good architectural decision.
However, verify that all implementations of this interface have been updated accordingly.
✅ Verification successful
Interface implementations are correctly aligned with generic query types
The verification confirms that all implementations (DefaultController, ControllerWithTraces) and test mocks have been properly updated to use the new ResourceQuery[any] type signature. The changes are consistently applied across:
- Main implementation in
controller_default.go - Tracing wrapper in
controller_with_traces.go - Test mocks in both v1 and v2 API packages
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Find all implementations of the Controller interface
ast-grep --pattern 'type $_ struct {
$$$
}
func ($_ $_) GetAccount($_, $_) ($_, $_) {
$$$
}'
Length of output: 99
Script:
#!/bin/bash
# Let's try a different approach to find implementations
# First, let's find all files that might contain implementations
rg -l "GetAccount.*ResourceQuery" .
# Then look for struct types in those files
rg "type \w+Controller struct" .
# And also check for interface implementation patterns
rg "var _ .+Controller = &\w+Controller{}" .
Length of output: 1737
Script:
#!/bin/bash
# Let's examine the DefaultController implementation in detail
cat ./internal/controller/ledger/controller_default.go | grep -A 2 "func.*GetAccount"
# Also check the controller with traces
cat ./internal/controller/ledger/controller_with_traces.go | grep -A 2 "func.*GetAccount"
# Let's also verify the method signatures in test mocks to ensure they match
rg "func.*GetAccount.*ResourceQuery" ./internal/api/v1/mocks_ledger_controller_test.go ./internal/api/v2/mocks_ledger_controller_test.go
Length of output: 969
internal/api/v2/controllers_accounts_count_test.go (1)
29-29: LGTM: Type change aligns with the refactoring goals
The transition from PaginatedQueryOptions[ledgercontroller.PITFilterWithVolumes] to ResourceQuery[any] provides more flexibility while maintaining type safety.
internal/storage/ledger/moves_test.go (2)
173-186: Consider using a fixed timestamp for deterministic testing
The test relies on runtime timestamps which could lead to flaky behavior. Consider using fixed timestamps for more deterministic results.
173-177: Verify the impact of UseInsertionDate option
The test should explicitly verify that using insertion dates produces different results compared to using effective dates.
internal/storage/ledger/logs_test.go (3)
124-127: LGTM! The pagination changes maintain hash consistency.
The switch to Logs().Paginate with explicit ascending order ensures that logs are processed in chronological order, which is crucial for hash consistency verification.
185-186: LGTM! Basic pagination test case is properly structured.
The test case correctly verifies the default pagination behavior using the new ColumnPaginatedQuery type.
192-208: LGTM! Comprehensive test coverage for filtered queries.
The test case thoroughly validates the query builder functionality with date range filtering.
internal/storage/ledger/resource_aggregated_balances.go (1)
133-150: LGTM! Clear and well-structured filter resolution.
The method effectively handles both address and metadata filtering with proper error handling and clear logic flow.
internal/controller/ledger/store.go (2)
60-64: LGTM! Well-designed resource-oriented interface methods
The new methods provide a clean and type-safe way to access different resources using generic types. The consistent naming and return types make the interface intuitive to use.
124-126: Handle potential nil account
This code segment still needs nil checks as mentioned in the previous review.
internal/storage/bucket/default_bucket.go (1)
287-304: LGTM! Well-structured index dependencies
The indexes are properly gated behind relevant feature flags, and the combination of requirements is clearly expressed. These indexes will optimize segment-based transaction queries.
internal/storage/ledger/legacy/transactions_test.go (1)
9-9: LGTM! Clean migration to new store types
The test updates properly reflect the new store interface changes while maintaining comprehensive test coverage. The transitions from old to new types are consistent throughout the file.
Also applies to: 50-50, 72-72, 108-108, 162-162, 179-179, 186-186, 191-191, 197-197, 203-203, 209-209, 215-216, 224-224, 230-230, 236-237, 246-246, 252-252, 266-266, 274-274
internal/controller/ledger/controller_with_traces.go (4)
Line range hint 198-208: LGTM: Transaction-related methods successfully refactored to use generic query types
The refactoring of ListTransactions, CountTransactions, and GetTransaction methods to use generic query types (ColumnPaginatedQuery[any] and ResourceQuery[any]) improves code reusability while maintaining type safety.
Also applies to: 210-220, 222-232
Line range hint 234-244: LGTM: Account-related methods successfully refactored to use generic query types
The refactoring of CountAccounts, ListAccounts, and GetAccount methods to use generic query types (ResourceQuery[any] and OffsetPaginatedQuery[any]) maintains consistency with the transaction-related methods.
Also applies to: 246-256, 258-268
Line range hint 270-280: LGTM: Volume and balance methods successfully refactored
The refactoring of GetAggregatedBalances to use ResourceQuery[GetAggregatedVolumesOptions] and ListLogs to use ColumnPaginatedQuery[any] aligns with the overall pattern while preserving specific options where needed.
Also applies to: 282-292
Line range hint 330-342: LGTM: GetVolumesWithBalances successfully refactored
The method now uses OffsetPaginatedQuery[GetVolumesOptions], maintaining consistency with other paginated queries while preserving volume-specific options.
internal/api/v2/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM: V2 API mock correctly implements the updated interface
The mock implementations for the v2 API correctly reflect the interface changes, maintaining consistency with the main controller implementation.
internal/api/v1/mocks_ledger_controller_test.go (2)
178-178: Validate the specialized query type usage
The GetAggregatedBalances method uses a specific type parameter GetAggregatedVolumesOptions while other methods use any. This inconsistency might be intentional but should be documented.
✅ Verification successful
Based on the shell script results, I can now generate the final response:
The specialized query type usage is intentional and consistent
The GetAggregatedVolumesOptions type is consistently used across the codebase for aggregated balance queries. It's specifically designed to support the useInsertionDate flag, which is a unique requirement for aggregated balance operations. Other methods use any when they don't need specialized options.
Key evidence:
- The type is properly defined in
internal/controller/ledger/store.go - It's consistently used across all implementations (DefaultController, ControllerWithTraces, mock implementations)
- The specialized options are actively used in tests and API handlers to control insertion date behavior
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for other specialized query types that might need similar treatment
# Search for potential specialized query options
rg -l "type.*Options.*struct" --type go
# Search for usage of GetAggregatedVolumesOptions
rg "GetAggregatedVolumesOptions" -A 5
Length of output: 24327
73-73: Verify type safety after transitioning to ResourceQuery[any]
The transition from specific query types to ResourceQuery[any] could potentially hide type-related issues at compile time. Ensure that all test cases properly validate the query structure.
Consider using more specific type parameters instead of any where possible to maintain compile-time type safety.
Also applies to: 88-88, 163-163, 223-223
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! Changes match v1 package implementation
The changes in this file mirror those in the v1 package, maintaining consistency across the codebase.
internal/api/bulking/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! Changes match v1 package implementation
The changes in this file mirror those in the v1 package, maintaining consistency across the codebase.
internal/controller/ledger/store_generated_test.go (2)
45-57: LGTM! Well-structured mock implementations.
The new Store interface methods (Accounts, Logs, Transactions, Volumes) follow a consistent pattern and properly utilize generic types for type safety. The implementation correctly uses gomock's recording mechanism for testing.
Also applies to: 233-245, 292-304, 355-367
369-441: LGTM! Well-implemented generic Resource interface.
The MockResource implementation properly handles generic types and includes all necessary methods (Count, GetOne, Query) with correct error handling and parameter passing.
internal/controller/ledger/controller_generated_test.go (3)
Line range hint 237-250: Verify pagination strategy choices
The code uses two different pagination strategies:
OffsetPaginatedQueryfor volumes and accountsColumnPaginatedQueryfor logs and transactions
This split suggests a deliberate choice in pagination strategy. Column-based pagination is typically more efficient for time-series data (logs, transactions) while offset-based pagination works well for relatively static data (accounts).
Also applies to: 281-294, 296-309, 311-324
✅ Verification successful
Pagination strategy choices are correctly implemented
The codebase analysis confirms the deliberate and appropriate use of different pagination strategies:
-
ColumnPaginatedQueryfor time-series data:- Transactions (internal/controller/ledger/controller_default.go)
- Logs (internal/controller/ledger/controller_default.go)
-
OffsetPaginatedQueryfor relatively static data:- Accounts (internal/controller/ledger/controller_default.go)
- Volumes with balances (internal/controller/ledger/controller_default.go)
The implementation is backed by:
- Consistent usage patterns across controllers and stores
- Different page size limits (v1: 1000, v2: 15) showing evolution in API design
- Test coverage for pagination scenarios (test/e2e/api_transactions_list_test.go)
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the pagination strategy implementation and usage patterns
# Look for pagination type definitions
echo "Looking for pagination type definitions..."
rg -p "type\s+(Offset|Column)PaginatedQuery\s*\[" --type go
# Look for pagination cursor usage patterns
echo "Looking for cursor usage patterns..."
rg "bunpaginate\.Cursor\[" --type go -A 3
# Look for any performance-related comments about pagination
echo "Looking for pagination-related comments..."
rg -p "//.*pagination" --type go
Length of output: 40083
Line range hint 177-190: Verify the specialized GetAggregatedVolumesOptions type
Unlike other methods that use ResourceQuery[any], this method uses a specialized GetAggregatedVolumesOptions type. This is appropriate as aggregated volumes likely require specific options for aggregation.
✅ Verification successful
Let me check the definition of GetAggregatedVolumesOptions to ensure it's properly specialized.
Specialized type usage is appropriate and consistent
The GetAggregatedVolumesOptions type is correctly specialized with a specific field UseInsertionDate that's relevant for aggregated volume queries. The type is consistently used across the codebase in the ledger controller, store, and API layers, maintaining type safety. The specialized type helps enforce proper options configuration when querying aggregated balances.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the GetAggregatedVolumesOptions type definition and usage
# Look for the type definition
echo "Looking for GetAggregatedVolumesOptions type definition..."
rg -p "type\s+GetAggregatedVolumesOptions\s+struct" --type go
# Look for other usages to ensure consistency
echo "Looking for other usages of GetAggregatedVolumesOptions..."
rg "GetAggregatedVolumesOptions" --type go
Length of output: 9027
Script:
#!/bin/bash
# Get the definition of GetAggregatedVolumesOptions
rg -A 10 "type\s+GetAggregatedVolumesOptions\s+struct" --type go
Length of output: 572
Line range hint 72-85: Verify interface alignment with implementation
The methods have been updated to use the generic ResourceQuery[any] type, which is part of a broader refactoring effort to unify query handling. While the changes look good, we should verify that these mock interfaces align with the actual implementation.
Also applies to: 87-100, 162-175, 222-235
✅ Verification successful
Let me fix the regex and try again to verify the interface alignment.
Let me search for the complete interface definition and its implementation to verify the alignment.
Mock implementation correctly aligns with the interface and concrete implementations
The verification shows perfect alignment between:
- The interface definition in
controller.gowhich declaresCountAccounts(ctx context.Context, query ResourceQuery[any]) (int, error) - The mock implementation in the test file
- The concrete implementations in both
controller_default.goandcontroller_with_traces.go
The changes to use ResourceQuery[any] are consistent across the codebase, including other similar methods like GetAccount, CountTransactions, and GetTransaction.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that the mock interfaces align with the actual implementation
# Look for the Controller interface definition and its implementation
# Find the Controller interface definition
echo "Looking for Controller interface definition..."
rg -p "type\s+Controller\s+interface\s*{" --type go
# Find the actual implementation of these methods
echo "Looking for actual implementations..."
ast-grep --pattern 'func $_($_, $_ ResourceQuery[any]) ($_, $_) {
$$$
}'
Length of output: 448
Script:
#!/bin/bash
# Find the Controller interface definition
echo "Looking for Controller interface definition..."
rg -p "type Controller interface" --type go -A 10
# Find the actual implementation of these methods
echo "Looking for actual implementations..."
ast-grep --pattern 'type Controller interface {
$$$
CountAccounts(context.Context, ResourceQuery[any]) (int, error)
$$$
}'
# Also check the concrete implementation
echo "Looking for concrete implementation..."
ast-grep --pattern 'func ($_ *$_) CountAccounts(ctx context.Context, query ResourceQuery[any]) (int, error) {
$$$
}'
Length of output: 2608
Script:
#!/bin/bash
# Get the complete ledger Controller interface
echo "Looking for complete ledger Controller interface..."
rg -p "type Controller interface" internal/controller/ledger/controller.go -A 30
# Find the actual implementation of CountAccounts in the concrete controller
echo "Looking for concrete CountAccounts implementation..."
rg -p "func.*CountAccounts.*ResourceQuery" --type go -g '!*test.go' -g '!*mock*.go'
# Check if there are any other query-related methods in the interface
echo "Looking for other query-related methods..."
rg "Query" internal/controller/ledger/controller.go -B 1 -A 1
Length of output: 3821
gfyrag
force-pushed
the
refactor/store-read-part
branch
from
881f50a to
48ed807
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (6)
internal/storage/bucket/default_bucket.go (1)
264-271:⚠️ Potential issueRemove unnecessary array length index
The
accounts_address_array_lengthindex appears to be unused in queries, as identified in previous reviews. Consider removing this index to reduce storage overhead and improve write performance unless there's a specific upcoming use case.internal/api/v1/utils.go (1)
46-65:⚠️ Potential issuePotential SQL injection vulnerability in column parameter
The
columnparameter is used directly in database queries without validation.Apply this diff to implement column validation:
+func isValidColumnName(column string) bool { + allowedColumns := map[string]bool{ + "id": true, + "name": true, + "date": true, + // Add other allowed columns here + } + return allowedColumns[column] +} func getColumnPaginatedQuery[v any](r *http.Request, column string, order bunpaginate.Order, modifiers ...func(*v) error) (*ledgercontroller.ColumnPaginatedQuery[v], error) { + if !isValidColumnName(column) { + return nil, fmt.Errorf("invalid column name: %s", column) + } // Rest of the functioninternal/api/v2/common.go (3)
4-4: 🛠️ Refactor suggestionAvoid dot imports for better code clarity
Using dot imports can lead to namespace pollution and reduce code readability.
Apply this diff:
- . "github.com/formancehq/go-libs/v2/collectionutils" + "github.com/formancehq/go-libs/v2/collectionutils"Update function calls accordingly in getExpand.
56-62: 🛠️ Refactor suggestionAdd validation for expand parameters
The expand parameters are processed without validation, which could lead to security issues.
Add validation to ensure only allowed expand paths are processed.
83-102:⚠️ Potential issueValidate defaultPaginationColumn parameter
Similar to v1, the column parameter needs validation to prevent SQL injection.
Implement column name validation as suggested in the previous review.
internal/controller/ledger/store.go (1)
124-126:⚠️ Potential issueAdd nil check to prevent potential panic
The GetAccount implementation should check if the account is nil before returning it.
Apply this diff:
account, err := v.Store.Accounts().GetOne(ctx, ResourceQuery[any]{ Builder: query.Match("address", address), }) if err != nil { return nil, err } +if account == nil { + return nil, fmt.Errorf("account not found: %s", address) +} return account, nil
🧹 Nitpick comments (14)
internal/storage/ledger/legacy/adapters.go (1)
21-21: Clarify the TODO comment for compatibility.The comment indicates a future plan to handle compatibility with v1. Be explicit about what tasks remain or remove this comment if it’s no longer relevant.
Would you like help creating an issue to track any remaining v1 compatibility work?
internal/storage/ledger/resource.go (1)
116-120: Potential performance overhead with frequent regex matching.
You're constructing a new regular expression pattern for each property in every filter iteration. Depending on usage frequency, it may be beneficial to precompile these patterns or use simpler string comparisons for known property name formats to improve performance.internal/storage/ledger/logs.go (1)
Line range hint
114-134: Enhance error handling in ReadLogWithIdempotencyKeyThe function should distinguish between "not found" and other database errors.
Apply this diff:
if err := store.db.NewSelect(). Model(ret). ModelTableExpr(store.GetPrefixedRelationName("logs")). Column("*"). Where("idempotency_key = ?", key). Where("ledger = ?", store.ledger.Name). Limit(1). Scan(ctx); err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, ledgercontroller.ErrNotFound + } return nil, postgres.ResolveError(err) }internal/storage/ledger/accounts.go (1)
Line range hint
93-126: Consider adding comments to explain the complex query logicThe function implementation is correct and handles all edge cases properly. However, the complex upsert query with multiple conditions would benefit from inline comments explaining the logic, especially around the first_usage comparison and metadata merging strategy.
internal/storage/ledger/balances.go (1)
55-103: Consider improving query documentationThe SQL query construction is well-structured using CTEs, but its complexity warrants additional inline documentation. While there's a good comment about locking order, similar explanations for other parts of the query would improve maintainability.
For example:
- Explain the purpose of the
selectMovesCTE- Document why
UnionAllis used instead ofUnion- Clarify the role of
DistinctOnin the queryinternal/controller/ledger/controller.go (1)
34-35: Consider documenting the GetAggregatedVolumesOptions typeThe method signature changes for volume-related queries introduce new option types. Consider adding documentation for
GetVolumesOptionsandGetAggregatedVolumesOptionsto help API consumers understand the available options.internal/api/v2/controllers_accounts_count_test.go (1)
50-54: Consider reducing duplication in test query initializationThere's repeated initialization of
ResourceQuery[any]with similar base fields (PIT and empty Expand slice) across multiple test cases.Consider extracting a helper function:
+func newTestResourceQuery(pit *time.Time, builder query.Builder) ledgercontroller.ResourceQuery[any] { + return ledgercontroller.ResourceQuery[any]{ + PIT: pit, + Builder: builder, + Expand: make([]string, 0), + } +} testCases := []testCase{ { name: "using metadata", body: `{"$match": { "metadata[roles]": "admin" }}`, expectBackendCall: true, - expectQuery: ledgercontroller.ResourceQuery[any]{ - PIT: &before, - Builder: query.Match("metadata[roles]", "admin"), - Expand: make([]string, 0), - }, + expectQuery: newTestResourceQuery(&before, query.Match("metadata[roles]", "admin")), },Also applies to: 60-63, 70-74, 80-84
internal/storage/ledger/resource_aggregated_balances.go (2)
133-150: Consider adding input validation for metadata keys.The metadata filtering logic should validate the metadata key length and format to prevent potential issues with very long or malformed keys.
func (h aggregatedBalancesResourceRepositoryHandler) resolveFilter(store *Store, query ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions], operator, property string, value any) (string, []any, error) { + const maxMetadataKeyLength = 256 switch { case property == "address": return filterAccountAddress(value.(string), "accounts_address"), nil, nil case metadataRegex.Match([]byte(property)) || property == "metadata": + if len(property) > maxMetadataKeyLength { + return "", nil, ledgercontroller.NewErrInvalidQuery("metadata key exceeds maximum length of %d characters", maxMetadataKeyLength) + } if property == "metadata" {
49-102: Consider adding index hints for query optimization.The complex queries with joins and window functions could benefit from index hints to ensure optimal query plan selection.
Consider adding appropriate indexes for:
(ledger, accounts_address, asset)for the moves table(ledger, accounts_address, date)for the accounts_metadata tableinternal/controller/ledger/store.go (1)
202-202: Address TODO comment about backportingThe TODO comment indicates that some functionality needs to be backported to go-libs.
Would you like me to help create a GitHub issue to track this backporting task?
internal/storage/ledger/legacy/transactions_test.go (1)
179-236: Consider adding edge cases to the test suite.While the current test cases cover the basic scenarios, consider adding tests for:
- Transactions with extremely large volumes
- Transactions with zero-value postings
- Transactions with multiple currency conversions
internal/api/v2/controllers_transactions_list_test.go (1)
153-155: Consider adding boundary test cases for pagination.While the current tests cover basic pagination scenarios, consider adding tests for:
- Maximum allowed page size edge case
- Cursor with complex sorting conditions
- Invalid cursor formats
Also applies to: 178-236
internal/storage/ledger/balances_test.go (1)
325-404: Consider adding error scenarios to metadata filtering tests.While the happy path is well tested, consider adding tests for:
- Invalid metadata keys
- Non-existent metadata values
- Concurrent metadata updates during aggregation
internal/api/v2/mocks_ledger_controller_test.go (1)
Line range hint
1-400: LGTM! Changes are consistent across all mock files.The identical changes across all three mock files (
v2,common, andbulkingpackages) demonstrate a consistent implementation of the new query type system. This consistency is good for maintainability.Consider documenting the new query type system and its proper usage patterns in the codebase to help future contributors understand the design decisions.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (4)
go.modis excluded by!**/*.modgo.sumis excluded by!**/*.sum,!**/*.suminternal/storage/bucket/migrations/25-accounts-volumes-index/notes.yamlis excluded by!**/*.yamltools/generator/go.sumis excluded by!**/*.sum,!**/*.sum
📒 Files selected for processing (82)
internal/README.md(2 hunks)internal/api/bulking/mocks_ledger_controller_test.go(9 hunks)internal/api/common/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/controllers_accounts_count.go(1 hunks)internal/api/v1/controllers_accounts_count_test.go(6 hunks)internal/api/v1/controllers_accounts_list.go(1 hunks)internal/api/v1/controllers_accounts_list_test.go(5 hunks)internal/api/v1/controllers_accounts_read.go(2 hunks)internal/api/v1/controllers_accounts_read_test.go(4 hunks)internal/api/v1/controllers_balances_aggregates.go(1 hunks)internal/api/v1/controllers_balances_aggregates_test.go(1 hunks)internal/api/v1/controllers_balances_list.go(1 hunks)internal/api/v1/controllers_logs_list.go(1 hunks)internal/api/v1/controllers_logs_list_test.go(3 hunks)internal/api/v1/controllers_transactions_count.go(1 hunks)internal/api/v1/controllers_transactions_count_test.go(3 hunks)internal/api/v1/controllers_transactions_list.go(1 hunks)internal/api/v1/controllers_transactions_list_test.go(4 hunks)internal/api/v1/controllers_transactions_read.go(2 hunks)internal/api/v1/controllers_transactions_read_test.go(2 hunks)internal/api/v1/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/utils.go(1 hunks)internal/api/v2/common.go(2 hunks)internal/api/v2/controllers_accounts_count.go(1 hunks)internal/api/v2/controllers_accounts_count_test.go(4 hunks)internal/api/v2/controllers_accounts_list.go(1 hunks)internal/api/v2/controllers_accounts_list_test.go(5 hunks)internal/api/v2/controllers_accounts_read.go(2 hunks)internal/api/v2/controllers_accounts_read_test.go(4 hunks)internal/api/v2/controllers_balances.go(1 hunks)internal/api/v2/controllers_balances_test.go(2 hunks)internal/api/v2/controllers_logs_list.go(1 hunks)internal/api/v2/controllers_logs_list_test.go(5 hunks)internal/api/v2/controllers_transactions_count.go(1 hunks)internal/api/v2/controllers_transactions_count_test.go(4 hunks)internal/api/v2/controllers_transactions_list.go(1 hunks)internal/api/v2/controllers_transactions_list_test.go(4 hunks)internal/api/v2/controllers_transactions_read.go(2 hunks)internal/api/v2/controllers_transactions_read_test.go(2 hunks)internal/api/v2/controllers_volumes.go(1 hunks)internal/api/v2/controllers_volumes_test.go(4 hunks)internal/api/v2/mocks_ledger_controller_test.go(9 hunks)internal/controller/ledger/controller.go(1 hunks)internal/controller/ledger/controller_default.go(4 hunks)internal/controller/ledger/controller_default_test.go(10 hunks)internal/controller/ledger/controller_generated_test.go(9 hunks)internal/controller/ledger/controller_with_traces.go(9 hunks)internal/controller/ledger/stats.go(1 hunks)internal/controller/ledger/stats_test.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/controller/ledger/store_generated_test.go(5 hunks)internal/storage/bucket/default_bucket.go(3 hunks)internal/storage/bucket/migrations/25-accounts-volumes-index/up.sql(1 hunks)internal/storage/ledger/accounts.go(3 hunks)internal/storage/ledger/accounts_test.go(11 hunks)internal/storage/ledger/balances.go(3 hunks)internal/storage/ledger/balances_test.go(1 hunks)internal/storage/ledger/debug.go(1 hunks)internal/storage/ledger/errors.go(0 hunks)internal/storage/ledger/legacy/accounts.go(6 hunks)internal/storage/ledger/legacy/accounts_test.go(19 hunks)internal/storage/ledger/legacy/adapters.go(2 hunks)internal/storage/ledger/legacy/balances.go(1 hunks)internal/storage/ledger/legacy/balances_test.go(8 hunks)internal/storage/ledger/legacy/logs.go(1 hunks)internal/storage/ledger/legacy/logs_test.go(2 hunks)internal/storage/ledger/legacy/queries.go(1 hunks)internal/storage/ledger/legacy/transactions.go(6 hunks)internal/storage/ledger/legacy/transactions_test.go(8 hunks)internal/storage/ledger/legacy/volumes.go(4 hunks)internal/storage/ledger/legacy/volumes_test.go(27 hunks)internal/storage/ledger/logs.go(3 hunks)internal/storage/ledger/logs_test.go(3 hunks)internal/storage/ledger/moves.go(1 hunks)internal/storage/ledger/moves_test.go(1 hunks)internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)
⛔ Files not processed due to max files limit (11)
- internal/storage/ledger/resource_transactions.go
- internal/storage/ledger/resource_volumes.go
- internal/storage/ledger/store.go
- internal/storage/ledger/transactions.go
- internal/storage/ledger/transactions_test.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/volumes.go
- internal/storage/ledger/volumes_test.go
- internal/volumes.go
- pkg/generate/generator.go
- test/e2e/api_transactions_list_test.go
💤 Files with no reviewable changes (1)
- internal/storage/ledger/errors.go
✅ Files skipped from review due to trivial changes (1)
- internal/storage/bucket/migrations/25-accounts-volumes-index/up.sql
🚧 Files skipped from review as they are similar to previous changes (48)
- internal/api/v1/controllers_transactions_read_test.go
- internal/api/v1/controllers_transactions_count.go
- internal/api/v1/controllers_accounts_list.go
- internal/api/v1/controllers_accounts_count.go
- internal/api/v1/controllers_balances_aggregates.go
- internal/api/v1/controllers_transactions_list.go
- internal/api/v1/controllers_accounts_read.go
- internal/api/v2/controllers_transactions_read_test.go
- internal/api/v1/controllers_transactions_read.go
- internal/storage/ledger/paginator.go
- internal/storage/ledger/legacy/balances_test.go
- internal/api/v2/controllers_accounts_count.go
- internal/api/v2/controllers_logs_list.go
- internal/controller/ledger/stats.go
- internal/README.md
- internal/storage/ledger/legacy/logs.go
- internal/api/v2/controllers_transactions_count.go
- internal/api/v2/controllers_accounts_list.go
- internal/api/v2/controllers_balances_test.go
- internal/api/v2/controllers_balances.go
- internal/api/v2/controllers_accounts_read.go
- internal/controller/ledger/stats_test.go
- internal/api/v1/controllers_balances_aggregates_test.go
- internal/api/v1/controllers_accounts_list_test.go
- internal/api/v2/controllers_transactions_list.go
- internal/api/v1/controllers_accounts_count_test.go
- internal/api/v1/controllers_balances_list.go
- internal/api/v2/controllers_volumes_test.go
- internal/storage/ledger/debug.go
- internal/storage/ledger/legacy/accounts_test.go
- internal/api/v1/controllers_accounts_read_test.go
- internal/api/v1/controllers_transactions_count_test.go
- internal/storage/ledger/legacy/logs_test.go
- internal/api/v2/controllers_logs_list_test.go
- internal/storage/ledger/moves.go
- internal/api/v2/controllers_transactions_read.go
- internal/api/v2/controllers_volumes.go
- internal/storage/ledger/accounts_test.go
- internal/storage/ledger/legacy/volumes.go
- internal/api/v2/controllers_accounts_read_test.go
- internal/storage/ledger/legacy/balances.go
- internal/storage/ledger/resource_logs.go
- internal/storage/ledger/paginator_column.go
- internal/storage/ledger/legacy/transactions.go
- internal/controller/ledger/controller_default_test.go
- internal/storage/ledger/resource_accounts.go
- internal/storage/ledger/legacy/accounts.go
- internal/storage/ledger/legacy/queries.go
🔇 Additional comments (79)
internal/storage/bucket/default_bucket.go (1)
287-304: Verify necessity of additional array indexes
Given that we already have GIN indexes on sources and destinations, verify if separate indexes on sources_arrays and destinations_arrays are necessary. These additional indexes will impact write performance and storage space.
✅ Verification successful
Let me gather more information to understand how these array columns are used in queries.
The array indexes are necessary and serve a distinct purpose
The search results reveal that these array indexes are actively used in query conditions with the @> containment operator, which is different from how the regular sources and destinations columns are queried:
- In
internal/storage/ledger/transactions.goandinternal/storage/ledger/legacy/utils.go, the code usessources_arrays @> '{...}'anddestinations_arrays @> '{...}'for specialized array containment queries - The arrays are populated by the
explode_addressfunction in migrations, suggesting they store decomposed address segments for hierarchical querying - The GIN indexes with
jsonb_path_opsare optimal for the@>operator being used
These indexes serve a different query pattern than the regular sources/destinations indexes and are necessary for efficient hierarchical address lookups.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if these columns are used in queries differently than sources/destinations
rg -A 5 "sources_arrays|destinations_arrays" --type sql
# Check if there are specific query patterns that would benefit from these indexes
ast-grep --pattern 'Where($$$sources_arrays$$$)'
ast-grep --pattern 'Where($$$destinations_arrays$$$)'
Length of output: 11031
Script:
#!/bin/bash
# Check for SQL queries in Go code that might use these columns
rg -t go "sources_arrays|destinations_arrays" -B 2 -A 2
# Look for query builder patterns or WHERE clauses in Go code
ast-grep --pattern 'Where($$$"sources_arrays"$$$)'
ast-grep --pattern 'Where($$$"destinations_arrays"$$$)'
# Check if there are specific functions/methods that work with these arrays
rg -t go "func.*sources_arrays|func.*destinations_arrays"
Length of output: 11036
internal/storage/ledger/legacy/adapters.go (5)
22-25: Implementation of Accounts retrieval looks good.
The method directly delegates to the newStore's “Accounts” function. This is straightforward and aligns with the new resource-oriented approach. No issues spotted.
26-29: Logs delegation is straightforward.
The method simply delegates to the newStore’s “Logs” and properly returns the paginated resource. Implementation is concise and consistent with the refactoring goal.
30-33: Transactions delegation logic looks sound.
Again, this method aligns with the pattern of returning a resource from the newStore. No logical issues identified.
38-41: Volumes delegation is consistent.
The newly introduced method “Volumes” properly returns the PaginatedResource, aligning with the overall refactoring pattern.
70-70: UpsertAccounts with variadic parameter is fine.
Upserting multiple accounts in a single call using a variadic parameter is a convenient design. The direct delegation to newStore looks correct.
internal/storage/ledger/resource.go (2)
17-31: Avoid panicking for unsupported operators.
This function still panics when encountering an unsupported operator. Returning an error would help avoid unexpected crashes and provide clearer feedback to callers.
283-283: Avoid panic in case of unrecognized paginated query type.
Relying on “panic(‘should not happen’)” can cause undesired application crashes if a new or unrecognized paginated query type is introduced in the future. Instead, return an error to allow graceful handling.
internal/controller/ledger/store_generated_test.go (12)
45-57: Mock method creation for Accounts looks good.
Implementation correctly returns the expected paginated resource type.
59-71: AggregatedBalances mock method is consistent.
Method signature aligns with the new resource approach.
233-245: Logs mock method implementation is straightforward.
No issues detected—returns the correct PaginatedResource.
292-304: Transactions mock method properly returns a PaginatedResource.
This is in line with the shift to generic paginated resources.
355-367: Volumes mock method is consistent with the new design.
Matches the pattern of returning a PaginatedResource with the correct generic parameters.
369-385: Introduction of MockResource is well-structured.
Implementation provides a neat way to mock counting and retrieving single items.
392-399: MockResource Count method generation.
The mock call pattern is well-formed, returning the correct types.
407-414: MockResource GetOne method generation provides clarity.
Implementation of the single-item retrieval mock remains consistent.
422-438: Definition of MockPaginatedResource is aligned with the package’s approach.
No issues found in the struct or its recorder.
445-452: MockPaginatedResource Count method is correct.
Signature and return types follow the new pattern.
460-467: MockPaginatedResource GetOne method is valid.
Method correctly simulates a single resource fetch scenario.
475-482: MockPaginatedResource Paginate method.
Implementation is properly structured for cursor-based pagination mocking.
internal/api/v1/controllers_logs_list.go (2)
38-41: Usage of getColumnPaginatedQuery simplifies pagination logic.
Error handling is handled right after the call. Good practice to reduce complexity in the main function body.
44-45: Attaching the query builder for logs.
Directly assigning the result of buildGetLogsQuery to paginatedQuery.Options.Builder is straightforward and keeps concerns separated (filter building vs. pagination).
internal/storage/ledger/paginator_offset.go (2)
44-52: Guard against potential overflow or underflow in offset calculation.
The offset is derived from user-supplied values, and subtracting PageSize could result in underflow. Though you check for negative offsets, consider verifying extremely large offsets to avoid integer overflow.
55-60: Check for offset overflow in next page calculation.
When offset is incremented by PageSize, extremely large values can overflow. A boundary check would make the pagination safer.
internal/api/v1/utils.go (1)
34-34: Verify MaxPageSize constant value
The MaxPageSize constant is used for pagination limits. Ensure it's set to a reasonable value to prevent performance issues with large result sets.
✅ Verification successful
MaxPageSize value is appropriate for pagination
Based on the search results, the MaxPageSize in v1 API is set to 1000 records, which is a reasonable limit that balances between performance and usability. This value helps prevent server overload while allowing sufficient data retrieval for most use cases. The v2 API inherits its MaxPageSize from a bunpaginate package, suggesting a standardized approach to pagination across the codebase.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Find MaxPageSize constant declaration
rg -A 1 "MaxPageSize\s*=\s*"
Length of output: 274
Script:
#!/bin/bash
# Let's check the bunpaginate package's MaxPageSize value
rg -A 1 "MaxPageSize\s*=\s*" pkg/bunpaginate/
Length of output: 104
Script:
#!/bin/bash
# Let's try to find the bunpaginate package location first
fd bunpaginate
Length of output: 16
internal/api/v2/common.go (1)
17-29: LGTM: Robust date parsing implementation
The date parsing implementation is well-structured with proper error handling and null safety.
internal/storage/ledger/logs.go (1)
Line range hint 47-106: Well-structured error handling for idempotency key conflicts
The implementation properly handles idempotency key conflicts and uses advisory locks when the feature is enabled.
internal/storage/ledger/accounts.go (2)
Line range hint 21-71: LGTM! Improved timestamp tracking
The function is well-structured with proper error handling and atomic updates. The addition of updated_at in the conflict resolution ensures accurate tracking of metadata modifications.
74-90: LGTM! Clean and secure implementation
The function implements a secure way to delete metadata using parameterized queries and proper error handling.
internal/api/v1/controllers_logs_list_test.go (1)
38-78: LGTM! Comprehensive test coverage
The test cases have been properly updated to use the new ColumnPaginatedQuery type while maintaining thorough coverage of all scenarios including nominal cases, time-based filtering, and cursor handling.
internal/storage/ledger/balances.go (2)
88-95: LGTM! Good locking strategy
The For("update") with ordered locking (Order("accounts_address", "asset")) is a solid approach to prevent deadlocks. The comment explaining this is particularly helpful.
Line range hint 111-131: Consider adding error handling for big.Int operations
While the balance calculation using big.Int is correct, it's good practice to check for potential overflow conditions when performing arithmetic operations, especially subtraction.
internal/controller/ledger/controller.go (1)
27-33: LGTM: Consistent use of generic query types across methods
The refactoring to use generic query types (ResourceQuery[any], OffsetPaginatedQuery[any], ColumnPaginatedQuery[any]) improves code flexibility and reduces duplication. The changes are consistent across all query methods.
Let's verify that all callers have been updated to use the new generic types:
✅ Verification successful
Let's gather more information to understand if these occurrences are actual issues or just legacy code references.
Let's check if there are any implementations of the controller interface to see if they've been updated to use the new generic types.
Let's check the actual implementation files directly to see how they're handling the query types.
Let's check one more thing to verify the implementation in the legacy package and its relationship with the new generic types.
Based on the gathered information, I can now provide the final response:
LGTM: The generic query types refactoring is properly implemented
The changes to use generic query types (ResourceQuery[any], OffsetPaginatedQuery[any], ColumnPaginatedQuery[any]) are correctly implemented across the codebase:
- The controller interface defines the generic types
- The
ControllerWithTracesimplementation uses the new generic types - The legacy package properly converts between old and new types using type assertions and generic type parameters
- The adapters in the legacy package correctly map to the new generic interfaces
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for any remaining usages of the old specific query types
rg -l "GetAccountQuery|ListAccountsQuery|GetTransactionQuery|ListTransactionsQuery|GetLogsQuery"
Length of output: 447
Script:
#!/bin/bash
# Check the content of files containing old query types to understand the context
echo "=== Checking legacy/queries.go ==="
rg -A 5 "GetAccountQuery|ListAccountsQuery|GetTransactionQuery|ListTransactionsQuery|GetLogsQuery" internal/storage/ledger/legacy/queries.go
echo -e "\n=== Checking controllers_logs_list.go ==="
rg -A 5 "GetAccountQuery|ListAccountsQuery|GetTransactionQuery|ListTransactionsQuery|GetLogsQuery" internal/api/v1/controllers_logs_list.go
echo -e "\n=== Checking README.md ==="
rg -A 5 "GetAccountQuery|ListAccountsQuery|GetTransactionQuery|ListTransactionsQuery|GetLogsQuery" internal/controller/ledger/README.md
Length of output: 24050
Script:
#!/bin/bash
# Search for implementations of the controller interface methods
echo "=== Checking DefaultController implementation ==="
ast-grep --pattern 'func (ctrl *DefaultController) GetAccount(ctx context.Context, query $_) (*ledger.Account, error)'
echo -e "\n=== Checking ControllerWithTraces implementation ==="
ast-grep --pattern 'func (ctrl *ControllerWithTraces) GetAccount(ctx context.Context, query $_) (*ledger.Account, error)'
Length of output: 474
Script:
#!/bin/bash
# Search for implementation files and check their content
echo "=== Checking DefaultController implementation file ==="
fd "default_controller.go" | xargs cat
echo -e "\n=== Checking controller_with_traces.go ==="
fd "controller_with_traces.go" | xargs cat
Length of output: 13137
Script:
#!/bin/bash
# Check how the legacy types are being used with the new generic types
echo "=== Checking legacy package usage ==="
rg -A 5 "type ResourceQuery|type OffsetPaginatedQuery|type ColumnPaginatedQuery" internal/controller/ledger/
echo -e "\n=== Checking where legacy types are converted to new generic types ==="
rg -A 5 "ResourceQuery\[|OffsetPaginatedQuery\[|ColumnPaginatedQuery\[" internal/storage/ledger/legacy/
Length of output: 8166
internal/api/v2/controllers_accounts_count_test.go (2)
29-29: LGTM: Test structure properly updated for new query type
The test cases have been correctly updated to use ResourceQuery[any], maintaining consistency with the interface changes.
Also applies to: 40-43
98-101: LGTM: Error cases properly handle the new query type
The error test cases correctly maintain the same query structure while testing different error scenarios.
Also applies to: 109-112, 120-123
internal/storage/ledger/moves_test.go (1)
173-186: LGTM: Proper validation of aggregated volumes in concurrent scenario
The test correctly validates the aggregated volumes after concurrent transactions, ensuring that the total input and output volumes match (1000 each) across all concurrent operations.
Let's verify the test's determinism:
✅ Verification successful
Test is deterministic and properly structured
The test uses relative time offsets (t0 through t4) with fixed intervals (1 hour) between timestamps, making it deterministic. While it starts with time.Now(), the subsequent timestamps are derived using fixed offsets, and the test validates aggregated volumes using UseInsertionDate: true which ensures consistent behavior regardless of the initial timestamp.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for any time.Now() usage in test files that might cause flakiness
rg -l 'time\.Now\(\)' --type go --glob '*_test.go'
Length of output: 1965
Script:
#!/bin/bash
# Check the specific usage of time.Now() in moves_test.go to understand if it affects test determinism
rg 'time\.Now\(\)' -C 5 internal/storage/ledger/moves_test.go
# Also check if the test uses fixed timestamps for the specific test case
rg 'UseInsertionDate' -C 5 internal/storage/ledger/moves_test.go
Length of output: 855
Script:
#!/bin/bash
# Let's check the full test context to understand how the timestamps are used
rg -B 10 -A 20 't0 := now' internal/storage/ledger/moves_test.go
Length of output: 976
internal/api/v2/controllers_transactions_count_test.go (2)
32-32: LGTM: Type change aligns with the refactoring.
The change from PaginatedQueryOptions[PITFilterWithVolumes] to ResourceQuery[any] is consistent with the broader refactoring to use more generic query types.
43-46: LGTM: Test cases properly updated.
All test cases have been correctly updated to use the new ResourceQuery[any] type with appropriate initialization of fields.
Also applies to: 52-56, 62-66, 72-76, 82-86, 92-96, 102-106, 112-116, 121-124, 136-139, 147-150
internal/storage/ledger/logs_test.go (2)
124-127: LGTM: Proper usage of the new pagination method.
The transition to Logs().Paginate with ColumnPaginatedQuery[any] is implemented correctly, including proper order specification.
192-208: LGTM: Query builder implementation is correct.
The test case properly demonstrates the use of query builder with complex conditions (date range filtering) using the new pagination structure.
internal/storage/ledger/resource_aggregated_balances.go (2)
51-73: LGTM: Proper feature flag validation and SQL safety.
The implementation correctly:
- Validates required features before execution
- Uses parameterized queries to prevent SQL injection
- Handles both insertion date and effective date scenarios
156-170: Consider adding error handling for aggregation overflow.
The aggregation of volumes using sum could potentially overflow for large datasets. Consider adding checks or using appropriate numeric types to handle large sums.
internal/controller/ledger/store.go (2)
60-64: LGTM! Well-structured interface design
The new Store interface methods are well-organized by resource type and leverage generic types effectively for improved type safety and reusability.
149-181: LGTM! Well-implemented ResourceQuery type
The ResourceQuery implementation is clean, with proper JSON unmarshaling and clear time handling methods.
internal/api/v1/controllers_transactions_list_test.go (1)
30-30: LGTM! Comprehensive test coverage
The test cases have been properly updated to use the new ColumnPaginatedQuery type and provide good coverage of various query scenarios.
Also applies to: 39-46, 53-61, 68-76, 83-91, 98-106, 113-121, 128-136, 143-151, 156-161, 185-192
internal/api/v2/controllers_accounts_list_test.go (1)
32-32: LGTM! Well-structured test cases
The test cases have been properly updated to use the new OffsetPaginatedQuery type and provide good coverage of both success and error scenarios.
Also applies to: 43-49, 56-63, 69-76, 82-89, 114-120, 126-133, 139-146, 160-166, 174-180, 188-194
internal/storage/ledger/legacy/transactions_test.go (3)
Line range hint 50-72: LGTM! Test coverage for transaction volumes looks good.
The test case properly validates both the transaction data and post-commit volumes using the new ledgerstore.NewGetTransactionQuery structure.
108-110: LGTM! Transaction count validation is properly implemented.
The test correctly validates the count of transactions using the new query structure with appropriate pagination options.
162-165: LGTM! Transaction retrieval test is well structured.
The test properly validates transaction retrieval with volumes and effective volumes expansion using the new query structure.
internal/api/v2/controllers_transactions_list_test.go (2)
Line range hint 33-50: LGTM! Test structure properly updated for new query type.
The test case structure correctly implements the new ColumnPaginatedQuery[any] type with appropriate default values.
55-64: LGTM! Comprehensive test coverage for filtering scenarios.
The test cases thoroughly cover:
- Metadata filtering
- Time-based filtering (startTime, endTime)
- Account-based filtering (source, destination)
Also applies to: 69-92, 97-148
internal/storage/ledger/balances_test.go (3)
242-261: LGTM! Comprehensive test for volume aggregation.
The test properly validates the aggregated volumes structure with both input and output values across multiple currencies.
265-280: LGTM! Address filtering test is well implemented.
The test correctly validates filtering by address prefix while maintaining proper volume calculations.
284-321: LGTM! Point-in-time queries are thoroughly tested.
The tests cover both effective date and insertion date scenarios with appropriate volume calculations.
internal/controller/ledger/controller_with_traces.go (3)
Line range hint 198-208: LGTM! Consistent refactoring to generic query types.
The refactoring of query methods to use generic types (ResourceQuery[any], ColumnPaginatedQuery[any], OffsetPaginatedQuery[any]) improves code flexibility while maintaining type safety. The implementation details remain unchanged, showing a clean transition to the new type system.
Also applies to: 210-220, 222-232, 234-244, 246-256, 258-268
Line range hint 282-292: LGTM! Consistent pagination type usage.
The refactoring maintains a clear distinction between different pagination types:
ColumnPaginatedQuery[any]for transactions and logsOffsetPaginatedQuery[any]for accountsOffsetPaginatedQuery[GetVolumesOptions]for volumes with balances
This consistency helps maintain clear pagination behavior across different resource types.
Also applies to: 296-308, 311-324, 330-340
Line range hint 270-280: Verify type compatibility for aggregated balances.
The change from specific query type to ResourceQuery[GetAggregatedVolumesOptions] maintains type safety while allowing for more flexible query handling. However, we should verify that all existing query parameters are compatible with the new options type.
✅ Verification successful
Let me gather more information about the type definitions and usage to verify type compatibility.
Let me check one more aspect to verify the complete type compatibility by looking at the previous implementation and usage.
Based on the gathered information, I can now provide the final response:
Type compatibility for aggregated balances is verified and consistent.
The ResourceQuery[GetAggregatedVolumesOptions] type is consistently used across the codebase with proper type safety. The GetAggregatedVolumesOptions struct is well-defined with a single boolean field UseInsertionDate, and all usages in tests and implementations maintain this contract. The type hasn't changed from the previous implementation, and all controller implementations (DefaultController, ControllerWithTraces, and mocks) use the same type signature.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check usage of GetAggregatedBalances to ensure parameter compatibility
# Search for GetAggregatedBalances calls and their parameter construction
rg -A 5 "GetAggregatedBalances.*ResourceQuery"
Length of output: 5494
Script:
#!/bin/bash
# Check type definitions and usage patterns
# Find the definition of GetAggregatedVolumesOptions
ast-grep --pattern 'type GetAggregatedVolumesOptions struct { $$$ }'
# Find the definition of ResourceQuery
ast-grep --pattern 'type ResourceQuery[$_] struct { $$$ }'
# Look for actual construction/usage of these types
rg -A 3 "GetAggregatedVolumesOptions\{"
Length of output: 3150
Script:
#!/bin/bash
# Look for previous implementation and usage patterns
# Find git history of the file to see previous type
git show HEAD~1:internal/controller/ledger/controller_with_traces.go | grep -A 5 "GetAggregatedBalances"
# Check for any other implementations of this interface
ast-grep --pattern 'func $_($_, $q $_) (ledger.BalancesByAssets, error) {
$$$
}'
Length of output: 924
internal/controller/ledger/controller_generated_test.go (1)
Line range hint 72-324: LGTM! Auto-generated mock matches interface changes.
The mock implementations correctly reflect the interface changes, with proper type parameters and return values. The file was auto-generated using mockgen, maintaining consistency with the controller interface.
internal/api/v1/mocks_ledger_controller_test.go (1)
Line range hint 73-312: LGTM! Consistent mock implementation across packages.
The mock implementation in the v1 package correctly mirrors the interface changes, maintaining consistency with the controller interface and other mock implementations. The use of generic query types is properly reflected throughout the mock methods.
internal/api/v2/mocks_ledger_controller_test.go (4)
Line range hint 178-193: Verify specialized query options handling.
The GetAggregatedBalances method uses a specialized GetAggregatedVolumesOptions type, indicating special handling requirements. This needs verification to ensure proper integration.
✅ Verification successful
Specialized query options are properly integrated across the codebase
The GetAggregatedVolumesOptions type is consistently used throughout the codebase with proper integration:
- Defined in
internal/controller/ledger/store.gowith clear structure - Properly implemented in store layer (
internal/storage/ledger/) - Correctly used in both v1 and v2 API controllers
- Well-covered by tests across multiple packages
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find usage of GetAggregatedVolumesOptions
# to verify proper integration
# Search for the specialized options type
rg -A 3 "GetAggregatedVolumesOptions" --type go
Length of output: 18118
Line range hint 297-312: Verify column-based pagination implementation.
The introduction of ColumnPaginatedQuery[any] for ListLogs and ListTransactions represents a different pagination approach. This needs verification to ensure proper handling of large datasets.
✅ Verification successful
Based on the search results, I can now generate the final response since I have enough information about the column-based pagination implementation:
Column-based pagination implementation is properly handled and consistent.
The implementation shows a robust and consistent approach to column-based pagination:
- Uses a type-safe
ColumnPaginatedQuery[any]structure across all pagination operations - Implements default ordering (DESC) and column ("id") settings
- Includes proper cursor handling with Previous/Next navigation
- Has comprehensive test coverage validating pagination behavior
- Properly handles large datasets through:
- Configurable page sizes with defaults and maximums
- Cursor-based navigation to avoid offset issues
- Consistent column-based ordering across all paginated resources
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find column-based pagination implementations
# to verify consistency
# Search for column pagination usage
rg -A 5 "ColumnPaginatedQuery|Column.*Paginate" --type go
Length of output: 62926
Line range hint 282-297: Verify offset-based pagination implementation.
The ListAccounts method now uses OffsetPaginatedQuery[any], which suggests a change in pagination strategy. This change needs careful verification to ensure pagination behavior remains consistent.
✅ Verification successful
Based on the search results, I can now generate the final response as I have enough context about the pagination implementation in the codebase:
The OffsetPaginatedQuery usage in ListAccounts is consistent with the codebase's pagination strategy.
The codebase shows a clear pattern where:
- Accounts use
OffsetPaginatedQueryconsistently (as seen ininternal/storage/ledger/store.go) - Transactions and Logs use
ColumnPaginatedQuery - The implementation has proper support for both pagination types through dedicated paginators (
offsetPaginatorandcolumnPaginator)
The change aligns with the existing architecture where accounts specifically use offset-based pagination, while other resources like transactions and logs use column-based pagination.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find existing pagination implementations to verify
# consistency with the new approach
# Search for pagination-related code
rg -A 5 "OffsetPaginatedQuery|Paginate" --type go
Length of output: 68027
Line range hint 73-88: Verify the impact of ResourceQuery[any] migration.
The transition to ResourceQuery[any] for multiple methods (CountAccounts, GetAccount, GetTransaction, etc.) represents a significant architectural change. While this improves flexibility, we should verify that all callers can adapt to the new generic type.
Also applies to: 163-178, 223-238
✅ Verification successful
Let me analyze the callers to verify the impact of the ResourceQuery[any] migration.
Based on the comprehensive search results, I can now provide a final response about the ResourceQuery[any] migration impact:
Migration to ResourceQuery[any] is safe and properly implemented across the codebase.
The verification shows:
- All callers consistently use
ResourceQuery[any]type across v1 and v2 APIs - The implementation is uniform in both controller layers (DefaultController and ControllerWithTraces)
- All test cases properly handle the generic type
- No incompatible usages were found in the codebase
The migration represents a type-safe enhancement that maintains backward compatibility while providing better type safety through generics.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Find all callers of these methods to verify compatibility
# with the new ResourceQuery[any] type
# Search for method calls
rg -A 3 "(\.|->)(CountAccounts|GetAccount|GetTransaction)\(" --type go
Length of output: 6204
Script:
#!/bin/bash
# Let's check the imports to verify the type compatibility
rg -B 2 -A 1 "ResourceQuery\[any\]" --type go
Length of output: 65584
Script:
#!/bin/bash
# Let's also check the definition of ResourceQuery to understand the type constraints
ast-grep --pattern 'type ResourceQuery[$_] struct {
$$$
}'
Length of output: 62
Script:
#!/bin/bash
# Check if there are any other query types being used with these methods
rg "(\.|->)(CountAccounts|GetAccount|GetTransaction)\([^)]*\)" --type go
Length of output: 2031
internal/controller/ledger/controller_default.go (11)
110-112: LGTM!
The method is a clean pass-through implementation that follows the single responsibility principle.
118-119: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
122-123: Update calls to CountTransactions in legacy store implementation
The CountTransactions method in internal/storage/ledger/legacy/transactions.go still uses the old ListTransactionsQuery type, while the controller interface has been updated to use ResourceQuery[any]. This needs to be updated for compatibility.
126-127: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
130-131: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
134-135: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
138-140: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
142-147: LGTM!
The method has been updated to use a more generic query type while maintaining proper error handling and clean data transformation.
150-151: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
154-155: LGTM!
The method signature has been updated to use a more generic query type while maintaining clean pass-through behavior.
292-296: LGTM!
The pagination query has been updated to use a more generic type while maintaining proper pagination handling.
internal/storage/ledger/legacy/volumes_test.go (1)
Line range hint 104-665: LGTM!
The test cases have been properly updated to use the new query types from the ledgerstore package, maintaining comprehensive test coverage for the refactored functionality.
internal/api/common/mocks_ledger_controller_test.go (3)
73-73: LGTM: Consistent use of generic ResourceQuery[any]
The refactoring to use ResourceQuery[any] for simple query operations (CountAccounts, CountTransactions, GetAccount, GetTransaction) improves code consistency and reduces duplication.
Also applies to: 88-88, 163-163, 223-223
178-178: LGTM: Specialized ResourceQuery for aggregated volumes
The use of ResourceQuery[ledger0.GetAggregatedVolumesOptions] for GetAggregatedBalances is appropriate as it maintains type safety while allowing specialized options.
238-238: LGTM: Consistent pagination handling
The refactoring introduces clear separation between different pagination types:
OffsetPaginatedQueryfor offset-based pagination (GetVolumesWithBalances, ListAccounts)ColumnPaginatedQueryfor column-based pagination (ListLogs, ListTransactions)
Let's verify the consistency of pagination types across the codebase:
Also applies to: 282-282, 297-297, 312-312
✅ Verification successful
Let me try a different approach to verify the pagination patterns in the codebase.
Consistent pagination types verified across the codebase
The codebase shows a clear and consistent pattern in pagination handling:
OffsetPaginatedQueryis used for:- GetVolumesWithBalances
- ListAccounts
ColumnPaginatedQueryis used for:- ListLogs
- ListTransactions
This pattern is consistently implemented across all layers (API, controller, storage) and all versions (v1, v2, bulking).
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify consistent usage of pagination types
# Test: Search for pagination query usage patterns
echo "Checking OffsetPaginatedQuery usage..."
ast-grep --pattern 'OffsetPaginatedQuery[$_]'
echo "Checking ColumnPaginatedQuery usage..."
ast-grep --pattern 'ColumnPaginatedQuery[$_]'
Length of output: 269
Script:
#!/bin/bash
# Search for pagination-related function signatures and type definitions
echo "Searching for pagination-related patterns..."
rg -A 2 "type.*Paginated.*Query|func.*Paginated.*Query" --type go
echo -e "\nSearching for specific pagination method calls..."
rg -A 2 "(GetVolumesWithBalances|ListAccounts|ListLogs|ListTransactions).*Paginated.*Query" --type go
Length of output: 55511
internal/api/bulking/mocks_ledger_controller_test.go (1)
73-73: LGTM: Consistent mock implementations
The changes in both mock implementations are identical and generated from the same source, ensuring consistent behavior across packages. The modifications maintain type safety while providing flexibility through generic query types.
Let's verify the consistency between the two mock implementations:
Also applies to: 88-88, 163-163, 178-178, 223-223, 238-238, 282-282, 297-297, 312-312
✅ Verification successful
LGTM: Verified identical mock implementations
After examining both files, I can confirm that the mock implementations in both packages are identical, generated from the same source file (../../controller/ledger/controller.go), and maintain consistent method signatures and behavior. The only difference is the package declaration (package common vs package bulking).
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify consistency between mock implementations
# Test: Compare the method signatures in both files
echo "Comparing mock implementations..."
diff <(grep -A 1 "^func.*Controller)" internal/api/common/mocks_ledger_controller_test.go) <(grep -A 1 "^func.*Controller)" internal/api/bulking/mocks_ledger_controller_test.go)
Length of output: 366
Script:
#!/bin/bash
# Let's first verify if these files exist and check their content
echo "Checking files existence:"
ls -l internal/api/common/mocks_ledger_controller_test.go internal/api/bulking/mocks_ledger_controller_test.go
echo -e "\nChecking common implementation:"
cat internal/api/common/mocks_ledger_controller_test.go
echo -e "\nChecking bulking implementation:"
cat internal/api/bulking/mocks_ledger_controller_test.go
Length of output: 32989
| func (d *DefaultStoreAdapter) AggregatedBalances() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] { | ||
| return d.newStore.AggregatedVolumes() | ||
| } | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Method name vs. resource name might be confusing.
The method is called “AggregatedBalances” but it instantiates a resource type for “AggregatedVolumes.” Consider renaming this method to make it consistent and more intuitive for maintainers.
-func (d *DefaultStoreAdapter) AggregatedBalances() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] {
- return d.newStore.AggregatedVolumes()
+func (d *DefaultStoreAdapter) AggregatedVolumes() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] {
+ return d.newStore.AggregatedVolumes()
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| func (d *DefaultStoreAdapter) AggregatedBalances() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] { | |
| return d.newStore.AggregatedVolumes() | |
| } | |
| func (d *DefaultStoreAdapter) AggregatedVolumes() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] { | |
| return d.newStore.AggregatedVolumes() | |
| } |
| func getColumnPaginatedQuery[v any](r *http.Request, column string, order bunpaginate.Order, modifiers ...func(*v) error) (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| return bunpaginate.Extract[ledgercontroller.ColumnPaginatedQuery[v]](r, func() (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| rq, err := getResourceQuery[v](r, modifiers...) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| pageSize, err := bunpaginate.GetPageSize(r, bunpaginate.WithMaxPageSize(MaxPageSize), bunpaginate.WithDefaultPageSize(DefaultPageSize)) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return &ledgercontroller.ColumnPaginatedQuery[v]{ | ||
| PageSize: pageSize, | ||
| Column: column, | ||
| Order: pointer.For(order), | ||
| Options: *rq, | ||
| }, nil | ||
| }) | ||
| } |
gfyrag
force-pushed
the
refactor/store-read-part
branch
from
48ed807 to
2f85f35
Compare
There was a problem hiding this comment.
Actionable comments posted: 11
🔭 Outside diff range comments (1)
internal/api/bulking/mocks_ledger_controller_test.go (1)
Based on the search results, I can now generate the final response:
Document pagination types and use specific type parameters
The codebase shows clear patterns for pagination type usage:
OffsetPaginatedQuery: Used for collections with stable ordering (accounts, volumes)ColumnPaginatedQuery: Used for time-series data with specific column-based ordering (transactions, logs)Recommendations:
- Replace
anywith specific type parameters:
ListAccounts: UsePITFilterWithVolumesinstead ofanyListLogs: Use specific options type instead ofanyListTransactions: UsePITFilterWithVolumesinstead ofany🔗 Analysis chain
Line range hint
282-295: Document the rationale for different pagination typesThe codebase uses both
OffsetPaginatedQueryandColumnPaginatedQuery. Consider:
- Adding documentation explaining when to use each pagination type
- Using specific type parameters instead of
anyto maintain type safetyLet's check for existing documentation or usage patterns:
Also applies to: 297-310, 312-325
Let me gather more information about the pagination types by looking at their implementations and usage patterns:
Let me try one more search with different patterns to find the implementation details:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Search for documentation about pagination types rg -A 5 "OffsetPaginatedQuery|ColumnPaginatedQuery" --type md # Search for other usages of these pagination types ast-grep --pattern '$name[OffsetPaginatedQuery[$type]]' ast-grep --pattern '$name[ColumnPaginatedQuery[$type]]'Length of output: 2624
Script:
#!/bin/bash # Search for the implementation of pagination types rg -A 10 "type (Offset|Column)PaginatedQuery" # Search for documentation or comments explaining the pagination types rg -B 2 -A 5 "type (Offset|Column)PaginatedQuery" # Look for specific type parameters being used ast-grep --pattern 'OffsetPaginatedQuery[$type]' ast-grep --pattern 'ColumnPaginatedQuery[$type]'Length of output: 201
Script:
#!/bin/bash # Search for files containing both pagination types rg -l "OffsetPaginatedQuery.*ColumnPaginatedQuery|ColumnPaginatedQuery.*OffsetPaginatedQuery" # Look for the package containing pagination types fd -e go | xargs rg "package.*bunpaginate" # Search for method implementations using these types rg -A 5 "func.*Paginated"Length of output: 49136
♻️ Duplicate comments (4)
internal/storage/ledger/resource.go (2)
17-31:⚠️ Potential issueHandle unsupported operators without panicking
This code still panics instead of returning an error for unrecognized operators, which may lead to crashes if new operators are introduced or inputs are malformed.Apply the same fix recommended previously:
-func convertOperatorToSQL(operator string) string { +func convertOperatorToSQL(operator string) (string, error) { switch operator { case "$match": - return "=" + return "=", nil case "$lt": - return "<" + return "<", nil case "$gt": - return ">" + return ">", nil case "$lte": - return "<=" + return "<=", nil case "$gte": - return ">=" + return ">=", nil } - panic("unreachable") + return "", fmt.Errorf("unsupported operator: %s", operator) }Then handle the returned error in callers accordingly.
283-283:⚠️ Potential issueAvoid panic for unexpected pagination type
As previously suggested, returning an error is more robust than panicking in production services.Example approach:
- panic("should not happen") + return nil, fmt.Errorf("unsupported pagination query type: %T", v)internal/storage/ledger/resource_accounts.go (2)
114-114:⚠️ Potential issueEnsure regex match before accessing submatches
Accessing balanceRegex.FindAllStringSubmatch(property, 2)[0][1] without checking its length can cause an out-of-range panic.Reapply the prior fix to validate the match first:
- selectBalance = selectBalance.Where("asset = ?", balanceRegex.FindAllStringSubmatch(property, 2)[0][1]) + matches := balanceRegex.FindAllStringSubmatch(property, 2) + if len(matches) > 0 && len(matches[0]) > 1 { + selectBalance = selectBalance.Where("asset = ?", matches[0][1]) + } else { + return "", nil, fmt.Errorf("invalid balance property format for: %s", property) + }
180-180:⚠️ Potential issuePrevent SQL injection by avoiding string concatenation
Concatenating untrusted data directly into queries can lead to injection vulnerabilities. Use parameterized identifiers instead.Example approach:
- .ColumnExpr("public.aggregate_objects(json_build_object(asset, json_build_object('input', (volumes).inputs, 'output', (volumes).outputs))::jsonb) as " + strcase.SnakeCase(property)) + columnAlias := strcase.SnakeCase(property) + .ColumnExpr("public.aggregate_objects(json_build_object(asset, json_build_object('input', (volumes).inputs, 'output', (volumes).outputs))::jsonb) as ?", bun.Ident(columnAlias))
🧹 Nitpick comments (34)
internal/storage/ledger/legacy/adapters.go (2)
21-24: TODO comment needs more detailsThe TODO comment "handle compat with v1" is vague. Consider adding more context about what specific v1 compatibility issues need to be addressed.
Would you like me to help create a more detailed TODO or open a GitHub issue to track this compatibility work?
The
Accounts()method implementation looks good.
Line range hint
15-19: Consider removing or documenting the unused flagThe
isFullUpToDatefield appears to be unused after the removal of conditional logic that previously determined whether to uselegacyStoreornewStore. If it's being kept for future use or backward compatibility, please add documentation explaining its purpose.internal/api/v2/mocks_ledger_controller_test.go (2)
Line range hint
238-251: LGTM! Well-structured pagination strategy.The pagination implementation shows a thoughtful approach:
OffsetPaginatedQueryfor volumes and accountsColumnPaginatedQueryfor logs and transactions (optimized for large datasets)The choice of column-based pagination for logs and transactions is particularly good for performance, as it avoids the increasing offset penalty when dealing with large datasets.
Also applies to: 282-295, 297-310, 312-325
Line range hint
1-1: Reminder: This is a generated file.This file is generated by mockgen. Any changes should be made to the source interface definition rather than directly to this file.
internal/api/v1/controllers_transactions_list_test.go (3)
30-30: Consider using a concrete type instead ofanyWhile using
anyprovides flexibility, it might hide type information and make the code less type-safe. Consider using a concrete type that represents the expected query result type, such asColumnPaginatedQuery[ledger.Transaction].
53-61: Consider adding more metadata test scenariosThe current test case covers basic metadata querying. Consider adding test cases for:
- Multiple metadata fields
- Nested metadata structures
- Special characters in metadata keys/values
98-151: Enhance test coverage for edge casesWhile the basic query scenarios are well covered, consider adding test cases for:
- Case sensitivity handling
- Special characters in values
- Multiple values for the same parameter
- Invalid account/reference formats
internal/api/v2/controllers_accounts_list_test.go (1)
43-49: Consider reducing test case boilerplate.While the test cases are comprehensive, there's repetitive initialization of empty
Expandslices and similarOptionsstructures across multiple test cases.Consider creating helper functions to reduce boilerplate:
func newTestQuery(pageSize int, opts ledgercontroller.ResourceQuery[any]) ledgercontroller.OffsetPaginatedQuery[any] { if opts.Expand == nil { opts.Expand = make([]string, 0) } return ledgercontroller.OffsetPaginatedQuery[any]{ PageSize: pageSize, Options: opts, } }This would simplify test cases, for example:
-expectQuery: ledgercontroller.OffsetPaginatedQuery[any]{ - PageSize: DefaultPageSize, - Options: ledgercontroller.ResourceQuery[any]{ - PIT: &before, - Expand: make([]string, 0), - }, -}, +expectQuery: newTestQuery(DefaultPageSize, ledgercontroller.ResourceQuery[any]{ + PIT: &before, +}),Also applies to: 56-63, 69-76, 82-89, 114-120, 126-133, 139-146, 160-166, 174-180, 188-194
internal/api/v2/common.go (3)
56-62: Simplify expand parameter handlingWhile the current implementation works, it could be simplified and made more explicit without relying on dot imports.
Consider this alternative implementation:
func getExpand(r *http.Request) []string { - return Flatten( - Map(r.URL.Query()["expand"], func(from string) []string { - return strings.Split(from, ",") - }), - ) + var result []string + for _, expand := range r.URL.Query()["expand"] { + result = append(result, strings.Split(expand, ",")...) + } + return result }
64-102: Consider flattening error handlingThe nested error handling in both pagination functions could be simplified using early returns.
Here's a suggested refactoring for improved readability:
func getOffsetPaginatedQuery[v any](r *http.Request, modifiers ...func(*v) error) (*ledgercontroller.OffsetPaginatedQuery[v], error) { return bunpaginate.Extract[ledgercontroller.OffsetPaginatedQuery[v]](r, func() (*ledgercontroller.OffsetPaginatedQuery[v], error) { - rq, err := getResourceQuery[v](r, modifiers...) - if err != nil { - return nil, err - } - - pageSize, err := bunpaginate.GetPageSize(r, bunpaginate.WithMaxPageSize(MaxPageSize), bunpaginate.WithDefaultPageSize(DefaultPageSize)) - if err != nil { - return nil, err - } + rq, err := getResourceQuery[v](r, modifiers...) + if err != nil { + return nil, fmt.Errorf("getting resource query: %w", err) + } + + pageSize, err := bunpaginate.GetPageSize(r, + bunpaginate.WithMaxPageSize(MaxPageSize), + bunpaginate.WithDefaultPageSize(DefaultPageSize), + ) + if err != nil { + return nil, fmt.Errorf("getting page size: %w", err) + } return &ledgercontroller.OffsetPaginatedQuery[v]{ PageSize: pageSize, Options: *rq, }, nil }) }
104-131: Enhance error handling and operation orderWhile the implementation is solid, consider:
- Wrapping errors with context
- Moving the options initialization and modification before the expensive operations
Here's a suggested refactoring:
func getResourceQuery[v any](r *http.Request, modifiers ...func(*v) error) (*ledgercontroller.ResourceQuery[v], error) { + var options v + for _, modifier := range modifiers { + if err := modifier(&options); err != nil { + return nil, fmt.Errorf("applying modifier: %w", err) + } + } + pit, err := getPIT(r) if err != nil { - return nil, err + return nil, fmt.Errorf("getting PIT: %w", err) } oot, err := getOOT(r) if err != nil { - return nil, err + return nil, fmt.Errorf("getting OOT: %w", err) } builder, err := getQueryBuilder(r) if err != nil { - return nil, err + return nil, fmt.Errorf("building query: %w", err) } - var options v - for _, modifier := range modifiers { - if err := modifier(&options); err != nil { - return nil, err - } - } return &ledgercontroller.ResourceQuery[v]{ PIT: pit, OOT: oot, Builder: builder, Expand: getExpand(r), Opts: options, }, nil }internal/storage/ledger/legacy/transactions_test.go (2)
108-111: Consider improving query construction readability.While functionally correct, the nested query construction could be made more readable by breaking it down into multiple lines:
-count, err := store.CountTransactions(context.Background(), ledgerstore.NewListTransactionsQuery(ledgercontroller.NewPaginatedQueryOptions(ledgerstore.PITFilterWithVolumes{}))) +queryOptions := ledgercontroller.NewPaginatedQueryOptions(ledgerstore.PITFilterWithVolumes{}) +query := ledgerstore.NewListTransactionsQuery(queryOptions) +count, err := store.CountTransactions(context.Background(), query)
179-180: Consider consolidating query types into a single package.The current implementation mixes types from both
ledgercontrollerandledgerstorepackages when constructing queries:ledgercontroller.PaginatedQueryOptions[ledgerstore.PITFilterWithVolumes] ledgercontroller.NewPaginatedQueryOptions(ledgerstore.PITFilterWithVolumes{})This coupling between packages might make future refactoring more difficult. Consider:
- Moving all query-related types to a dedicated package
- OR fully migrating
PaginatedQueryOptionsto theledgerstorepackageAlso applies to: 186-186, 191-192, 197-198, 203-204, 209-210, 215-219, 224-225, 230-231, 236-241, 246-247, 252-253
internal/storage/ledger/accounts.go (2)
103-110: Standardize parameter binding approachThe use of
Value("ledger", "?", store.ledger.Name)is inconsistent with other queries in the file that use direct WHERE clause binding. Consider standardizing the approach.- Value("ledger", "?", store.ledger.Name). + Where("ledger = ?", store.ledger.Name).
Line range hint
93-126: Consider extracting common metadata merge logicBoth
UpsertAccountsandUpdateAccountsMetadatashare similar metadata merging logic. Consider extracting this into a reusable function to maintain consistency and reduce duplication.Consider creating a helper function:
func (store *Store) mergeMetadata(existing, new metadata.Metadata) metadata.Metadata { return existing.Merge(new) }internal/storage/ledger/legacy/queries.go (3)
28-33: Consider validating page size inputs.In "NewGetVolumesWithBalancesQuery", the function sets the page size and aggregates query options but lacks explicit page size validation. If pageSize is zero or excessively large, the query might become inefficient. Consider adding a validation or boundary check to avoid potential performance overhead.
36-43: Clarify column usage in ListTransactionsQuery.By default, the query is set to use the "id" column. If users of this struct misunderstand or provide an unused column input, it might lead to confusion. Consider documenting or enforcing the accepted column parameters to prevent unexpected behavior.
129-133: Inconsistent usage of insertion date.The GetAggregatedBalanceQuery uses the UseInsertionDate flag in combination with a PIT filter. The logic might become confusing when both PIT and insertion date are used. Consider clarifying or enforcing a hierarchical priority to avoid contradictory filters.
internal/storage/ledger/resource_volumes.go (1)
15-55: Test filter coverage.The filters method enumerates multiple validation rules, especially for metadata. Testing each branch thoroughly (e.g., when operator is $exists vs. $match) will reduce the risk of logic gaps.
internal/controller/ledger/controller_default.go (3)
110-112: Check DB readiness before queries.IsDatabaseUpToDate ensures database schema compatibility. If the result is “false,” consider guiding the caller to handle migration or upgrade steps, rather than continuing with queries that might fail unexpectedly.
142-147: Re-examine representation of aggregated balances.GetAggregatedBalances returns ledger.BalancesByAssets, but a user might want more descriptive structures or additional fields. Consider offering an option or a separate endpoint for richer aggregated data (e.g., effective volumes or PIT expansions).
190-192: Ensure consistent usage of PageSize during import checks.The importLogs function uses a page size of 1 to check ledger emptiness. This is fine but ensure that future changes (or dynamic page sizing) do not unintentionally alter the logic that checks the first page for existing logs.
internal/controller/ledger/store_generated_test.go (1)
233-246: Ensure consistent usage of Logs()
Logs() now returns a PaginatedResource, which significantly changes how logs are retrieved and paginated. Make sure that all modules utilizing log retrieval are updated to reflect this new structure.internal/api/v2/controllers_accounts_read.go (1)
30-34: Use of ResourceQuery in GetAccount
Replacing a specialized GetAccountQuery with ResourceQuery[any] is more generic. Verify that all required fields (e.g., address, expansions) are still properly handled, and that there’s no missing logic for advanced filtering not covered by ResourceQuery.internal/api/v1/controllers_logs_list.go (2)
38-41: Error handling improvements
The new getColumnPaginatedQuery approach shortens the custom handling for pagination errors. Ensure that error messages remain descriptive enough to guide users in debugging invalid pagination queries.
46-46: ListLogs usage
The direct call to l.ListLogs with the dereferenced paginatedQuery is consistent with the new interface changes. Confirm that older code references to partial or offset-based logs retrieval are fully removed.internal/api/v1/utils.go (1)
67-79: ResourceQuery usage
The getResourceQuery function allows the usage of Expand and custom modifiers. Confirm that these new expansions don’t unintentionally expose internal fields or create large result sets.internal/api/v1/controllers_logs_list_test.go (1)
38-79: Consider adding edge cases to the test suite.While the test cases cover basic scenarios, consider adding tests for:
- Invalid page size values
- Invalid column names
- Multiple concurrent cursor operations
internal/storage/ledger/balances.go (1)
Line range hint
16-62: Consider extracting SQL query components for better maintainability.The SQL query construction is complex but correct. Consider:
- Extracting the query components into separate methods
- Adding SQL comments to explain the query structure
- Using SQL query builder constants for repeated expressions
internal/controller/ledger/controller.go (1)
27-35: LGTM! Well-structured refactoring of query types.The transition to generic query types (
ResourceQuery[any],ColumnPaginatedQuery[any],OffsetPaginatedQuery[any]) improves the interface's flexibility while maintaining type safety. This change aligns well with the refactoring objectives.This refactoring:
- Reduces code duplication by consolidating query types
- Improves maintainability by centralizing query logic
- Enhances extensibility for future query requirements
internal/storage/ledger/moves_test.go (1)
173-186: Consider enhancing test coverage for edge cases.While the test effectively verifies the basic functionality of aggregated volumes, consider adding test cases for:
- Zero volumes
- Multiple assets
- Different time ranges using
UseInsertionDateaggregatedVolumes, err := store.AggregatedVolumes().GetOne(ctx, ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions]{ Opts: ledgercontroller.GetAggregatedVolumesOptions{ UseInsertionDate: true, + // Add test cases with different time ranges + From: pointer.For(time.Now().Add(-time.Hour)), + To: pointer.For(time.Now()), }, })internal/storage/ledger/logs_test.go (1)
124-127: Consider testing pagination edge cases.The test verifies basic pagination functionality but could benefit from additional edge cases:
- Empty result sets
- Maximum page size scenarios
- Invalid page sizes
internal/api/v2/controllers_transactions_list_test.go (1)
191-201: Consider adding test cases for invalid query optionsThe cursor encoding tests don't cover cases where ResourceQuery options are invalid.
Add test cases for invalid options:
{ + name: "using cursor with invalid options", + queryParams: url.Values{ + "cursor": []string{func() string { + return bunpaginate.EncodeCursor(ledgercontroller.ColumnPaginatedQuery[any]{ + Options: ledgercontroller.ResourceQuery[any]{ + PIT: &now, + OOT: &now, // Invalid: both PIT and OOT + }, + }) + }()}, + }, + expectStatusCode: http.StatusBadRequest, + expectedErrorCode: common.ErrValidation, + },internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint
73-312: Well-structured query type hierarchy improves code maintainabilityThe refactoring introduces a clear hierarchy of query types:
ResourceQuery[T]for basic resource queriesOffsetPaginatedQuery[T]for offset-based paginationColumnPaginatedQuery[T]for column-based paginationThis consistent approach across all query methods improves code maintainability and provides better type safety through generics.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
go.modis excluded by!**/*.modgo.sumis excluded by!**/*.sum,!**/*.sumtools/generator/go.sumis excluded by!**/*.sum,!**/*.sum
📒 Files selected for processing (82)
internal/README.md(2 hunks)internal/api/bulking/mocks_ledger_controller_test.go(9 hunks)internal/api/common/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/controllers_accounts_count.go(1 hunks)internal/api/v1/controllers_accounts_count_test.go(6 hunks)internal/api/v1/controllers_accounts_list.go(1 hunks)internal/api/v1/controllers_accounts_list_test.go(5 hunks)internal/api/v1/controllers_accounts_read.go(2 hunks)internal/api/v1/controllers_accounts_read_test.go(4 hunks)internal/api/v1/controllers_balances_aggregates.go(1 hunks)internal/api/v1/controllers_balances_aggregates_test.go(1 hunks)internal/api/v1/controllers_balances_list.go(1 hunks)internal/api/v1/controllers_logs_list.go(1 hunks)internal/api/v1/controllers_logs_list_test.go(3 hunks)internal/api/v1/controllers_transactions_count.go(1 hunks)internal/api/v1/controllers_transactions_count_test.go(3 hunks)internal/api/v1/controllers_transactions_list.go(1 hunks)internal/api/v1/controllers_transactions_list_test.go(4 hunks)internal/api/v1/controllers_transactions_read.go(2 hunks)internal/api/v1/controllers_transactions_read_test.go(2 hunks)internal/api/v1/mocks_ledger_controller_test.go(9 hunks)internal/api/v1/utils.go(1 hunks)internal/api/v2/common.go(2 hunks)internal/api/v2/controllers_accounts_count.go(1 hunks)internal/api/v2/controllers_accounts_count_test.go(4 hunks)internal/api/v2/controllers_accounts_list.go(1 hunks)internal/api/v2/controllers_accounts_list_test.go(5 hunks)internal/api/v2/controllers_accounts_read.go(2 hunks)internal/api/v2/controllers_accounts_read_test.go(4 hunks)internal/api/v2/controllers_balances.go(1 hunks)internal/api/v2/controllers_balances_test.go(2 hunks)internal/api/v2/controllers_logs_list.go(1 hunks)internal/api/v2/controllers_logs_list_test.go(5 hunks)internal/api/v2/controllers_transactions_count.go(1 hunks)internal/api/v2/controllers_transactions_count_test.go(4 hunks)internal/api/v2/controllers_transactions_list.go(1 hunks)internal/api/v2/controllers_transactions_list_test.go(4 hunks)internal/api/v2/controllers_transactions_read.go(2 hunks)internal/api/v2/controllers_transactions_read_test.go(2 hunks)internal/api/v2/controllers_volumes.go(1 hunks)internal/api/v2/controllers_volumes_test.go(4 hunks)internal/api/v2/mocks_ledger_controller_test.go(9 hunks)internal/controller/ledger/controller.go(1 hunks)internal/controller/ledger/controller_default.go(4 hunks)internal/controller/ledger/controller_default_test.go(10 hunks)internal/controller/ledger/controller_generated_test.go(9 hunks)internal/controller/ledger/controller_with_traces.go(9 hunks)internal/controller/ledger/stats.go(1 hunks)internal/controller/ledger/stats_test.go(1 hunks)internal/controller/ledger/store.go(5 hunks)internal/controller/ledger/store_generated_test.go(5 hunks)internal/storage/ledger/accounts.go(3 hunks)internal/storage/ledger/accounts_test.go(11 hunks)internal/storage/ledger/balances.go(3 hunks)internal/storage/ledger/balances_test.go(1 hunks)internal/storage/ledger/debug.go(1 hunks)internal/storage/ledger/errors.go(0 hunks)internal/storage/ledger/legacy/accounts.go(6 hunks)internal/storage/ledger/legacy/accounts_test.go(19 hunks)internal/storage/ledger/legacy/adapters.go(2 hunks)internal/storage/ledger/legacy/balances.go(1 hunks)internal/storage/ledger/legacy/balances_test.go(8 hunks)internal/storage/ledger/legacy/logs.go(1 hunks)internal/storage/ledger/legacy/logs_test.go(2 hunks)internal/storage/ledger/legacy/queries.go(1 hunks)internal/storage/ledger/legacy/transactions.go(6 hunks)internal/storage/ledger/legacy/transactions_test.go(8 hunks)internal/storage/ledger/legacy/volumes.go(4 hunks)internal/storage/ledger/legacy/volumes_test.go(27 hunks)internal/storage/ledger/logs.go(3 hunks)internal/storage/ledger/logs_test.go(3 hunks)internal/storage/ledger/moves.go(1 hunks)internal/storage/ledger/moves_test.go(1 hunks)internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_aggregated_balances.go(1 hunks)internal/storage/ledger/resource_logs.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)
⛔ Files not processed due to max files limit (8)
- internal/storage/ledger/store.go
- internal/storage/ledger/transactions.go
- internal/storage/ledger/transactions_test.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/volumes.go
- internal/storage/ledger/volumes_test.go
- internal/volumes.go
- test/e2e/api_transactions_list_test.go
💤 Files with no reviewable changes (1)
- internal/storage/ledger/errors.go
🚧 Files skipped from review as they are similar to previous changes (47)
- internal/storage/ledger/paginator.go
- internal/api/v2/controllers_accounts_count.go
- internal/api/v1/controllers_transactions_read_test.go
- internal/storage/ledger/legacy/balances_test.go
- internal/api/v2/controllers_balances.go
- internal/storage/ledger/legacy/logs_test.go
- internal/api/v2/controllers_accounts_list.go
- internal/api/v1/controllers_accounts_list.go
- internal/api/v1/controllers_balances_aggregates.go
- internal/api/v1/controllers_balances_aggregates_test.go
- internal/api/v1/controllers_accounts_count.go
- internal/README.md
- internal/api/v1/controllers_balances_list.go
- internal/api/v2/controllers_logs_list.go
- internal/api/v1/controllers_transactions_list.go
- internal/api/v2/controllers_balances_test.go
- internal/api/v1/controllers_transactions_count.go
- internal/api/v1/controllers_accounts_read.go
- internal/api/v2/controllers_transactions_read.go
- internal/api/v1/controllers_accounts_read_test.go
- internal/api/v2/controllers_volumes.go
- internal/api/v2/controllers_logs_list_test.go
- internal/api/v1/controllers_transactions_read.go
- internal/api/v2/controllers_transactions_count.go
- internal/api/v2/controllers_transactions_list.go
- internal/storage/ledger/legacy/accounts_test.go
- internal/api/v1/controllers_accounts_count_test.go
- internal/controller/ledger/stats.go
- internal/storage/ledger/moves.go
- internal/api/v1/controllers_accounts_list_test.go
- internal/storage/ledger/legacy/balances.go
- internal/controller/ledger/stats_test.go
- internal/api/v2/controllers_transactions_read_test.go
- internal/storage/ledger/paginator_column.go
- internal/storage/ledger/legacy/logs.go
- internal/api/v2/controllers_accounts_read_test.go
- internal/api/v1/controllers_transactions_count_test.go
- internal/storage/ledger/paginator_offset.go
- internal/api/v2/controllers_volumes_test.go
- internal/api/v2/controllers_transactions_count_test.go
- internal/api/v2/controllers_accounts_count_test.go
- internal/storage/ledger/legacy/volumes.go
- internal/storage/ledger/debug.go
- internal/storage/ledger/resource_logs.go
- internal/storage/ledger/accounts_test.go
- internal/storage/ledger/legacy/accounts.go
- internal/storage/ledger/legacy/transactions.go
🔇 Additional comments (70)
internal/storage/ledger/legacy/adapters.go (3)
26-32: LGTM!
The Logs() and Transactions() methods are well-implemented, following a consistent pattern and maintaining type safety with appropriate pagination types.
34-37: Method name vs. resource name might be confusing.
The method is called "AggregatedBalances" but it instantiates a resource type for "AggregatedVolumes." Consider renaming this method to make it consistent and more intuitive for maintainers.
-func (d *DefaultStoreAdapter) AggregatedBalances() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] {
- return d.newStore.AggregatedVolumes()
+func (d *DefaultStoreAdapter) AggregatedVolumes() ledgercontroller.Resource[ledger.AggregatedVolumes, ledgercontroller.GetAggregatedVolumesOptions] {
+ return d.newStore.AggregatedVolumes()
}38-40: LGTM!
The Volumes() method is well-implemented with proper typing for paginated resources and volume-specific options.
internal/api/v2/mocks_ledger_controller_test.go (2)
Line range hint 178-191: LGTM! Appropriate use of specialized query options.
The GetAggregatedBalances method correctly uses a specialized options type GetAggregatedVolumesOptions while adopting the new ResourceQuery pattern.
Line range hint 73-86: LGTM! Consistent transition to generic resource queries.
The changes consistently update the method signatures to use ResourceQuery[any], making the query interface more flexible while maintaining type safety.
Let's verify that all resource query methods have been updated consistently:
Also applies to: 88-101, 163-176, 223-236
✅ Verification successful
All resource query methods consistently use ResourceQuery[any] across the codebase
The verification confirms that the transition to ResourceQuery[any] has been consistently implemented across:
- All mock implementations (v1, v2, common, bulking APIs)
- The actual controller implementations (DefaultController, ControllerWithTraces)
- Resource handlers (accounts, transactions, logs)
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify all resource query methods use the new generic type
# Expected: All resource query methods should use ResourceQuery[any]
rg -U --multiline 'func.*ResourceQuery\[any\].*\{[\s\S]*?\}' | grep -v 'indicates an expected call'
Length of output: 21040
internal/api/v1/controllers_transactions_list_test.go (4)
38-46: LGTM!
The nominal test case correctly sets up the basic query structure with appropriate defaults and volume expansion.
156-161: LGTM!
The cursor-related test cases properly validate both successful and error scenarios.
185-192: LGTM!
The page size validation test cases properly handle boundary conditions and enforce the maximum limit.
214-214: LGTM!
The mock expectation correctly uses the new query type for the ListTransactions method.
internal/api/v2/controllers_accounts_list_test.go (2)
217-218: LGTM! Mock expectations properly updated.
The mock expectations are correctly updated to use the new query type while maintaining the same test behavior.
32-32: LGTM! Verify pagination behavior compatibility.
The change to OffsetPaginatedQuery[any] aligns with the refactoring goal of making the query system more generic and flexible.
Let's verify that this change is consistent across the codebase:
✅ Verification successful
Based on the search results, I can now generate the final response:
Migration to OffsetPaginatedQuery[any] is consistent with codebase patterns
The change from PaginatedQueryOptions[ledgercontroller.PITFilterWithVolumes] to OffsetPaginatedQuery[any] aligns with the broader codebase migration pattern. The search results show:
- The
OffsetPaginatedQuery[any]type is consistently used across test files and controllers for account listing functionality - Other test files (e.g.,
internal/api/v2/controllers_accounts_list_test.go) already use this exact type - The change maintains compatibility with the pagination system while making it more generic
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for any remaining uses of PaginatedQueryOptions to ensure complete migration
rg "PaginatedQueryOptions" --type go
# Search for all uses of OffsetPaginatedQuery to verify consistent usage
rg "OffsetPaginatedQuery" --type go
Length of output: 33686
internal/api/v2/common.go (2)
4-4: Avoid dot imports for better code clarity
The dot import for collectionutils can lead to namespace pollution and reduced code clarity.
17-37: Well-structured date handling refactoring!
The centralization of date parsing logic into getDate with proper error handling and nil returns for missing dates is a good improvement.
internal/storage/ledger/legacy/volumes_test.go (4)
8-8: LGTM! Import changes align with the refactoring goal.
The addition of the ledgerstore package import is consistent with the transition from controller to store-based query handling.
Line range hint 20-24: LGTM! Test structure follows best practices.
The test functions are well-structured with:
- Parallel test execution for better performance
- Clear test case naming
- Comprehensive test data setup
Also applies to: 404-408
Line range hint 465-675: LGTM! Comprehensive test coverage with proper assertions.
The test cases thoroughly cover various scenarios:
- Different group levels (0-3)
- PIT and OOT filtering
- Balance filtering
- Metadata filtering
Each test case includes proper assertions for both result length and specific data values.
104-104: Verify the completeness of package migration.
The changes consistently replace ledgercontroller with ledgerstore across all query constructions. Let's verify that no instances were missed.
Also applies to: 112-112, 120-122, 141-143, 153-155, 165-167, 186-188, 207-209, 219-221, 231-233, 252-254, 274-276, 296-298, 317-319, 341-344, 367-369, 381-383, 394-396, 465-467, 478-480, 491-493, 504-506, 518-521, 552-555, 579-582, 624-626, 642-646, 663-665
internal/storage/ledger/legacy/transactions_test.go (3)
9-9: LGTM: Import changes align with the refactoring effort.
The addition of the ledgerstore import with appropriate aliasing maintains clear package separation.
Line range hint 50-93: LGTM: GetTransactionWithVolumes test cases properly migrated.
The migration from ledgercontroller to ledgerstore maintains the same test coverage and assertions while properly testing both transaction volumes and effective volumes.
Line range hint 162-277: LGTM: Comprehensive test coverage maintained through refactoring.
The test cases have been properly migrated to use the new query types while maintaining thorough coverage of various scenarios including filtering, metadata handling, and point-in-time queries.
Let's verify the test coverage:
✅ Verification successful
Test coverage is comprehensive and correctly migrated
The verification confirms:
- All test cases consistently use the new
NewPaginatedQueryOptionspattern with proper query construction - The test suite covers essential scenarios including filtering, metadata operations, and point-in-time queries
- The changes are well-integrated across the codebase with tests at multiple layers (storage, API v1/v2, and e2e)
- No legacy query patterns were found, indicating complete migration
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify test coverage for transaction-related operations
# Check for any remaining references to old query types
rg -t go "ledgercontroller\.(?:NewGetTransactionQuery|NewListTransactionsQuery)"
# Verify test coverage patterns
ast-grep --pattern 'func Test$_($_, $_) {
$$$
store.GetTransactions($_, $_)
$$$
}'
Length of output: 173
Script:
#!/bin/bash
# Let's try a different approach to verify the test coverage
# Check test coverage for GetTransactions
rg -A 5 "GetTransactions\(" internal/storage/ledger/legacy/transactions_test.go
# Check test coverage for transaction-related queries
rg -A 3 "NewPaginatedQueryOptions" internal/storage/ledger/legacy/transactions_test.go
# Check for any other transaction test files
fd -e go -E "vendor|node_modules" transactions.*test
Length of output: 4135
internal/storage/ledger/accounts.go (4)
7-7: LGTM: Import changes align with refactoring goals
The addition of tracing capabilities and removal of unused imports reflect the architectural changes towards improved observability.
21-26: LGTM: Enhanced observability with proper tracing
The function improvements include better naming conventions and comprehensive tracing capabilities.
74-85: LGTM: Clean implementation with proper error handling
The function implements proper tracing, error handling, and ledger filtering. The changes are consistent with the overall refactoring goals.
48-53: Consider potential race condition in metadata merge
The current implementation using || for metadata merge might lead to race conditions if multiple concurrent updates occur. Consider using a more atomic approach or adding appropriate transaction isolation level.
Consider one of these approaches:
- Use explicit locking
- Implement optimistic concurrency control with version numbers
- Use a JSON merge function with conflict resolution strategy
✅ Verification successful
The previous pattern didn't yield results, possibly due to syntax. Let's try a broader search to understand the context better, including:
- How metadata merging is handled across the codebase
- Whether transactions are used around these operations
- The table schema to understand the metadata column type
We found that the metadata merge operation exists in accounts.go, but we need to see:
- The complete context around this operation to understand if it's wrapped in a transaction
- The table definition to understand the metadata column type and any constraints
This will help us determine if the current implementation is safe or needs additional safeguards.
From the accounts.go file, we can see two different metadata merge operations:
- In UpdateAccountsMetadata:
metadata = excluded.metadata || accounts.metadata - In UpsertAccounts:
metadata = accounts.metadata || excluded.metadata
Both use PostgreSQL's JSONB concatenation operator and have WHERE clauses to prevent unnecessary updates. Let's find:
- The migration files to understand the table structure
- The Store struct to see if there's any default transaction isolation level configuration
We need to:
- Check the initial schema to understand the accounts table structure and any constraints
- Look at how the store is initialized to understand transaction handling
This will help us determine if PostgreSQL's built-in MVCC and the current implementation provide sufficient protection against race conditions.
Based on the comprehensive analysis of the codebase, I can now generate the final response:
PostgreSQL JSONB concatenation is safe for concurrent metadata updates
The current implementation is actually safe from race conditions for several reasons:
- PostgreSQL's JSONB concatenation operator (
||) is atomic at the database level - The operation is wrapped in an UPSERT with a WHERE clause that prevents unnecessary updates:
WHERE not accounts.metadata @> excluded.metadata - The table has proper constraints with a unique index on
(ledger, address) - Changes are tracked in
accounts_metadatatable through triggers for audit purposes
The implementation leverages PostgreSQL's MVCC (Multi-Version Concurrency Control) which ensures consistency during concurrent operations. The metadata merge is handled at the database level within a single atomic operation, making it safe for concurrent updates.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for potential concurrent metadata update handling
ast-grep --pattern 'metadata = excluded.metadata || accounts.metadata'
Length of output: 72
Script:
#!/bin/bash
# Let's search for the metadata merge pattern in SQL statements more broadly
rg -l "metadata.*excluded\.metadata.*metadata" --type sql --type go
# Also check for transaction handling around this operation
rg "BEGIN|START TRANSACTION" --type go -A 5 -B 5 internal/storage/ledger/
# Look for the table definition to understand metadata column type
rg "CREATE TABLE.*accounts" --type sql -A 10
Length of output: 228
Script:
#!/bin/bash
# Let's look at the complete accounts.go file to understand the transaction context
cat internal/storage/ledger/accounts.go
# Also check for any migration files that might contain the table definition
fd -e sql . migrations/
Length of output: 3866
Script:
#!/bin/bash
# Let's find migration files in the correct location
fd -e sql .
# Check for any transaction handling in the store initialization
rg "type Store struct" -A 10 internal/storage/ledger/
Length of output: 4563
Script:
#!/bin/bash
# Let's check the bucket migrations for the accounts table definition
cat internal/storage/bucket/migrations/0-init-schema/up.sql
# Check if there's any transaction configuration in the store initialization
rg "NewStore" -A 10 internal/storage/ledger/
Length of output: 27345
internal/storage/ledger/resource.go (1)
132-132:
Avoid panic for unsupported filter scenarios
Using panic("unreachable") here can crash the system if an unrecognized property is provided. Return an error instead of panicking to handle unexpected conditions more gracefully.
Example fix:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unsupported filter property: %s", property)Likely invalid or redundant comment.
internal/storage/ledger/resource_accounts.go (1)
125-129:
Check metadataRegex submatches to avoid index out-of-range
Similar to the balance logic, you risk panicking if the regex doesn’t match. Validate submatch lengths before using them.
Example adjustment:
- match := metadataRegex.FindAllStringSubmatch(property, 3)
- return "metadata @> ?", []any{map[string]any{
- match[0][1]: value,
- }}, nil
+ matches := metadataRegex.FindAllStringSubmatch(property, 3)
+ if len(matches) > 0 && len(matches[0]) > 1 {
+ return "metadata @> ?", []any{map[string]any{
+ matches[0][1]: value,
+ }}, nil
+ }
+ return "", nil, fmt.Errorf("invalid metadata property format: %s", property)Likely invalid or redundant comment.
internal/storage/ledger/legacy/queries.go (2)
78-85: Potential double expansion of volumes.
ListAccountsQuery provides methods WithExpandVolumes and WithExpandEffectiveVolumes. Ensure that calling both expansions doesn’t cause any unforeseen duplication or data collisions. Also consider whether expansions can be combined or if calling both leads to performance overhead.
152-153: Avoid using panic for unreachable cases.
Using panic("unreachable") can cause unexpected crashes at runtime. It’s typically safer to return an error so callers can handle the situation gracefully.
internal/storage/ledger/resource_transactions.go (2)
86-92: Ensure metadata consistency under PIT conditions.
Line 94-111 conditionally joins transaction metadata based on PIT. If a mismatch occurs between the transaction timestamp and the metadata revision date, the dataset might exclude relevant metadata. Consider clarifying how conflicts are resolved or whether fallback logic is needed.
153-154: Avoid using panic for unreachable code.
Repeating the same reasoning from previous reviews: returning an error is safer than crashing the application.
internal/storage/ledger/resource_volumes.go (1)
184-185: Use error returns instead of panic.
As discussed previously, panic("unreachable") can cause runtime crashes. Return an error so clients can handle unexpected states gracefully.
internal/controller/ledger/controller_default.go (1)
118-119: Document pagination constraints.
When using ListTransactions with ColumnPaginatedQuery[any], clarify the recommended page size boundaries to avoid high memory usage or slow queries.
internal/controller/ledger/store_generated_test.go (7)
5-5: Auto-generated notice
This line simply documents the mock generation command.
45-57: Validate that the PaginatedResource matches project needs
The new Accounts() method returns a PaginatedResource tied to OffsetPaginatedQuery. Verify that existing mock expectations and tests for Accounts still apply correctly with the updated PaginatedResource approach.
59-72: AggregatedBalances resource API
The new AggregatedBalances() method returning Resource[ledger.AggregatedVolumes, GetAggregatedVolumesOptions] looks consistent with the new resource-oriented design. Ensure any calls that formerly fetched aggregated balances are updated to call AggregatedBalances() in unit tests.
292-305: Transactions resource approach
Similarly, Transactions() returns a PaginatedResource. If the application’s transaction retrieval logic is now consolidated here, confirm that existing references to older methods are replaced.
355-368: Volumes resource
Volumes() is now a PaginatedResource. Check that calls returning volumes do not rely on previous, removed methods.
369-421: MockResource
These lines introduce a generic MockResource that implements Count and GetOne. Ensure that this new mock’s usage remains consistent whenever Resource[ResourceType, OptionsType] is required, and that no legacy mocks conflict.
422-488: MockPaginatedResource
These lines define a generic PaginatedResource mock with Count, GetOne, and Paginate. This abstraction should simplify testing. Validate the order and correctness of parameter passing, especially for Paginate methods, to avoid incorrectly enumerating resources.
internal/api/v2/controllers_accounts_read.go (2)
4-4: Import usage
The addition of "github.com/formancehq/go-libs/v2/query" is aligned with the new resource-based queries. Confirm that no redundant imports remain.
24-24: PIT parameter
The function getPIT is used to parse a “Point In Time”. Verify the logic in getPIT to ensure it gracefully handles invalid PIT values in all corner cases (e.g., empty, invalid format).
internal/api/v1/controllers_logs_list.go (1)
44-45: Builder assignment
Attaching the buildGetLogsQuery result to paginatedQuery.Options.Builder correctly integrates custom log filtering. Double-check that buildGetLogsQuery includes all intended filter clauses (e.g., after, date range).
internal/api/v1/utils.go (1)
26-44: Offset-based pagination
getOffsetPaginatedQuery introduces a new code path for offset-based pagination. Make sure that existing client code is updated to respect the same logic, especially around default page sizes handled by bunpaginate.
internal/storage/ledger/logs.go (2)
Line range hint 114-134: LGTM! Clean and consistent changes.
The function maintains proper error handling and follows the consistent naming pattern.
Line range hint 47-109: LGTM! Verify the feature flag usage across environments.
The changes improve error handling and maintain proper transaction isolation. The code correctly uses advisory locks when the FeatureHashLogs feature is enabled.
Let's verify the feature flag usage:
✅ Verification successful
Feature flag FeatureHashLogs is consistently implemented across the codebase
The feature flag is properly defined and used consistently:
- Defined in
pkg/features/features.gowith valid states "SYNC" and "DISABLED" - Used in three key locations:
internal/storage/ledger/logs.go: Advisory locks for transaction isolationinternal/storage/bucket/default_bucket.go: Database trigger setupinternal/controller/ledger/controller_default.go: Hash validation logic
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for consistent feature flag usage across the codebase
# Test: Search for FeatureHashLogs usage to ensure consistent behavior
rg -A 3 "FeatureHashLogs"
Length of output: 2552
internal/api/v1/controllers_logs_list_test.go (1)
109-111: LGTM! Mock expectations properly updated.
The test correctly verifies the interaction with the new query interface.
internal/storage/ledger/balances.go (1)
Line range hint 82-134: Verify transaction isolation level requirements.
The code uses FOR UPDATE for proper locking, but we should verify:
- The transaction isolation level requirements
- The potential for deadlocks with the chosen locking order
Let's check the transaction configuration:
✅ Verification successful
Let me gather more information about the transaction handling in the balances code:
Let me check one more thing about the balance query's usage context:
Let me try one final check to see the transaction handling in the controller layer:
Transaction isolation level is correctly implemented
The code's transaction handling is well-designed:
- The
FOR UPDATEclause in the balance query ensures proper locking - The store interface explicitly documents that
GetBalancesmust lock accounts until the end of the transaction - The system uses READ COMMITTED as the default isolation level, with SERIALIZABLE used only when needed (e.g., during imports)
- The chosen locking order (by accounts_address, asset) helps prevent deadlocks
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check transaction isolation level configuration
# Test: Search for transaction isolation level settings
rg -A 3 "isolation level|BEGIN|START TRANSACTION"
Length of output: 3209
Script:
#!/bin/bash
# Check transaction handling in the balances code
rg -A 5 "BeginTX|RunInTransaction" internal/storage/ledger/
Length of output: 1022
Script:
#!/bin/bash
# Find where balances query is used and check transaction context
ast-grep --pattern 'GetBalances($$$)'
Length of output: 3990
Script:
#!/bin/bash
# Check how balances are retrieved in the controller layer
rg -A 5 "GetBalances|ReadBalances" internal/controller/ledger/
Length of output: 4617
internal/storage/ledger/logs_test.go (1)
200-208: LGTM! Well-structured query builder usage.
The query builder implementation with date range filtering demonstrates good use of the new query infrastructure. The test effectively verifies the filtering functionality.
internal/storage/ledger/resource_aggregated_balances.go (3)
12-12: LGTM: Clean struct definition
The empty struct is appropriate as this is a stateless handler implementing the repository interface.
156-170: Consider adding error handling for JSON aggregation
The project method uses PostgreSQL's json_build_object and aggregate_objects functions without handling potential JSON conversion errors.
Consider adding error handling for JSON operations:
func (h aggregatedBalancesResourceRepositoryHandler) project(
store *Store,
_ ledgercontroller.ResourceQuery[ledgercontroller.GetAggregatedVolumesOptions],
selectQuery *bun.SelectQuery,
) (*bun.SelectQuery, error) {
+ // Ensure numeric values are valid for JSON conversion
+ sumVolumesForAsset := store.db.NewSelect().
+ TableExpr("(?) values", selectQuery).
+ Group("asset").
+ Column("asset").
+ ColumnExpr("CASE WHEN sum(((volumes).inputs)::numeric) IS NULL OR sum(((volumes).outputs)::numeric) IS NULL "+
+ "THEN NULL "+
+ "ELSE json_build_object('input', sum(((volumes).inputs)::numeric), 'output', sum(((volumes).outputs)::numeric)) "+
+ "END as volumes")14-47: Consider adding validation for metadata value types
The metadata filter validation only checks operators but doesn't validate the value types. This could lead to runtime errors if invalid types are provided.
Consider adding type validation in the metadata validator:
propertyValidatorFunc(func(l ledger.Ledger, operator string, key string, value any) error {
if key == "metadata" {
if operator != "$exists" {
return fmt.Errorf("unsupported operator %s for metadata", operator)
}
return nil
}
if operator != "$match" {
return fmt.Errorf("unsupported operator %s for metadata", operator)
}
+ switch value.(type) {
+ case string, float64, bool:
+ return nil
+ default:
+ return fmt.Errorf("unsupported value type for metadata: %T", value)
+ }
return nil
}),internal/controller/ledger/store.go (2)
60-64: LGTM: Clear interface extension
The new Store interface methods provide a clean abstraction for different resource types with appropriate pagination strategies.
246-247:
Consider adding error handling for missing accounts
The GetAccountsMetadata method assumes accounts exist but doesn't handle the case when an account is not found.
Add proper error handling:
v, err := s.Store.Accounts().GetOne(ctx, ResourceQuery[any]{
Builder: query.Match("address", address),
})
if err != nil {
return nil, err
}
+ if v == nil {
+ return nil, fmt.Errorf("account not found: %s", address)
+ }Likely invalid or redundant comment.
internal/api/v2/controllers_transactions_list_test.go (1)
33-33: LGTM: Test cases properly updated
The test cases have been correctly updated to use the new ColumnPaginatedQuery type with appropriate default values and options.
Also applies to: 42-50
internal/storage/ledger/balances_test.go (8)
242-261: LGTM! Test case properly validates aggregated volumes.
The test case correctly validates the new AggregatedVolumes().GetOne() API, ensuring proper aggregation of both input and output volumes across all accounts.
265-280: LGTM! Test case properly validates address filtering.
The test case correctly validates address-based filtering using the new API, ensuring proper aggregation of volumes for accounts matching the "users:" prefix.
284-299: LGTM! Test case properly validates point-in-time filtering.
The test case correctly validates point-in-time filtering using the new API, ensuring proper aggregation of volumes based on the effective date.
303-321: LGTM! Test case properly validates insertion date filtering.
The test case correctly validates point-in-time filtering using insertion dates with the new API, ensuring proper aggregation of volumes.
325-340: LGTM! Test case properly validates combined metadata and point-in-time filtering.
The test case correctly validates the combination of metadata-based and point-in-time filtering using the new API, ensuring proper aggregation of volumes.
344-356: LGTM! Test case properly validates metadata-only filtering.
The test case correctly validates metadata-based filtering using the new API, ensuring proper aggregation of volumes for accounts with specific metadata.
360-366: LGTM! Test case properly validates empty result handling.
The test case correctly validates the scenario where no accounts match the filter criteria, ensuring proper handling of empty results.
371-385: LGTM! Test case properly validates metadata existence filtering.
The test case correctly validates filtering based on metadata field existence using the new API, ensuring proper aggregation of volumes.
internal/controller/ledger/controller_default_test.go (1)
203-220: LGTM! Consistent use of generic query types across test cases.
The test cases have been properly updated to use generic query types (ResourceQuery[T], ColumnPaginatedQuery[T], OffsetPaginatedQuery[T]), improving type safety and flexibility. The mock expectations are correctly set up to match the new method signatures.
Also applies to: 232-238, 250-261, 273-284, 296-302, 314-327, 339-348, 358-373, 385-398
internal/controller/ledger/controller_with_traces.go (1)
Line range hint 198-210: LGTM! Consistent method signature updates with preserved tracing functionality.
The controller methods have been properly updated to use generic query types while maintaining the existing tracing functionality. The changes improve type safety and flexibility without affecting the tracing capabilities.
Also applies to: 210-222, 222-234, 234-246, 246-258, 258-270, 270-282, 282-330, 330-342
internal/controller/ledger/controller_generated_test.go (1)
Line range hint 1-11: Reminder: This is a generated file
The changes look good, but remember that this is a generated file (as indicated by the header comment). Ensure that these changes are made to the source file that generates this mock, not directly to this file.
Let's verify the source file exists and contains the corresponding interface:
✅ Verification successful
Generated mock matches the source interface correctly
The source file controller.go contains the Controller interface with the updated generic query types (ResourceQuery, OffsetPaginatedQuery, ColumnPaginatedQuery) that match exactly with the changes in the generated mock file. This confirms that the changes are being made correctly in the source file and the mock is being regenerated appropriately.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the source file for the mock
# The mockgen command in the header indicates the source file is controller.go
# Check if the source file exists and contains the Controller interface
rg -A 5 "type Controller interface" internal/controller/ledger/controller.go
Length of output: 363
Script:
#!/bin/bash
# Let's verify if the source file (controller.go) contains the updated interface methods with generic query types
rg -A 2 "ResourceQuery|OffsetPaginatedQuery|ColumnPaginatedQuery" internal/controller/ledger/controller.go
Length of output: 1135
internal/api/v1/mocks_ledger_controller_test.go (1)
Line range hint 1-11: Reminder: This is a generated file
The changes look good and are consistent with the refactoring pattern seen in other files. Remember that this is a generated file and changes should be made to the source file specified in the header comment.
internal/api/common/mocks_ledger_controller_test.go (1)
Line range hint 1-11: Reminder: This is a generated file
The changes look good and maintain consistency with other mock implementations. Remember that this is a generated file and changes should be made to the source file specified in the header comment.
internal/api/bulking/mocks_ledger_controller_test.go (3)
Line range hint 178-191: Well-implemented type-safe query transformation
Good implementation of the generic query pattern while maintaining type safety through GetAggregatedVolumesOptions. This could serve as a model for other query methods.
Line range hint 238-251: Well-structured pagination with type safety
Good use of OffsetPaginatedQuery with specific GetVolumesOptions type, maintaining type safety while supporting the new query pattern.
Line range hint 73-86: Consider using more specific type parameters instead of any
While transitioning to ResourceQuery improves flexibility, using any as the type parameter might reduce type safety. Consider creating specific types or interfaces for the query options to maintain compile-time type checking.
Let's check if there are any specific query option types defined in the codebase:
Also applies to: 88-101, 163-176, 223-236
| expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ | ||
| PageSize: DefaultPageSize, | ||
| Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), | ||
| Column: "id", | ||
| Options: ledgercontroller.ResourceQuery[any]{ | ||
| Builder: query.Gte("date", now.Format(time.DateFormat)), | ||
| Expand: []string{"volumes"}, | ||
| }, | ||
| }, | ||
| }, | ||
| { | ||
| name: "using endTime", | ||
| queryParams: url.Values{ | ||
| "end_time": []string{now.Format(time.DateFormat)}, | ||
| }, | ||
| expectQuery: ledgercontroller.NewPaginatedQueryOptions(ledgercontroller.PITFilterWithVolumes{}). | ||
| WithQueryBuilder(query.Lt("date", now.Format(time.DateFormat))), | ||
| expectQuery: ledgercontroller.ColumnPaginatedQuery[any]{ | ||
| PageSize: DefaultPageSize, | ||
| Order: pointer.For(bunpaginate.Order(bunpaginate.OrderDesc)), | ||
| Column: "id", | ||
| Options: ledgercontroller.ResourceQuery[any]{ | ||
| Builder: query.Lt("date", now.Format(time.DateFormat)), | ||
| Expand: []string{"volumes"}, | ||
| }, | ||
| }, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add test case for combined time range query
The test suite covers individual start_time and end_time queries, but it's missing a test case that combines both to verify date range functionality. Consider adding a test case with both parameters to ensure proper handling of time ranges.
internal/storage/ledger/resource.go
Outdated
| if found, err = regexp.MatchString("^"+option+"$", key); err != nil { | ||
| panic(err) |
There was a problem hiding this comment.
Avoid panic on regex errors
Panicking on an unexpected regex failure will crash the service. Consider returning an error instead.
Suggested diff:
- if found, err = regexp.MatchString("^"+option+"$", key); err != nil {
- panic(err)
+ if found, err = regexp.MatchString("^"+option+"$", key); err != nil {
+ return nil, fmt.Errorf("failed to match regex for key '%s': %w", key, err)
}Also propagate the error or handle it gracefully in the surrounding code.
Committable suggestion skipped: line range outside the PR's diff.
| }}, nil | ||
| } | ||
|
|
||
| panic("unreachable") |
There was a problem hiding this comment.
Return error instead of panicking
Consider returning an error when encountering an unexpected branch. Panics can cause the entire application to crash.
Example fix:
- panic("unreachable")
+ return "", nil, fmt.Errorf("unsupported property or operator: %s", property)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| panic("unreachable") | |
| return "", nil, fmt.Errorf("unsupported property or operator: %s", property) |
| type GetTransactionQuery struct { | ||
| PITFilterWithVolumes | ||
| ID int | ||
| } |
There was a problem hiding this comment.
Validate transaction ID.
The GetTransactionQuery struct includes an integer ID. If negative or zero values slip in, it could lead to invalid queries or unintended results. Consider adding guards or validations to ensure the ID is valid.
| func (h transactionsResourceHandler) buildDataset(store *Store, opts repositoryHandlerBuildContext[any]) (*bun.SelectQuery, error) { | ||
| ret := store.db.NewSelect(). | ||
| ModelTableExpr(store.GetPrefixedRelationName("transactions")). | ||
| Column( | ||
| "ledger", | ||
| "id", | ||
| "timestamp", | ||
| "reference", | ||
| "inserted_at", | ||
| "updated_at", | ||
| "postings", | ||
| "sources", | ||
| "destinations", | ||
| "sources_arrays", | ||
| "destinations_arrays", | ||
| ). | ||
| Where("ledger = ?", store.ledger.Name) | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Caution with partial address validation.
The filters method includes address-based filters (account, source, destination). Ensure that partial addresses or wildcard logic do not inadvertently match unintended addresses. A stricter or well-documented approach can help prevent subtle query mismatches.
| func (h volumesResourceHandler) buildDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetVolumesOptions]) (*bun.SelectQuery, error) { | ||
|
|
||
| var selectVolumes *bun.SelectQuery | ||
|
|
||
| needAddressSegments := query.useFilter("address", isPartialAddress) | ||
| if !query.UsePIT() && !query.UseOOT() { | ||
| selectVolumes = store.db.NewSelect(). | ||
| Column("asset", "input", "output"). | ||
| ColumnExpr("input - output as balance"). | ||
| ColumnExpr("accounts_address as account"). | ||
| ModelTableExpr(store.GetPrefixedRelationName("accounts_volumes")). | ||
| Where("ledger = ?", store.ledger.Name). | ||
| Order("accounts_address", "asset") | ||
|
|
||
| if query.useFilter("metadata") || needAddressSegments { | ||
| subQuery := store.db.NewSelect(). | ||
| TableExpr(store.GetPrefixedRelationName("accounts")). | ||
| Column("address"). | ||
| Where("ledger = ?", store.ledger.Name). | ||
| Where("accounts.address = accounts_address") | ||
|
|
||
| if needAddressSegments { | ||
| subQuery = subQuery.ColumnExpr("address_array as account_array") | ||
| selectVolumes = selectVolumes.Column("account_array") | ||
| } | ||
| if query.useFilter("metadata") { | ||
| subQuery = subQuery.ColumnExpr("metadata") | ||
| selectVolumes = selectVolumes.Column("metadata") | ||
| } | ||
|
|
||
| selectVolumes = selectVolumes. | ||
| Join(`join lateral (?) accounts on true`, subQuery) | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Optimize subqueries for large datasets.
The buildDataset method may produce nested subqueries, especially if needAddressSegments or the metadata filter is used. For very large ledgers, multiple subqueries with JOIN LATERAL can degrade performance. Consider logging or monitoring this to detect potential slowdowns in production.
| ColumnPaginatedQuery[any]{ | ||
| PageSize: 100, | ||
| Order: pointer.For(bunpaginate.Order(bunpaginate.OrderAsc)), | ||
| }, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Confirm export usage for large data sets.
Export uses a fixed PageSize of 100. For ledgers with hundreds of thousands of logs, consider allowing dynamic page sizes, or a streaming approach, to avoid potential performance bottlenecks.
| func getColumnPaginatedQuery[v any](r *http.Request, column string, order bunpaginate.Order, modifiers ...func(*v) error) (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| return bunpaginate.Extract[ledgercontroller.ColumnPaginatedQuery[v]](r, func() (*ledgercontroller.ColumnPaginatedQuery[v], error) { | ||
| rq, err := getResourceQuery[v](r, modifiers...) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| pageSize, err := bunpaginate.GetPageSize(r, bunpaginate.WithMaxPageSize(MaxPageSize), bunpaginate.WithDefaultPageSize(DefaultPageSize)) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return &ledgercontroller.ColumnPaginatedQuery[v]{ | ||
| PageSize: pageSize, | ||
| Column: column, | ||
| Order: pointer.For(order), | ||
| Options: *rq, | ||
| }, nil | ||
| }) | ||
| } |
There was a problem hiding this comment.
Column-based pagination
The getColumnPaginatedQuery function introduces sorting columns and order. Validate potential column name injection if user input is not sanitized. If only a fixed column is allowed, consider restricting user-provided columns to a known safe set.
| func (h aggregatedBalancesResourceRepositoryHandler) buildDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) { | ||
|
|
||
| if query.UsePIT() { | ||
| ret := store.db.NewSelect(). | ||
| ModelTableExpr(store.GetPrefixedRelationName("moves")). | ||
| DistinctOn("accounts_address, asset"). | ||
| Column("accounts_address", "asset"). | ||
| Where("ledger = ?", store.ledger.Name) | ||
| if query.Opts.UseInsertionDate { | ||
| if !store.ledger.HasFeature(features.FeatureMovesHistory, "ON") { | ||
| return nil, ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistory) | ||
| } | ||
|
|
||
| ret = ret. | ||
| ColumnExpr("first_value(post_commit_volumes) over (partition by (accounts_address, asset) order by seq desc) as volumes"). | ||
| Where("insertion_date <= ?", query.PIT) | ||
| } else { | ||
| if !store.ledger.HasFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes, "SYNC") { | ||
| return nil, ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes) | ||
| } | ||
|
|
||
| ret = ret. | ||
| ColumnExpr("first_value(post_commit_effective_volumes) over (partition by (accounts_address, asset) order by effective_date desc, seq desc) as volumes"). | ||
| Where("effective_date <= ?", query.PIT) | ||
| } | ||
|
|
||
| if query.useFilter("address", isPartialAddress) { | ||
| subQuery := store.db.NewSelect(). | ||
| TableExpr(store.GetPrefixedRelationName("accounts")). | ||
| Column("address_array"). | ||
| Where("accounts.address = accounts_address"). | ||
| Where("ledger = ?", store.ledger.Name) | ||
|
|
||
| ret = ret. | ||
| ColumnExpr("accounts.address_array as accounts_address_array"). | ||
| Join(`join lateral (?) accounts on true`, subQuery) | ||
| } | ||
|
|
||
| if query.useFilter("metadata") { | ||
| subQuery := store.db.NewSelect(). | ||
| DistinctOn("accounts_address"). | ||
| ModelTableExpr(store.GetPrefixedRelationName("accounts_metadata")). | ||
| ColumnExpr("first_value(metadata) over (partition by accounts_address order by revision desc) as metadata"). | ||
| Where("ledger = ?", store.ledger.Name). | ||
| Where("accounts_metadata.accounts_address = moves.accounts_address"). | ||
| Where("date <= ?", query.PIT) | ||
|
|
||
| ret = ret. | ||
| Join(`left join lateral (?) accounts_metadata on true`, subQuery). | ||
| Column("metadata") | ||
| } | ||
|
|
||
| return ret, nil | ||
| } else { | ||
| ret := store.db.NewSelect(). | ||
| ModelTableExpr(store.GetPrefixedRelationName("accounts_volumes")). | ||
| Column("asset", "accounts_address"). | ||
| ColumnExpr("(input, output)::"+store.GetPrefixedRelationName("volumes")+" as volumes"). | ||
| Where("ledger = ?", store.ledger.Name) | ||
|
|
||
| if query.useFilter("metadata") || query.useFilter("address", isPartialAddress) { | ||
| subQuery := store.db.NewSelect(). | ||
| TableExpr(store.GetPrefixedRelationName("accounts")). | ||
| Column("address"). | ||
| Where("ledger = ?", store.ledger.Name). | ||
| Where("accounts.address = accounts_address") | ||
|
|
||
| if query.useFilter("address") { | ||
| subQuery = subQuery.ColumnExpr("address_array as accounts_address_array") | ||
| ret = ret.Column("accounts_address_array") | ||
| } | ||
| if query.useFilter("metadata") { | ||
| subQuery = subQuery.ColumnExpr("metadata") | ||
| ret = ret.Column("metadata") | ||
| } | ||
|
|
||
| ret = ret. | ||
| Join(`join lateral (?) accounts on true`, subQuery) | ||
| } | ||
|
|
||
| return ret, nil | ||
| } | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Verify feature flags before query construction
The buildDataset method checks feature flags after constructing partial queries. This could lead to wasted computation if features are disabled.
Consider moving the feature checks to the beginning of the method:
func (h aggregatedBalancesResourceRepositoryHandler) buildDataset(store *Store, query repositoryHandlerBuildContext[ledgercontroller.GetAggregatedVolumesOptions]) (*bun.SelectQuery, error) {
+ if query.UsePIT() {
+ if query.Opts.UseInsertionDate && !store.ledger.HasFeature(features.FeatureMovesHistory, "ON") {
+ return nil, ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistory)
+ }
+ if !query.Opts.UseInsertionDate && !store.ledger.HasFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes, "SYNC") {
+ return nil, ledgercontroller.NewErrMissingFeature(features.FeatureMovesHistoryPostCommitEffectiveVolumes)
+ }
+ }Committable suggestion skipped: line range outside the PR's diff.
| type ResourceQuery[Opts any] struct { | ||
| PIT *time.Time `json:"pit"` | ||
| OOT *time.Time `json:"oot"` | ||
| Builder query.Builder `json:"qb"` | ||
| Expand []string `json:"expand,omitempty"` | ||
| Opts Opts `json:"opts"` | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Consider adding validation for PIT and OOT combinations
The ResourceQuery type allows both PIT and OOT to be set simultaneously, which might lead to ambiguous queries.
Add validation in UnmarshalJSON:
func (rq *ResourceQuery[Opts]) UnmarshalJSON(data []byte) error {
type rawResourceQuery ResourceQuery[Opts]
type aux struct {
rawResourceQuery
Builder json.RawMessage `json:"qb"`
}
x := aux{}
if err := json.Unmarshal(data, &x); err != nil {
return err
}
+ if x.PIT != nil && !x.PIT.IsZero() && x.OOT != nil && !x.OOT.IsZero() {
+ return fmt.Errorf("cannot specify both PIT and OOT")
+ }
+
var err error
*rq = ResourceQuery[Opts](x.rawResourceQuery)
rq.Builder, err = query.ParseJSON(string(x.Builder))
return err
}Committable suggestion skipped: line range outside the PR's diff.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (17)
test/e2e/api_transactions_list_test.go (1)
52-52: Recommend caution with ephemeral timestamps in tests.
Using time.Now() might introduce non-determinism in test results if system time fluctuates or if parallel tests are run. You might consider freezing or injecting a fixed time to improve reproducibility.internal/storage/ledger/resource.go (1)
300-300: Replace direct printing with proper logging
Line 300 prints the query to stdout using fmt.Println, which is typically discouraged in production environments. Consider using a structured logger to control verbosity and avoid cluttering stdout.internal/storage/ledger/resource_volumes.go (6)
3-11: Consider adding descriptive package or file-level comments.
Having a high-level overview at the top of the file provides quick context on the purpose of the “volumesResourceHandler” and how it fits into the ledger's resource management.
15-21: Add docstring for the filters method.
While the method is fairly straightforward, a short docstring explaining its role in routing or validating filters (e.g., address, metadata) can help future maintainers.
57-89: Beware of potential performance bottlenecks when querying large datasets.
The subqueries with JOIN LATERAL may be expensive on large tables. Depending on usage patterns, you could consider indexes on “accounts.address” or “accounts.metadata,” or introduce caching strategies for repeated queries.
91-144: Reduce branching by unifying PIT vs. OOT logic.
The code branches significantly based on whether PIT or OOT is in play. Consider extracting shared query elements (like building the core SELECT) into a helper function to reduce duplication and centralize maintenance.
149-185: Expand operator coverage or document unsupported filters.
The resolveFilter method does a good job handling different filter types but might need more robust handling for future expansions (e.g., partial matches on address or asset). Add unit tests ensuring all valid/invalid paths remain well-defined.
212-214: Leave a TODO for future expansions or remove the expand method.
If no expansions are available for this resource, consider removing the method or adding a clarifying comment to avoid confusion.internal/storage/ledger/resource_transactions.go (3)
47-49: Add proper validators for timestamp.
The TODO comment at line 47 highlights the missing validators for the "timestamp" filter. This filter can accept different operators, so ensure user input is validated or sanitized to avoid invalid or unexpected queries.
68-93: Consider indexing frequently filtered columns.
Columns like “timestamp” and “id” appear regularly in filtering conditions, so adding appropriate indexes can improve query performance, especially in large ledgers.
158-190: Complex subquery for effectiveVolumes.
The expand method’s nested DISTINCT ON query can be demanding at scale. Consider adding indexes on (transactions_id, accounts_address, asset) if performance or concurrency issues arise in high-volume ledgers.internal/storage/ledger/transactions.go (3)
150-177: Consider adding documentation for the complex query logicWhile the implementation is correct, the complex union query logic would benefit from additional documentation explaining the atomic update-or-retrieve pattern.
Line range hint
180-207: Consider enhancing error handling for already reverted transactionsWhile the query correctly filters out already reverted transactions, consider returning a specific error type for this case to provide better feedback to the caller.
func (store *Store) RevertTransaction(ctx context.Context, id int, at time.Time) (tx *ledger.Transaction, modified bool, err error) { + var ErrTransactionAlreadyReverted = errors.New("transaction already reverted") _, err = tracing.TraceWithMetric( ctx, "RevertTransaction", store.tracer, store.revertTransactionHistogram, func(ctx context.Context) (*ledger.Transaction, error) { tx, modified, err = store.updateTxWithRetrieve(ctx, id, query) + if err == nil && !modified { + return nil, ErrTransactionAlreadyReverted + } return nil, err }, ) return tx, modified, err }
Line range hint
37-260: Consider extracting common query patternsThe codebase shows repeated patterns for:
- Tracing setup
- Table name prefixing
- Error handling
- Metadata operations
Consider creating helper functions or a query builder to reduce duplication and improve maintainability.
internal/storage/ledger/transactions_test.go (3)
153-156: Duplicate pattern for the GetOne call.
You might consider extracting a small helper to reduce boilerplate in tests, although this is minor.
160-163: Repeated usage.
Like the previous comment, this pattern is repeated multiple times; a helper function could streamline it.
855-855: DumpTables for debugging.
Useful for diagnosing test failures or data mismatches. Ensure logs are acceptable for CI environments.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)internal/storage/ledger/transactions.go(4 hunks)internal/storage/ledger/transactions_test.go(14 hunks)internal/storage/ledger/utils.go(2 hunks)test/e2e/api_transactions_list_test.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- internal/storage/ledger/utils.go
- internal/storage/ledger/resource_accounts.go
🔇 Additional comments (50)
test/e2e/api_transactions_list_test.go (5)
9-9: All good regarding new import usage.
No issues found with introducing the "github.com/formancehq/go-libs/v2/query" import.
11-11: Import of libtime library is appropriate.
Makes sense for handling and formatting time accurately.
79-84: Validation of multi-asset postings.
The additional posting for "EUR" in the same transaction is valid and helpful for robust testing of multi-asset transactions. Make sure the downstream logic handles multiple postings correctly.
243-248: New query structure usage is properly implemented.
The code aligns with the new ledgercontroller.ResourceQuery, ensuring consistent usage of the $match operator. No issues found.
286-294: Query with $and operator is well-structured.
Combining multiple match conditions properly tests the new flexible querying approach. This usage looks correct and enhances test specificity.
internal/storage/ledger/resource.go (2)
17-31: Handle unsupported operators without panicking
This function still uses panic("unreachable") for unsupported operators, which can cause the application to crash. Consider returning an error instead and handling it gracefully to avoid bringing the entire system down.
283-283: Avoid panic for unexpected pagination query type
Using panic("should not happen") can terminate the application if a new or unrecognized pagination query type is introduced. Instead, consider returning an error or handling this scenario more gracefully to ensure resilience.
internal/storage/ledger/resource_volumes.go (2)
187-210: Validate grouping for large address arrays.
When grouping by partial addresses, ensure the resulting grouped arrays will not exceed memory limits. You may want to log or monitor for unexpectedly large group outputs in production.
216-216: Good practice to ensure compile-time interface conformance.
This line is helpful for catching refactoring mistakes early and ensures that volumesResourceHandler fully implements the repositoryHandler interface.
internal/storage/ledger/resource_transactions.go (4)
14-66: Overall structure of filters looks solid.
The definition of multiple filters for different transaction properties is well-organized. Keep an eye on property-level validation coverage to ensure future maintainability.
94-111: Metadata handling alignment check.
When FeatureAccountMetadataHistory is enabled, you join transaction metadata with a subquery. Ensure you have tests covering scenarios where metadata changes over time, verifying correct data retrieval.
122-152: Safe property whitelisting and operator conversion.
Because the switch statement explicitly checks known properties (id, reference, timestamp, etc.), you mitigate the risk of injection by restricting arbitrary property usage. Confirm that any newly introduced property is properly whitelisted here.
140-145: Double-check partial address handling and JSON structure.
The logic with metadataRegex and “aggregate_objects” function can be powerful. However, confirm that partial matches or wildcard-like inputs do not cause unintended matches or performance issues.
Also applies to: 178-182
internal/storage/ledger/transactions.go (5)
65-66: LGTM! In-place modification issue fixed
The previous issue with modifying tx.Postings in-place has been properly addressed by creating a copy before reversing.
Line range hint 108-146: LGTM! Well-structured error handling and tracing
The implementation includes:
- Proper error resolution using postgres.ResolveError
- Explicit handling of transaction reference conflicts
- Comprehensive tracing with metrics
211-234: LGTM! Efficient metadata update implementation
The implementation includes:
- Optimistic locking to prevent unnecessary updates
- Proper metadata merging using PostgreSQL jsonb operators
- Comprehensive tracing
237-260: LGTM! Efficient metadata deletion implementation
The implementation includes:
- Proper validation of key existence before deletion
- Optimized query to avoid unnecessary updates
- Comprehensive tracing
Line range hint 37-260: LGTM! Changes align with refactoring objectives
The implementation successfully:
- Maintains backward compatibility
- Improves code structure and error handling
- Introduces better tracing and metrics
- Aligns with the goal of refactoring the read store
internal/storage/ledger/paginator_offset.go (4)
47-53: Guard against integer overflow in offset calculations.
As previously noted by another reviewer, subtracting query.PageSize from query.Offset after converting them to int could overflow if the values are extremely large. Consider stricter boundary checks to prevent potential int32 overflow on certain systems.
59-61: Good check for offset overflow on next page calculation.
This condition ensures that adding page size to the current offset does not exceed MaxUint64, preventing potential overflow errors.
67-73: Cursor construction logic looks consistent.
Returning the subset of records with potential for "next" pagination is well-handled. The approach of slicing off the extra record for the hasMore condition is a standard pattern.
16-17: Consider verifying usage for this seemingly unused type.
The "nolint:unused" directive suggests that the type might not be directly referenced within the codebase. If it is truly used externally or via generics, this is fine; otherwise, removing it could improve clarity.
internal/storage/ledger/transactions_test.go (28)
55-58: Looks good using the new GetOne method.
This change aligns well with the newly introduced generic ResourceQuery, ensuring consistent retrieval of a transaction by ID with expanded volumes.
79-82: Consistent usage of the GetOne method.
Reusing the same ResourceQuery approach keeps the test logic consistent and clear.
116-116: Good transition to Count with ResourceQuery.
This change replaces CountTransactions. It cleanly aligns with the new data retrieval pattern.
196-198: Retrieve transaction without expansions.
Skipping expansions here is fine if volumes are not needed in this test.
207-209: Consistent retrieval.
Same query style as above. The minimal approach (no expansions) focuses on validating only metadata.
452-457: Usage of Paginate with expanded volumes.
This looks correct. The approach ensures test coverage for pagination and partial expansions.
521-524: Retrieving transaction with expanded volumes.
Ensures correct PostCommitVolumes after insertion in the past. Good coverage.
536-538: New GetOne call on store.Accounts.
Retrieving an account with ResourceQuery fosters consistency in the codebase. Check for possible expansions if needed.
578-578: Verifying non-existent transaction revert.
Hardcoding ID 2 is reasonable for negative testing. Good approach to confirm error handling is correct.
679-679: Extra posting broadens test coverage.
Including a second currency (EUR) is a welcome coverage improvement to confirm multi-currency handling.
720-723: GetOne with expansions after revert.
Ensures volumes are recalculated appropriately. This is a good test scenario.
738-738: ColumnPaginatedQuery usage.
Adopting this generic type parameter helps maintain a consistent query interface.
745-745: Empty query object.
Provides a baseline case for pagination tests. This is a valid approach to cover default behavior.
750-754: Match “account” usage is clear.
Filtering by account is straightforward, verifying that the new query builder works for targeted addresses.
759-763: Filter with an unmatched account.
Good negative test coverage, ensuring no results are returned when a match does not exist.
768-772: Segment-based match usage.
Enables partial address matching for broader queries. A helpful test scenario.
777-781: Metadata filter test.
Clearly shows how the new query handles key lookup within metadata.
786-790: Point-in-time (PIT) usage.
Verifies transaction queries at a specific historical moment, an advanced and useful feature.
795-799: Invalid key check.
Ensures that erroneous query fields trigger the correct error. Good negative testing.
804-808: Reverted transaction filter.
Confirming that the new system can query only reverted transactions is an important scenario.
813-817: Exists() operator usage.
Tests searching for a transaction that has a particular metadata key. Clear coverage.
822-827: Combining metadata match with PIT.
Verifies multi-criteria queries. Ensures data consistency across different dimensions (metadata + PIT).
832-836: Not(...) operator usage.
Testing negative conditions on metadata ensures completeness of coverage.
841-845: Filtering with a timestamp.
Matches a precise value. Great to confirm time-based slices are handled as expected.
857-858: Dynamic expansions.
Modifying the expansion list at runtime is a concise way to keep tests flexible.
859-859: Pagination call.
Combines Paginate with expansions to retrieve volumes. The approach thoroughly validates querying.
867-867: Count after Paginate.
A robust check ensuring list and count queries align. Nicely done.
874-874: End of test file.
No comment needed, all changes look consistent with the broader refactor.
| if query.Offset > 0 { | ||
| sb = sb.Offset(int(query.Offset)) | ||
| } |
There was a problem hiding this comment.
💡 Codebase verification
Add bounds check for offset value to prevent integer overflow
The code shows that Offset is defined as uint64 in the query struct, and when converting to int for the bun query builder, there's a risk of overflow on 32-bit systems where int is 32 bits. Add a validation check before the conversion:
if query.Offset > math.MaxInt32 {
return nil, fmt.Errorf("offset value exceeds maximum allowed value")
}
if query.Offset > 0 {
sb = sb.Offset(int(query.Offset))
}🔗 Analysis chain
Ensure large offset values won't overflow int conversion.
Casting a potentially large uint64 offset to an int (for bun.SelectQuery.Offset) might cause unexpected overflows on some systems. Consider a safety check or boundary enforcement.
Let me check the query struct definition and any existing validation to better understand the potential overflow risk.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Search for usage of Offset(int(query.Offset)) and cross-check maximum supported offsets
rg -A 3 '(Offset\(int\(.*Offset\)\))'
Length of output: 280
Script:
#!/bin/bash
# Find the definition of the query struct containing Offset field
ast-grep --pattern 'type $_ struct {
$$$
Offset $_
$$$
}'
# Also search for any validation of the Offset field
rg -B 3 -A 3 'Offset.*uint64'
Length of output: 1073
gfyrag
force-pushed
the
refactor/store-read-part
branch
from
c048862 to
19d27b2
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
internal/storage/ledger/resource.go (2)
17-31:⚠️ Potential issueAvoid panicking for unsupported operators
This code calls panic("unreachable") for unsupported operators. Panics should be reserved for truly unrecoverable errors. Instead, return an error or handle the case gracefully to avoid crashing the application.
283-284:⚠️ Potential issueReplace panic with graceful error handling
Using panic("should not happen") can lead to unexpected application crashes when new or unrecognized pagination query types are introduced. Wrap this scenario in an error return or handle it gracefully to achieve safer, more maintainable code.
🧹 Nitpick comments (9)
test/e2e/api_transactions_list_test.go (2)
79-84: Consider using different amounts for different currencies.The test currently uses the same amount (100) for both USD and EUR transactions. To better reflect real-world scenarios and catch potential currency-specific issues, consider using different amounts for different currencies.
{ Amount: big.NewInt(100), Asset: "USD", Source: "world", Destination: fmt.Sprintf("account:%d", i), }, { - Amount: big.NewInt(100), + Amount: big.NewInt(85), // Using a different amount for EUR Asset: "EUR", Source: "world", Destination: fmt.Sprintf("account:%d", i), },
243-248: LGTM! Consider adding edge case tests.The query structure changes look good, providing better type safety with the new
querypackage. The test covers both simple and complex query scenarios.Consider adding test cases for:
- Empty query builder
- Maximum query complexity
- Invalid query structure
Also applies to: 286-294
internal/storage/ledger/resource_volumes.go (2)
15-53: Ensure consistent operator usage handling among filters
The filter definitions for "address"/"balance" implement$lt,$gt, etc., and the metadata filter implements$exists/$match. Ensure that all relevant operators are well-documented and validate user input consistently for each filter type to prevent confusion or potential errors.
212-214: Expansion placeholder
Currently, expand() returns an error. If future specifications call for expansions in this resource handler, a graceful fallback or partial expansion might be preferred over an outright error.internal/storage/ledger/resource_transactions.go (2)
14-66: Consider centralizing address filter validation for maintainability.Within the filters() method, multiple entries (account, source, destination) rely on validateAddressFilter. While this approach works, you might centralize or unify the logic to reduce repetition and potential divergence in the future.
158-190: Validate large expansions in the expand() method.The expand method dynamically generates a nested query for computing post_commit_effective_volumes (lines 163-190). While this is a powerful feature, consider how it scales for high transaction volumes. A batch or incremental retrieval strategy might be necessary if expansions become too large or slow.
internal/storage/ledger/paginator_offset.go (1)
11-14: Well-designed generic type structureThe
offsetPaginatortype is well-designed with generic parameters that allow for type-safe implementations across different resource types. The default configuration fields provide good flexibility for customization.Consider documenting the type's purpose and generic parameters in a comment block above the type definition to help future maintainers understand its usage.
internal/storage/ledger/transactions_test.go (2)
452-457: Check Expand fields in Paginate
Currently only “volumes” is requested. If effective volumes are also required, consider adding them.- Expand: []string{"volumes"}, + Expand: []string{"volumes", "effectiveVolumes"},
855-855: Dumping transactions table
Consider removing or guarding this debug statement in production.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
internal/storage/ledger/paginator.go(1 hunks)internal/storage/ledger/paginator_column.go(1 hunks)internal/storage/ledger/paginator_offset.go(1 hunks)internal/storage/ledger/resource.go(1 hunks)internal/storage/ledger/resource_accounts.go(1 hunks)internal/storage/ledger/resource_transactions.go(1 hunks)internal/storage/ledger/resource_volumes.go(1 hunks)internal/storage/ledger/transactions.go(4 hunks)internal/storage/ledger/transactions_test.go(14 hunks)internal/storage/ledger/utils.go(2 hunks)test/e2e/api_transactions_list_test.go(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- internal/storage/ledger/paginator.go
- internal/storage/ledger/utils.go
- internal/storage/ledger/paginator_column.go
- internal/storage/ledger/resource_accounts.go
🔇 Additional comments (75)
internal/storage/ledger/resource.go (6)
127-131: LGTM: Good use of property validation
This loop properly invokes each property validator, ensuring that only valid operators and values are processed. This design is clear and modular.
155-162: Clear design for building filtered dataset
The code neatly separates filter validation (validateFilters) from dataset building, promoting readability and maintainability.
185-215: LGTM: Expand logic is well-structured
You’ve provided a tidy mechanism for performing left joins and dataset expansions based on query parameters. This approach is straightforward and consistent.
217-241: GetOne method is concise and robust
Fetching a single record after building and expanding the dataset is cleanly implemented, returning a clear not-found error when appropriate.
266-309: Pagination flow is consistent and well-organized
Your approach to building the filtered dataset, paginating, then expanding the results is logically sequenced. Combining these steps ensures each layer of functionality is composed in a clear manner.
323-351: Mapper pattern effectively transforms data
Using a mapper to convert the original resource type into a core representation (via ToCore) cleanly separates transformations from the underlying storage logic. This helps maintain a cleaner domain model.
internal/storage/ledger/resource_volumes.go (6)
57-89: Potential performance bottleneck in subqueries
Nested subqueries joined via JOIN LATERAL (lines 88–89) may degrade performance on large datasets. This issue is reminiscent of concerns highlighted in previous reviews, where we recommended monitoring or refactoring for efficiency.
91-93: Feature flag usage
The error handling for missing features is correct. Confirm that the feature flag name "FeatureMovesHistory" matches the intended feature toggle, and that any calling code is prepared to handle the returned error.
132-143: Check metadata retrieval logic
When retrieving metadata via the subquery on “accounts_metadata”, the current approach uses “DISTINCT ON” and picks the first revision in descending order by date. Confirm that this meets the business requirement for retrieving the most recent or relevant metadata.
149-185: Validate filter value binding against SQL injection
The code appears to rely on Bun’s parameter binding, which should be safe. However, confirm that the dynamic operator strings (e.g., "balance "+convertOperatorToSQL(operator)) cannot be manipulated in ways that compromise security.
187-210: Flexibly handle grouping
The grouping strategy with grouping level 0 returning raw columns, and > 0 grouping on partial account slices, is nicely designed. If future expansions require deeper grouping logic, consider introducing smaller helper methods to keep this function streamlined.
216-216: Interface compliance
Declaring “var _ repositoryHandler…” confirms that volumesResourceHandler satisfies the repositoryHandler interface, which is a good pattern to catch integration issues at compile time.
internal/storage/ledger/resource_transactions.go (3)
122-128: Timestamp and reference filters are now consolidated.
The consolidation of "reference" and "timestamp" filters into a single case helps avoid duplication. This is a good improvement over potential double handling. Ensure that the combined logic (property == "reference" || property == "timestamp") covers all use cases, and no extraneous logic was dropped.
191-191: Neat type assertion ensures compile-time enforcement.
Using var _ repositoryHandler[any] = transactionsResourceHandler{} is a well-known pattern in Go to ensure that transactionsResourceHandler implements repositoryHandler[any]. This is a clean approach to guaranteeing interface compliance.
68-120: Check for potential performance issues with complex queries.
The buildDataset method can produce large and complex queries, especially when metadata history is included (lines 94-109). Depending on data volume, such queries may impact performance. Consider reviewing query execution plans and adding appropriate indexes or caching to ensure efficiency.
✅ Verification successful
Performance concerns are adequately addressed through existing indexes
Based on the SQL definitions found, the query in buildDataset is well-supported by existing indexes:
- The main transactions query is covered by
transactions_ledgerindex (onledger, id) - The metadata subquery is optimized with:
transactions_metadata_ledgerindex (onledger, transactions_seq, revision)transactions_metadata_revisionsindex specifically designed for theDISTINCT ONpattern, covering(transactions_seq, revision desc)with included columns
- Additional indexes exist for all JSON columns using GIN indexes
The query structure follows PostgreSQL best practices by using appropriate indexes for the join conditions and WHERE clauses. The performance concerns raised in the review comment are already addressed through the comprehensive indexing strategy.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# This script checks if relevant indexes exist on "transactions(id)", "transactions_metadata(transactions_id)" and "transactions_metadata(date)"
# that could help performance. We look for CREATE INDEX statements referencing these columns.
rg -A 2 "CREATE INDEX"
Length of output: 24
Script:
#!/bin/bash
# Let's check for SQL migrations or schema files that might contain index definitions
fd -e sql -x cat {} \; | grep -i -A 2 'index\|CREATE TABLE'
Length of output: 5410
Script:
#!/bin/bash
# Let's also check if there are any Go files containing SQL schema definitions
rg -A 5 'CREATE TABLE|INDEX' --type go
Length of output: 562
Script:
#!/bin/bash
# Let's specifically look for files related to schema or migrations
fd "schema|migration" -t f
Length of output: 446
internal/storage/ledger/paginator_offset.go (4)
1-9: LGTM: Clean package structure and imports
The imports are well-organized and all dependencies are necessary for the implementation.
48-56: Guard against integer overflow in offset calculations
The offset calculation could potentially overflow when dealing with large offsets or page sizes.
79-79: LGTM: Good interface compliance check
The type assertion ensures that offsetPaginator properly implements the paginator interface at compile time.
16-77: Verify error handling coverage
Let's ensure all error cases are properly handled throughout the pagination implementation.
✅ Verification successful
Error handling is comprehensive and well-implemented
The paginator implementation shows proper error handling for critical scenarios:
- Validates offset values against maximum allowed limits (MaxInt32)
- Checks for potential integer overflow when calculating next page offset
- Returns clear, specific error messages for each failure case
No issues found with nil pointer handling or unchecked error conditions.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for comprehensive error handling in pagination implementation
# Look for error returns and error handling patterns
rg -B 2 -A 2 'return.*error|if.*err.*!= nil|fmt\.Errorf'
# Look for potential panic scenarios
ast-grep --pattern 'if $_ != nil {
$$$
} else {
$$$
}'
Length of output: 105986
Script:
#!/bin/bash
# Let's specifically check error handling in the paginator code
rg -B 2 -A 2 'return.*error|if.*err.*!= nil|fmt\.Errorf' internal/storage/ledger/paginator_offset.go
# Also check for potential nil pointer dereferences
ast-grep --pattern 'if $_ != nil {
$$$
} else {
$$$
}' internal/storage/ledger/paginator_offset.go
Length of output: 522
internal/storage/ledger/transactions_test.go (28)
55-58: Confirm correct usage of ResourceQuery and expansions
No issues found here. The usage of “volumes” and “effectiveVolumes” in the Expand list is consistent with the new resource querying approach.
79-82: Usage of ResourceQuery expansions
This duplicated pattern is consistent with lines 55-58.
153-156: Usage of ResourceQuery expansions
No issues discovered; this usage is consistent with the new approach.
160-163: Repetitive usage of ResourceQuery expansions
This appears to mirror lines 153-156.
196-198: GetOne call without expansions
No volumes expansions here, but seems intentional for this test scenario.
207-209: Repeated pattern of GetOne usage
Mirrors lines 196-198.
521-524: ResourceQuery expansions
No issues found; usage looks consistent.
720-723: Consistent usage of GetOne expansions
Looks good. Matches other calls that expand “volumes” and “effectiveVolumes.”
738-738: New testCase struct
Struct extension for resource query tests is clear and well structured.
745-745: Nominal test case
No issues identified; query is empty by design to retrieve all.
750-754: Address filter test
Logic for “bob” is verified.
759-763: Address filtering with segment
Same approach as previous filter tests.
768-772: Filter using segment
Repetitive usage; no issues noted.
777-781: Metadata-based filter
No issues found; test logic is consistent.
786-786: Point-in-time filter
Usage of PIT is valid.
Also applies to: 790-790
795-799: Filter using invalid key
Test is properly capturing error scenario.
804-808: Reverted transactions filter
Properly tests “reverted” condition.
813-817: Exists metadata filter
Covers queries with metadata existence checks.
822-827: Filter with PIT and metadata
Mixed usage test is valid and thorough.
832-836: Not Exists metadata filter
Same pattern as above, but negation.
841-845: Filter by exact timestamp
Ensures time-based matching is correct.
857-858: Applying expansions in query
Expanding both “volumes” and “effectiveVolumes” ensures thorough coverage.
859-859: Executing paginate
No issues found. Return handling is consistent with prior code.
874-874: No meaningful change
This appears to be just the closing brace.
867-867: Ensuring count matches Paginate
Double-check that query filters match in both Paginate and Count.
✅ Verification successful
Let me gather more context about the Paginate usage to verify if the same query filters are used.
Let me check the test case structure to verify if the same query options are used for both Count and Paginate.
Query filters match correctly between Count and Paginate operations
The code shows that both operations use the same query options (tc.query.Options) within the same test case, and there's a validation that the count matches the paginated results length. The implementation is correct and consistent.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Verify usage of Count with the same query as Paginate
rg -A 3 "store\.Transactions\(\)\.Count"
Length of output: 1440
Script:
#!/bin/bash
# Search for Paginate usage in the same test file
rg -B 5 -A 5 "store\.Transactions\(\)\.Paginate" internal/storage/ledger/transactions_test.go
Length of output: 719
Script:
#!/bin/bash
# Get the test case context where both Count and Paginate are used
rg -B 20 -A 10 "tc\.query\.Options\)" internal/storage/ledger/transactions_test.go
Length of output: 749
116-116: Confirm behavior for empty ResourceQuery
Empty queries typically count all transactions. Consider verifying that all relevant results are indeed retrieved.
✅ Verification successful
Let me gather more context about the test case and the implementation to verify the behavior of empty ResourceQuery.
Let me check the test implementation to understand how the empty query case is being tested.
Based on the test implementation, I can now provide a final response as I have enough context to verify the behavior of empty ResourceQuery.
Empty ResourceQuery correctly counts all transactions
The test case TestTransactionsCount explicitly verifies that an empty ResourceQuery returns the total count of transactions. The test creates 3 transactions and confirms that the empty query returns the count as 3, which is the expected behavior.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Attempt to verify that an empty query returns the total number of transactions
rg -A 3 "store\.Transactions\(\)\.Count"
Length of output: 1440
Script:
#!/bin/bash
# Check the test setup and implementation
ast-grep --pattern 'func TestTransactions_Count($$$)' internal/storage/ledger/transactions_test.go
# Check ResourceQuery implementation and usage
rg -B 2 -A 5 "type ResourceQuery" internal/controller/ledger/
Length of output: 686
Script:
#!/bin/bash
# Get the full test implementation
cat internal/storage/ledger/transactions_test.go
Length of output: 25244
536-538: Ensure address-based filter correctness
Confirm that matching “address = bank” retrieves accurate results and no expansions are needed here.
✅ Verification successful
Address-based filter is correctly used and verified
Based on the codebase analysis, the address = "bank" filter in GetOne is consistently used across the codebase for account lookups. The test context shows this is part of a transaction test where the "bank" account is properly initialized and used in multiple transactions, and the query pattern matches the standard approach used throughout the application for address-based account lookups.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Verify usage of GetOne with 'address' filter.
rg -A 4 $'Accounts().GetOne'
Length of output: 29
Script:
#!/bin/bash
# Let's try a broader search for GetOne usage patterns
rg -A 4 "GetOne.*address"
# Also search for account-related tests
rg -A 4 'func Test.*Account'
# Let's also check the actual implementation of GetOne
ast-grep --pattern 'func ($x *$_) GetOne($$$) {
$$$
}'
Length of output: 9873
Script:
#!/bin/bash
# Let's check the specific test file and its context
rg -A 10 "bank" internal/storage/ledger/transactions_test.go
# Also check the implementation of the Accounts interface
ast-grep --pattern 'type $x interface {
$$$
GetOne($$$)
$$$
}'
# Look for other similar address-based queries
rg "Match.*address"
Length of output: 9401
679-679: Additional EUR posting
Ensure ledger processes multiple currencies gracefully in subsequent logic.
✅ Verification successful
Let me search for more context about currency handling in the codebase.
Let me search for the transaction handling logic to verify multi-currency support.
Based on the extensive search results, I can now provide a final response:
Multiple currencies are well-supported in the codebase
The codebase shows comprehensive test coverage for multi-currency transactions, including EUR and USD postings in the same transaction (transactions_test.go) and proper balance tracking across different currencies (balances_test.go). The additional EUR posting is consistent with existing patterns.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search the codebase for usage referencing multiple currencies in test scenarios
rg -A 5 'NewPosting.*EUR'
Length of output: 4560
Script:
#!/bin/bash
# Search for test cases with multiple currencies in the same transaction or related logic
rg -A 5 'NewPosting.*USD|NewPosting.*GBP'
# Search for currency-related test assertions
rg -B 2 -A 2 'require.*Currency|assert.*Currency'
# Look for currency validation or processing logic
ast-grep --pattern 'func $_($$$ Currency $$$) {
$$$
}'
Length of output: 53549
Script:
#!/bin/bash
# Search for transaction validation and processing logic
ast-grep --pattern 'func $_($$$ *Transaction $$$) {
$$$
}'
# Look for currency-related validation or constraints
rg -B 2 -A 5 'currency|Currency' internal/storage/ledger/
Length of output: 1118
internal/storage/ledger/transactions.go (28)
37-37: Refactored CommitTransaction signature
Usage of (store *Store) clarifies context.
39-39: Updating post commit volumes
Error handling appears appropriate.
45-45: Inserting transaction
No issues found; errors are correctly wrapped.
50-50: Upsert accounts
Properly ensures accounts exist with timestamps updated.
52-56: Fields for newly upserted accounts
Initializing “Metadata” as an empty map is a safe default.
63-63: Conditional moves history
Feature gating logic is well-structured.
65-66: Reversing postings
This is the fix recommended in a previous review to avoid in-place reversal.
96-96: InsertMoves call
No issues with batch insert of moves.
100-100: Applying post commit effective volumes
Checks a feature flag to conditionally set effective volumes.
108-108: InsertTransaction method
Clear top-level wrapper for insertion logic.
109-109: Tracing and metrics
Utilizes tracing for performance and error insights.
112-113: Instrumentation references
No issues found using store.tracer and store.insertTransactionHistogram.
115-118: Building insert query
Correctly sets table name, ledger name, and returns ID, timestamp, and inserted_at.
122-122: Auto-increment ID handling
Uses a ledger-scoped sequence for transaction IDs.
146-146: End of InsertTransaction function
No changes requiring comment.
150-150: updateTxWithRetrieve helper
Combines update logic and row retrieval.
157-157: Ensuring updated row retrieval
UnionAll strategy merges updated and original row.
180-180: RevertTransaction method
Straightforward approach for marking transactions reverted.
184-185: Tracing revertTransaction
No issues found.
187-187: Building revert update query
Uses bun’s chainable methods to set fields.
192-192: Ledger matching
Ensures revert only affects the correct ledger.
204-204: Performing the updateTxWithRetrieve
Captures updated or not modified transaction.
211-211: UpdateTransactionMetadata
Method for partial metadata updates.
215-216: Tracing updateTransactionMetadata
Instrumentation usage is consistent with other methods.
218-218: Query structure for updating metadata
Appropriately merges metadata with “||” operator.
Also applies to: 221-221, 223-223, 225-225, 231-231
237-237: DeleteTransactionMetadata
Removes a single metadata key from transaction.
241-242: Tracing deleteTransactionMetadata
Follows the same pattern as other methods.
244-244: Updating metadata by removal
Query uses JSONB subtraction operator to remove the key.
Also applies to: 247-247, 249-249, 253-253, 257-257
| orderExpression := fmt.Sprintf("%s %s", paginationColumn, originalOrder) | ||
| sb = sb.ColumnExpr("row_number() OVER (ORDER BY " + orderExpression + ")") |
There was a problem hiding this comment.
Potential SQL injection vulnerability in order expression
The order expression is constructed using string concatenation, which could be vulnerable to SQL injection if the paginationColumn or originalOrder values are not properly sanitized.
Consider using parameterized queries or implementing strict validation:
-orderExpression := fmt.Sprintf("%s %s", paginationColumn, originalOrder)
-sb = sb.ColumnExpr("row_number() OVER (ORDER BY " + orderExpression + ")")
+// Add validation for allowed column names
+if !isValidColumn(paginationColumn) {
+ return nil, fmt.Errorf("invalid pagination column")
+}
+sb = sb.ColumnExpr("row_number() OVER (?)", bun.Safe(fmt.Sprintf("ORDER BY %s %s", paginationColumn, originalOrder)))Committable suggestion skipped: line range outside the PR's diff.
* feat: improve queries performance and refactor storage read part * fix: post commit effective volumes rendering * fix: from review
No description provided.