Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime error: nil pointer dereference in roaring.go", line 667, in (*Bitmap).GetCardinality #1662

Closed
agnivade opened this issue Mar 10, 2022 · 7 comments
Closed

runtime error: nil pointer dereference in roaring.go", line 667, in (*Bitmap).GetCardinality #1662

agnivade opened this issue Mar 10, 2022 · 7 comments
Milestone
v2.4.3

Comments

@agnivade
Copy link

Sorry to bother you with more issues. We have another Sentry crash, this time in the roaring bitmap library, but it gets called from the zap library.

runtime.errorString: runtime error: invalid memory address or nil pointer dereference
  File "github.com/RoaringBitmap/[email protected]/roaring.go", line 667, in (*Bitmap).GetCardinality
  File "github.com/blevesearch/zapx/[email protected]/posting.go", line 239, in (*PostingsList).Count
  File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go", line 177, in (*IndexSnapshotTermFieldReader).Count
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go", line 76, in (*TermSearcher).Count
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 259, in (*ConjunctionSearcher).Count
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 259, in (*ConjunctionSearcher).Count
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 403, in (*BooleanSearcher).Count
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/ordered_searchers_list.go", line 30, in OrderedSearcherList.Less
  File "sort/sort.go", line 40, in insertionSort
  File "sort/sort.go", line 222, in quickSort
  File "sort/sort.go", line 231, in Sort
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 54, in NewConjunctionSearcher
  File "github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go", line 78, in (*ConjunctionQuery).Searcher
  File "github.com/blevesearch/bleve/[email protected]/search/query/boolean.go", line 132, in (*BooleanQuery).Searcher
  File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 471, in (*indexImpl).SearchInContext
  File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 368, in (*indexImpl).Search
  File "github.com/mattermost/mattermost-server/v6/services/searchengine/bleveengine/search.go", line 215, in (*BleveEngine).SearchPosts
  File "github.com/mattermost/mattermost-server/v6/store/searchlayer/post_layer.go", line 150, in SearchPostStore.searchPostsForUserByEngine
  File "github.com/mattermost/mattermost-server/v6/store/searchlayer/post_layer.go", line 176, in SearchPostStore.SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/store/timerlayer/timerlayer.go", line 5295, in (*TimerLayerPostStore).SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/app/post.go", line 1322, in (*App).SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/api4/post.go", line 602, in searchPosts
  File "github.com/mattermost/mattermost-server/v6/api4/post.go", line 555, in searchPostsInTeam
  File "github.com/mattermost/mattermost-server/v6/web/handlers.go", line 339, in Handler.ServeHTTP
  File "github.com/mattermost/[email protected]/gzip.go", line 337, in GzipHandlerWithOpts.func1.1
  File "net/http/server.go", line 2049, in HandlerFunc.ServeHTTP
  File "github.com/gorilla/[email protected]/mux.go", line 210, in (*Router).ServeHTTP
  File "net/http/server.go", line 2049, in HandlerFunc.ServeHTTP
  File "net/http/server.go", line 2867, in serverHandler.ServeHTTP
  File "net/http/server.go", line 1932, in (*conn).serve

Looking at the exact line of error, it's here: https://github.com/RoaringBitmap/roaring/blob/e6471ed95f1d549fe1b284310217906a9d62035e/roaring.go#L667. This seems a race condition, or somehow a nil container was passed.
@agnivade
Copy link
Author

Hi, any progress on this? We still receive sentry crashes for this.

@jgschis
Copy link

jgschis commented Jan 10, 2023

This happens to me too sometimes, I will look into this as I thought it was just me.

@jgschis
Copy link

jgschis commented Mar 3, 2023

I stopped using the min property of the disjunction query, and then this error stopped.

however, it's started happening again.

So again I am looking into it.

@agarciamontoro
Copy link

We are seeing several crash reports, either equal to the one reported here, or very similar, using Bleve 2.3.8. Some end up crashing in zapx, others in RoaringBitmap:

runtime.boundsError: runtime error: index out of range [4] with length 0
  File "github.com/RoaringBitmap/[email protected]/roaringarray.go", line 347, in (*roaringArray).getKeyAtIndex
  File "github.com/RoaringBitmap/[email protected]/roaring.go", line 1382, in AndNot
  File "github.com/blevesearch/zapx/[email protected]/posting.go", line 224, in (*PostingsList).iterator
  File "github.com/blevesearch/zapx/[email protected]/posting.go", line 155, in (*PostingsList).Iterator
  File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index.go", line 582, in (*IndexSnapshot).TermFieldReader
  File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go", line 149, in (*IndexSnapshotTermFieldReader).Advance
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go", line 106, in (*TermSearcher).Advance
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 253, in (*ConjunctionSearcher).advanceChild
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 200, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 218, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 165, in (*BooleanSearcher).advanceNextMust
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 316, in (*BooleanSearcher).Next
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 396, in (*BooleanSearcher).Advance
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 253, in (*ConjunctionSearcher).advanceChild
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 200, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 165, in (*BooleanSearcher).advanceNextMust
  File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 316, in (*BooleanSearcher).Next
  File "github.com/blevesearch/bleve/[email protected]/search/collector/topn.go", line 205, in (*TopNCollector).Collect
  File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 551, in (*indexImpl).SearchInContext
  File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 369, in (*indexImpl).Search

runtime.errorString: runtime error: invalid memory address or nil pointer dereference
File "github.com/RoaringBitmap/[email protected]/roaring.go", line 810, in (*Bitmap).GetCardinality
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 243, in (*PostingsList).Count
File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go", line 194, in (*IndexSnapshotTermFieldReader).Count
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go", line 77, in (*TermSearcher).Count
File "github.com/blevesearch/bleve/[email protected]/search/searcher/ordered_searchers_list.go", line 30, in OrderedSearcherList.Less
File "sort/zsortinterface.go", line 12, in insertionSort
File "sort/zsortinterface.go", line 73, in pdqsort
File "sort/sort.go", line 48, in Sort
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 56, in NewConjunctionSearcher
File "github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go", line 79, in (*ConjunctionQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/search/query/match.go", line 169, in (*MatchQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go", line 59, in (*ConjunctionQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/search/query/boolean.go", line 133, in (*BooleanQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go", line 59, in (*ConjunctionQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/search/query/boolean.go", line 133, in (*BooleanQuery).Searcher
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 485, in (*indexImpl).SearchInContext
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 369, in (*indexImpl).Search

runtime.errorString: runtime error: integer divide by zero
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 644, in (*PostingsIterator).nextDocNumAtOrAfter
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 537, in (*PostingsIterator).nextAtOrAfter
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 526, in (*PostingsIterator).Next
File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go", line 88, in (*IndexSnapshotTermFieldReader).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go", line 89, in (*TermSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 218, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 218, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 246, in (*ConjunctionSearcher).Advance
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 356, in (*BooleanSearcher).Advance
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 253, in (*ConjunctionSearcher).advanceChild
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 190, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 165, in (*BooleanSearcher).advanceNextMust
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 316, in (*BooleanSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/collector/topn.go", line 226, in (*TopNCollector).Collect
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 551, in (*indexImpl).SearchInContext
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 369, in (*indexImpl).Search

These come from Sentry, so I don't have much more data, hope it's somewhat useful.

@jgschis
Copy link

jgschis commented Feb 17, 2024

I still get the same issue in Couchbase. I have talked to the team about this and gave them an application to reproduce it, and they did reproduce it. But not sure what's come of it since then (almost a year ago).

@cpoile
Copy link

cpoile commented Sep 11, 2024

We received another crash, bleve -> zapx -> RoaringBitmap, this time in AndCardinality:

runtime.errorString: runtime error: invalid memory address or nil pointer dereference
github.com/RoaringBitmap/[email protected]/roaring.go in (*Bitmap).AndCardinality at line 996
github.com/blevesearch/zapx/[email protected]/posting.go in (*PostingsList).Count at line 245
github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go in (*IndexSnapshotTermFieldReader).Count at line 194
github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go in newTermSearcherFromReader at line 64
github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go in NewTermSearcherBytes at line 54
github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go in NewTermSearcher at line 45
github.com/blevesearch/bleve/[email protected]/search/query/term.go in (*TermQuery).Searcher at line 62
github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go in (*ConjunctionQuery).Searcher at line 60
github.com/blevesearch/bleve/[email protected]/search/query/match.go in (*MatchQuery).Searcher at line 169
github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go in (*ConjunctionQuery).Searcher at line 60
github.com/blevesearch/bleve/[email protected]/search/query/boolean.go in (*BooleanQuery).Searcher at line 134
github.com/blevesearch/bleve/[email protected]/search/query/conjunction.go in (*ConjunctionQuery).Searcher at line 60
github.com/blevesearch/bleve/[email protected]/search/query/boolean.go in (*BooleanQuery).Searcher at line 134
github.com/blevesearch/bleve/[email protected]/index_impl.go in (*indexImpl).SearchInContext at line 499
github.com/blevesearch/bleve/[email protected]/index_impl.go in (*indexImpl).Search at line 371

@cpoile
Copy link

cpoile commented Sep 16, 2024

We've received a couple more of these. The first:

runtime.errorString: runtime error: integer divide by zero
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 644, in (*PostingsIterator).nextDocNumAtOrAfter
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 537, in (*PostingsIterator).nextAtOrAfter
File "github.com/blevesearch/zapx/[email protected]/posting.go", line 526, in (*PostingsIterator).Next
File "github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go", line 88, in (*IndexSnapshotTermFieldReader).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go", line 89, in (*TermSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 218, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 218, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 246, in (*ConjunctionSearcher).Advance
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 356, in (*BooleanSearcher).Advance
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 253, in (*ConjunctionSearcher).advanceChild
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go", line 190, in (*ConjunctionSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 165, in (*BooleanSearcher).advanceNextMust
File "github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go", line 316, in (*BooleanSearcher).Next
File "github.com/blevesearch/bleve/[email protected]/search/collector/topn.go", line 226, in (*TopNCollector).Collect
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 551, in (*indexImpl).SearchInContext
File "github.com/blevesearch/bleve/[email protected]/index_impl.go", line 369, in (*indexImpl).Search

The second:

runtime error: integer divide by zero
github.com/blevesearch/zapx/[email protected]/posting.go in (*PostingsIterator).nextDocNumAtOrAfterClean at line 747
github.com/blevesearch/zapx/[email protected]/posting.go in (*PostingsIterator).nextDocNumAtOrAfter at line 632
github.com/blevesearch/zapx/[email protected]/posting.go in (*PostingsIterator).nextAtOrAfter at line 537
github.com/blevesearch/zapx/[email protected]/posting.go in (*PostingsIterator).Advance at line 532
github.com/blevesearch/bleve/[email protected]/index/scorch/snapshot_index_tfr.go in (*IndexSnapshotTermFieldReader).Advance at line 169
github.com/blevesearch/bleve/[email protected]/search/searcher/search_term.go in (*TermSearcher).Advance at line 109
github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go in (*ConjunctionSearcher).advanceChild at line 253
github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go in (*ConjunctionSearcher).Next at line 200
github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go in (*ConjunctionSearcher).Next at line 218
github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go in (*BooleanSearcher).advanceNextMust at line 165
github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go in (*BooleanSearcher).Next at line 316
github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go in (*BooleanSearcher).Advance at line 396
github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go in (*ConjunctionSearcher).advanceChild at line 253
github.com/blevesearch/bleve/[email protected]/search/searcher/search_conjunction.go in (*ConjunctionSearcher).Next at line 190
github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go in (*BooleanSearcher).advanceNextMust at line 165
github.com/blevesearch/bleve/[email protected]/search/searcher/search_boolean.go in (*BooleanSearcher).Next at line 316
github.com/blevesearch/bleve/[email protected]/search/collector/topn.go in (*TopNCollector).Collect at line 228
github.com/blevesearch/bleve/[email protected]/index_impl.go in (*indexImpl).SearchInContext at line 580
github.com/blevesearch/bleve/[email protected]/index_impl.go in (*indexImpl).Search at line 371

Hope that helps?

@abhinavdangeti abhinavdangeti mentioned this issue Sep 16, 2024
Merged
abhinavdangeti added a commit to blevesearch/zapx that referenced this issue Oct 16, 2024
@abhinavdangeti
Add nil pointer checks for postings before dereference
f23ed6a 
For blevesearch/bleve#1662
@abhinavdangeti abhinavdangeti mentioned this issue Oct 16, 2024
Merged
abhinavdangeti added a commit to blevesearch/zapx that referenced this issue Oct 16, 2024
@abhinavdangeti
Add nil pointer checks for postings before dereference (#271)
bf971e6 
For blevesearch/bleve#1662
@abhinavdangeti abhinavdangeti mentioned this issue Oct 16, 2024
Merged
abhinavdangeti added a commit that referenced this issue Oct 16, 2024
@abhinavdangeti
MB-57871: Upgrade zapx/v16 and zapx/v15 for fixes (#2091)
02d37a4 
v15:
* a2c264a Abhi Dangeti | [v15] Add nil pointer checks for postings
before dereference
* 96e55e7 Mohd Shaad Khan | Handle zero chunk size

v16:
* 7bab2f9 Abhi Dangeti | MB-57871: One more nil check before
dereferencing postings
* bf971e6 Abhi Dangeti | Add nil pointer checks for postings before
dereference
* 9a80f5f Abhi Dangeti | One more case for zero chunk size / mode
* 0c6f397 Mohd Shaad Khan | Handle zero chunk size

For #1662
@abhinavdangeti abhinavdangeti added this to the v2.4.3 milestone Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v2.4.3
Development

No branches or pull requests
7 participants
@cpoile @agnivade @abhinavdangeti @agarciamontoro @Thejas-bhat @moshaad7 @jgschis