Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Built-in indexer RocksDB filling disk with logs #3730

Closed
gavriilos opened this issue Nov 29, 2022 · 2 comments
Closed

Built-in indexer RocksDB filling disk with logs #3730

gavriilos opened this issue Nov 29, 2022 · 2 comments
Labels
t:bug Type: This doesn't seem right.

Comments

@gavriilos
Copy link

Bug Report

The RocksDB for the built-in indexer is filling my disk with logs. Under ~/.ckb/data/indexer/secondary_path/ I had 25 GB accumulated in the LOG* files. I also had about 5 GB in the logs under ~/.ckb/data/indexer/store/.

On the core RocksDB, I was able to control this by setting keep_log_file_num=1 in the options file (~/.ckb/default.db-options). Unfortunately I found no such file for the built-in indexer.

Please provide a way to limit the indexer RocksDB's log files size.

Environment

  • CKB version: 0.105.1
  • Chain: mainnet
  • Operating system: Ubuntu 20.04
  • Arch: x64
  • Installation: GitHub Release
@gavriilos gavriilos added the t:bug Type: This doesn't seem right. label Nov 29, 2022
@driftluo
Copy link
Collaborator

ref: #3696

@zhangsoledad zhangsoledad mentioned this issue Dec 2, 2022
Merged
bors bot added a commit that referenced this issue Dec 6, 2022
@bors @zhangsoledad
Merge #3735
9ebc667 
3735: feat: indexer db simple configuration r=zhangsoledad a=zhangsoledad

<!--
Thank you for contributing to nervosnetwork/ckb!

If you haven't already, please read [CONTRIBUTING](https://github.com/nervosnetwork/ckb/blob/develop/CONTRIBUTING.md) document.

If you're unsure about anything, just ask; somebody should be along to answer within a day or two.

PR Title Format:
1. module [, module2, module3]: what's changed
2. *: what's changed
-->

### What problem does this PR solve?
Resolve configuration issues with Indexer db
#3696
#3730


### What is changed and how it works?

the RocksDB debug log is not really of much use to the average user, just setting keep_log_file_num to 1 (0 is not allowed). only maintain a single LOG file which is probably the cleanest and simplest solution.

I don't  introduce `FullOptions::load_from_file` here, because `FullOptions::load_from_file` is also a legacy dirty patch, and users don't want to see all the configuration items of rocksdb in front of them and have no idea what they can do, so for now we'll just provide the simplest and most useful options.

### Related changes

- PR to update `owner/repo`:
- Need to cherry-pick to the release branch

### Check List <!--REMOVE the items that are not applicable-->

Tests <!-- At least one of them must be included. -->

- Unit test
- Integration test


### Release note <!-- Choose from None, Title Only and Note. Bugfixes or new features need a release note. -->

```release-note
Title Only: Include only the PR title in the release note.
```



Co-authored-by: zhangsoledad <[email protected]>
@chenyukang
Copy link
Collaborator

Closing it since #3735 is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
t:bug Type: This doesn't seem right.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@chenyukang @driftluo @gavriilos