Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when using LoadFromFile multiple times #55

Open
Link512 opened this issue Jul 6, 2021 · 0 comments
Open

Memory leak when using LoadFromFile multiple times #55

Link512 opened this issue Jul 6, 2021 · 0 comments

Comments

@Link512
Copy link

Link512 commented Jul 6, 2021

I have a service that every N minutes loads a fastcache from file. Every time this happens, the program leaks memory. This was easily observable by simply loading a cache multiple times in the same program and then using htop/ActivityMonitor to see this steady increase of memory. In my particular use case, the cache is rather large (~18M keys, 662MB on disk).

The snippet to reproduce this is straight-forward:

package main

import (
	"fmt"
	"os"
	"os/signal"
	"runtime"
	"syscall"
	"time"

	"github.com/VictoriaMetrics/fastcache"
)

func main() {
	cachePath := "path_to_cache"

	tick := time.NewTicker(15 * time.Second)
	sig := make(chan os.Signal, 2)
	signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
	fmt.Println(os.Getpid())
	for {
		select {
		case <-tick.C:
			newCache, err := fastcache.LoadFromFile(cachePath)
			if err != nil {
				panic(err)
			}
			_ = newCache
			runtime.GC()
			fmt.Println("reloading cache done")
		case <-sig:
			tick.Stop()
			return
		}
	}
}

After some profiling and debugging I've found that the use of getChunk() with memmap is the culprit. When we reload the cache every time, we never call putChunk, only getChunk, which causes the library to continuosly allocate memory via memmap which is never munmaped. To verify this, I added a panic to putChunk, which was never triggered.

I think perhaps one solution is to add LoadFromFile as a method also to the Cache object, so that the chunks from the old buckets can be reused when loading a new cache. I might be able to help with a PR if time permits, but I thought in the meantime you should be aware of this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant