File Store have bad behavior

[gamelaster]

Hello,
I'm using File Store to store my sessions. I don't know why, but many times I got error the some jsons not exists. Okay, its have some problem. But bigger problem is the file store not storaging variables (mainly from passport), just it not storage. Any ideas how to trace where is problem?

[valery-barysok]

Can you describe what do you mean? can you provide basic app based on examples from https://github.com/valery-barysok/session-file-store/tree/master/examples ?

[gamelaster]

@valery-barysok
Well, example works nice. But working with passport actually not. I just log in via passport-steam and sessions and it's just not save (still im not logged + session file contains nothing). I'm sure the problem isn't in my code because it's working before. I think it's because I'm using latest LTS version of node (on my laptop where I have older version of NodeJS the passport-facebook and file store working nice without any problem).

[valery-barysok]

@gamelaster, as i said before it will be helpful if you can create basic app with your issue. You have to understand that just said "I'm sure the problem isn't in my code because it's working before." is not working :)

[gamelaster]

In example you must insert an Steam API key, you can generate it here:
http://steamcommunity.com/dev/apikey (WEB not must be a real)
App:
https://marekkraus.sk/steam/app.rar

[gamelaster]

I tried v4.0.0 , v4.4.1 and still not work. But anyway, funny thing is in Linux (node v4.1.1) it's works lol

[valery-barysok]

i am busy now and still can not dive into this issue. So i will see as soon as can.

[patrickjane]

I am also using passport, but I am not actually using their functionality to serialize/deserialize user information. Instead, after successful login, I look up the user in my local database, and attach information by hand to the req.session object. As far as I can see, this gets successfully stored to the session-store and will be re-attached to all future requests.

However, I am also seeing a ton of:

 [session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'sessions/633CV4WiL1F_8jKar1zQQ-2igGUmi8vo.json'
 [session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'sessions/633CV4WiL1F_8jKar1zQQ-2igGUmi8vo.json'
 [session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'sessions/633CV4WiL1F_8jKar1zQQ-2igGUmi8vo.json'
 [session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'sessions/633CV4WiL1F_8jKar1zQQ-2igGUmi8vo.json'

when a) logging out explicitely and calling req.logout() as well as req.session.destroy(), and b) as it seems in cyclic intervals when session-store is checking outdated sessions.
Could this be fixed? This is pretty annoying in a productive environment, as it seems to be not an actual error, since everything works fine for me.

[gamelaster]

Try it on Linux

[patrickjane]

Im on RHEL5 / MacOS.

[gamelaster]

On Linux and OS X this problem is not happend, only Windows

[rukshn]

I had a similar error in Ubuntu, which I solved by following,

store : new Filestore({path : './sessions/'})

instead of ,

store: new Filestore

[leehankyeol]

Same thing here. It seems that the plugin basically tries to load a session file even when it doesn't have to, like when saveUninitialized: false.

[valery-barysok]

thanks. will look on it closely after the new year )

[leehankyeol]

@patrickjane Maybe you can just stop logging them if you're sure that's the only problem regarding this plugin by setting logFn option to just empty function.

[tswaters]

I'm not sure if this is related or not, if I visit a logout route, then immediately after logging in I would get ENOENT errors.... My code that logged out looked like this:

req.session.destroy()
res.json({})

And after logging in, something like this:

req.session.token = 'some-token-id'
res.json({})

This is the only code that modifies the session.

All I had to do was add the following to logout:

res.clearCookie('connect.sid')

And the problem went away. This is with the following configuration:

app.use(session({
  store: new Filestore({
    path: `${process.env.DATA_PATH}/session`
  }),
  secret: process.env.SESSION_SECRET,
  resave: false,
  saveUninitialized: false
}))

Also, this is under windows 10.

@valery-barysok

If you wanted a repro, just use one of the express example, set saveUninitialized: false and add a /reset route that destroys the session. After visiting reset, the first time visiting the index route should show this.

It seems that the store will be called with destroyed sids if the client still passes along the cookie for a destroyed session - without the clearCookie this will happen until either a new session is created or the cookie expires.... I'm not entire sure what file-store can do in this case, maybe distinguish between ENOENT (don't retry/log) and other FS errors (retry and log)?

I've looked through some of the other session stores and if they don't find anything for the provided sid, they silently invoke the callback function with null.

[valery-barysok]

@tswaters, please start read from here: #26 (comment). As I remember it is your use case.

Basically i can not distinct when ENOENT happens as result of logout and when deletes through retry logic or during renaming of file (by the way, i am going to remove retry logic because file renaming was introduced).

[valery-barysok]

@rukshn, can you provide reproducible example or just test with

store : new Filestore({path : './sessions'})

without / as in your worked case

[valery-barysok]

@patrickjane, can you look at #41 (comment).

[valery-barysok]

@patrickjane and @tswaters, what if i add emit of error event with some error code that you can ignore instead of log something?

[asotog]

@gamelaster you said on windows did not work but it does on linux ? can we see your configuration ?

[emilioplatzer]

I have the same problem in window. Not in linux

[gamelaster]

@asotog https://marekkraus.sk/steam/app.rar

[ferm10n]

#justWindowsThings

I believe I have discovered that the issue is connected to the write-file-atomic. Writes to the same file asynchronously are fine on Linux but fail on Windows due to the nature of the POSIX rename.

I just recently made a PR to (hopefully) fix this and it's included in [email protected]... please give it a try and let me know if those errors are gone!

Read about this in my write-up here npm/write-file-atomic#22

It essentially serializes writes to the same file... But honestly I think to totally resolve this, session-file-store must serialize writes for the same session.

[gamelaster]

The problem still persists on Windows ☹️

[ferm10n]

@gamelaster is there any way we could try to reproduce the issue on our end?

Last time I worked on this it seemed that multiple writes would be attempted for the same session file, and then the second write would start before the first one was finished, causing the EPERM error.

From what you've described this is actually an ENOENT error, but I suspect the issues might still might be related due to the differences in how POSIX rename works in Linux vs Windows.

My strategy for figuring out the sequence of events leading to the error was to insert console.log messages when write-file-atomic begins writing to the file, and when it finishes. Looking at the output from that as session-file-store calls write-file-atomic, I pieced together the concurrency issue by looking at the sequence of events.

I suggest trying something similar and sharing your results!

[pouyan2275]

hi
im change code
app.use(session({ store:new FileStore, secret: "keyboard cat", resave: true, saveUninitialized: true, cookie: { maxAge: 1 * 1 * 30 * 60 * 1000 } }))
to
app.use(session({ store:new FileStore, secret: "keyboard cat", resave: true, saveUninitialized: true, }))
and fixed this error and i see new error
Error: EPERM: operation not permitted, rename

[stevendaye]

I am actually facing the same issue as well, and have been trying to fix the bug in my code since 5 days now.

The bug is that, I am using session-file-store to store session files, passport for local authentication. But the deserializeUser is not always called while the serializeUser is always successfully called. So upon many attempts to login, at times desesiralizeUser is called so logged in, and at times it is not called again when logged out trying to re-login . And in my log report, I see many times :
Error: EPERM: operation not permited, rename 'C:\Users\Steven....\project\session\iRTrcUr18v_bvEIej....qmd.json.27494726392' . This often happens anytime I am trying to login

Especially when I clean everything up and restart my server, the first log messages are a bunch of this:
[session-file-store] will retry, error on last attempt: Error ENOENT:no such file or directory, open 'C:\Users\Steven.....\session\VYWQ1cdVx_Bzz....RZ4I.json'
Then after when trying to log, the same EPERM error appears multiple times before I could login.

Conclusion: After pulling out my hair for over 5 days to figure out why the deserializeUser logs me in at times and at times not, I have understood that it is because it does not always find a session file to create the needed cookie to log the user in. By the way Im using windows.

PS: Please let me know if Im wrong

[theogravity]

I'm going to post here a year later and say that I have the same problem in Ubuntu docker.

[vanBrakel]

Problem still exists on Windows 10 too

[Bandit]

Getting this issue on WSL2 Ubuntu 18.04. Resolved by changing saveUninitialized to false rather than true - time will tell if that's going to have any negative impacts on my app.

[dadonc]

This answer has helped in my case.

[jusfeel]

I'm using 1.5.0.

In my case, user can open many browser tabs. If he logs out, from on tab(the session file will be gone). The other tab will cause Error: ENOENT: no such file or directory as expected. I want to quench these errors.

But after tried many ways

1. add empty logFn logFn: function(){}
2. retries: 0
3. retries: 5 ( default)
4. add custom logFn: logFn: (_) => console.log('xx')
5. tried to use saveUninitialized: true

this error message still shows.

logFn works when retries>0. But it does not stop this error being thrown no matter having retries or not.

[Tsenzuk]

Looks like @gamelaster describes two independent issues. Both of them happened in my project.

One of them is about ENOENT error when session is saved into file system, and it's separately described in the issue #106

Another one is about session object updated after authentication action and not stored. We've used cas-authentication library to auth and it's updating session asynchronously, so we needed to override the private function _handleTicket(), that checks auth for success:

_handleTicket(req, res){
  // ... placeholder code where `requestOptions` is defined
  const request = this.request_client.request(requestOptions, (response) => {
    // ... placeholder code where `body` is defined
    response.on('end', () => {
      this._validate(body, (err, user, attributes) => {
        if (err) {
          res.sendStatus(401);
        } else {
          req.session[this.session_name] = user; // session is updated

          req.session.save(() => { // wait for session to be saved before redirect to application
            res.redirect(req.session.cas_return_to);
          });
        }
      });
    });
  });
  // ... placeholder code where request.on('error', () => {}) is defined
},