WSL2 date incorrect after waking from sleep

[garyo]

• Your Windows build number (run ver at a Windows Command Prompt), your distribution (f.e. on Debian/Ubuntu run lsb_release -r), and whether the issue manifests on WSL 2 and/or WSL 1 (cat /proc/version):

Windows: version 2004, build 19041.264
Distro: Ubuntu 18.04
Issue manifests only on WSL2 (Linux version 4.19.84-microsoft-standard)

• What you're doing and what's happening. Copy&paste the full set of specific command-line steps necessary to reproduce the behavior, and their output. Include screenshots if that helps demonstrate the problem:

I upgraded an Ubuntu 18.04 WSL1 instance to WSL2. Ever since then, the date does not advance when the machine is asleep or hibernating. To repro:

• Install Ubuntu 18.04
• Check that Windows system date is correct
• Check that date is correct (use date command in WSL2)
• Put laptop to sleep for a while
• Resume from sleep
• Check Windows date: correct
• Check WSL2 date: wrong, still where it was when machine went to sleep
• Wait a while to see if it auto-corrects
• Notice that it doesn't

What's wrong / what should be happening instead:

• Date does not update after sleep. It should be correct after resume, as it was in WSL1.

Note that there was a previous issue on this topic, #4245 which is now closed, I'm not sure why; it was recommended there by @therealkenc that I open a new issue.

There are temporary workarounds; if I use sudo hwclock -s or shut down the WSL2 VM with wsl --shutdown and restart, the time is correct, until the next sleep/hibernate.

As noted in various other issues on this topic (#5184, #4975, #4677 etc.), the time being incorrect is more than a minor inconvenience. It can cause trouble with apt install, validity of SSL certs, build problems, and many other issues.

[benhillis]

@garyo - Thanks for opening a new issue. We made a previous change to send clock sync packets when the system resumes from sleep, but it looks like we are not receiving those notifications in some cases. I have an ongoing conversation with the Windows Power team, it looks like they are not correctly delivering the notifications upon system resume in some cases.

[dwaynebradley]

@garyo @benhillis I just tried this again on my laptop (original Surface Book) and the date/time seems to be keeping up-to-date after wake up from sleep in my setup now.

Windows: version 2004, build 19041.329
Distro: Ubuntu 20.04
Linux kernel version: 4.19.104-microsoft-standard

I put my laptop to sleep for about a minute or so, woke it back up and checked the date/time again in WSL and it seems to be updated now. Probably need to try on a longer sleep timeframe just to double-check though. I'll try this again later and report back when I have a longer period to put it to sleep.

[dwaynebradley]

Follow up on my previous comment...

I put my laptop to sleep last night. When I opened it up this morning, the first thing I did was to check the time in WSL (Ubuntu 20.04). Low and behold...it was correct!!! :-)

For my setup at least, this appears to be fixed now.

[garyo]

@dwaynebradley are you saying it's fixed by something Microsoft did, or did you install one of the workarounds such as #4245 (comment)?

[dwaynebradley]

@garyo fixed by Microsoft apparently. No workarounds on my end.

[dwaynebradley]

Well, I just tried it again today after leaving my laptop in sleep overnight and the date is way off again. Apparently it isn't fixed after all...

[MauriceS-gti]

Second this. Not fixed. Windows 10/2004 build 19041.329

[Balothar12]

I'm assuming this is the same problem, but neither wsl --shutdown nor hwclock fixes it for me, so I'm not completely sure. Whenever I try to build anything (configured with cmake, built with make), I get the clock skew warning, albeit with very small times (usually below 2 seconds and quite often below 1). Interestingly it does not appear that any builds have actually failed due to this yet, things seem to be complete even after the warning appeared. I also haven't had any issues with apt yet that relate to this.

Windows: Version 2004, build 19041.329
Distro: Ubuntu 20.04
Linux Kernel: 4.19.104-microsoft-standard

[onomatopellan]

@Balothar12 Are your projects on /mnt/* folders? See #4975

[Balothar12]

@onomatopellan Indeed they are. Didn't realize that at first since I symlinked the respective directory from my home directory and kinda forgot abut that, but yeah, the files are technically in the mounted folders. And as mentioned in #4975 the issue does not occur when the same files are actually on the Linux filesystem.

[garyo]

Everyone's saying that with WSL2 you want to move your project folders into the Linux side -- opposite of WSL1. Filesystem performance accessing Windows files from WSL2 is abysmal. (But this is getting off topic.)

[MysticRyuujin]

I'm not sure if I have the same issue but I have the same problem...

My workstation NEVER sleeps, but still, somehow my WSL2 Ubuntu host became 20 seconds behind...

sudo hwclock -s didn't do anything, but wsl --shutdown did cause the clocks to re-sync.

Update: Checked the time again, and it's drifted once again. It's now 2 seconds behind. No reboots or sleep since the screenshot above

[eajfpeters]

@Balothar12 and @onomatopellan I found the same issue with files on /mnt/* and using make. sudo hwclock -s does not help. After executing this command I still get warnings like
make[2]: Warning: File 'CMakeFiles/yaml-cpp.dir/depend.make' has modification time 0.6 s in the future
and
make[2]: warning: Clock skew detected. Your build may be incomplete.

The warnings are not in vain. Later I found that in fact the library I had build was incomplete and I got issues using it when trying to link it to my own code. When I moved all to the Linux side and rebuild the library it was complete. I also find that i/o of files located in /mnt/* is very slow. Probably this clock warning due to sub-second lags and slow /mnt/* file handling are related.

[wsy]

@garyo - Thanks for opening a new issue. We made a previous change to send clock sync packets when the system resumes from sleep, but it looks like we are not receiving those notifications in some cases. I have an ongoing conversation with the Windows Power team, it looks like they are not correctly delivering the notifications upon system resume in some cases.

Hi @benhillis , I'd like to remind you of another scenario about clock sync packet and Windows Power state.
That is NESTED VIRTUALIZATION.
I got a physical computer running Hyper-V. I created a virtual machine with nested virtualization enabled and I installed Windows 10 2004 on that virtual machine. Inside Windows 10, I enabled Hyper-V and enabled WSL2.
Now my Ubuntu 20.04 is 50 minutes behind my Windows 10 clock. This is possibly because the virtual machine got paused when I restarted my host machine.
I guess in this scenario, Windows 10 would never have received any event notification because there's nothing a guest OS needed to do. Thus Windows 10 would never have notified WSL2 kernel of something that's "never happened".

[realtica]

@Balothar12 and @onomatopellan I found the same issue with files on /mnt/* and using make. sudo hwclock -s does not help. After executing this command I still get warnings like
make[2]: Warning: File 'CMakeFiles/yaml-cpp.dir/depend.make' has modification time 0.6 s in the future
and
make[2]: warning: Clock skew detected. Your build may be incomplete.

The warnings are not in vain. Later I found that in fact the library I had build was incomplete and I got issues using it when trying to link it to my own code. When I moved all to the Linux side and rebuild the library it was complete. I also find that i/o of files located in /mnt/* is very slow. Probably this clock warning due to sub-second lags and slow /mnt/* file handling are related.

Yes, you are totally right the error only happens inside /mnt/* folder, I tried with hwclock and shutdown etc.. but none works that is the reason why the difference is only by seconds or milliseconds. Thanks.

[relston]

Seeing the same issue here

Microsoft Windows [Version 10.0.20197.1000]
% lsb_release -r
Release:        9.5
at /proc/version
Linux version 4.19.121-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Fri Jun 19 21:06:10 UTC 2020

Surface laptop usually sleeps, I have seen the Linux clock more than a day behind the windows clock

[stuartleeks]

I posted this on another issue, but adding here in case it helps anyone.

I created a workaround for this issue using Windows Events to trigger the clock sync via hwclock on resume from sleep. Originally a PowerShell script, I've recently updated it to a go binary (partly because that avoids the flash of a PowerShell window). Since implementing this workaround, I've not had any issues: https://github.com/stuartleeks/wsl-clock

[nate2014jatc]

Still present in W10 2004 (Build 19041.508).
This was a fun rabbit-hole to dive down.

[muchtall]

I like the PowerShell/Go workaround above, but I wanted something running natively in the WSL2 VM. This is what I cooked up. It polls the hardware clock every minute, and if the clock has drifted more than 5 seconds, it syncs the VM to the hardware clock: https://github.com/muchtall/muchtall-scripts/blob/master/wsl2-time-sync.txt

Not as quick as triggering on wake from hibernate, but it's close enough for my needs.

EDIT: Nevermind. I got into work today, and noticed that it's not working. Not sure why it worked when I was developing the solution, but it's not working for me now. 🤷‍♂️

[oising]

Can confirm fully patched 2004 still has this issue. This is particularly annoying if you're using Docker with WSL2 backend, as this drift is inherited by all containers.

[r0gi]

I have the same issue on W10 2004 discovered when my git commits were days out of date. sudo hwclock -s works ok, but it sets my WSL time to ~60 seconds in the future. The Windows taskbar clock display is always correct when comparing with online sources.

Using ntpdate instead and running sudo ntpdate time.windows.com makes it accurate.

It doesn't fix this issue permanently, and you have to manually run the command if the drift becomes too high again. You can try my workaround below or stuartleeks' workaround above.

[mattclegg]

sudo ntpdate time.windows.com fixed the issue and then 12 hours later its out of sync by 5 minutes again?!

[r0gi]

Sorry, I should have clarified that running the ntpdate command just gave my WSL a more accurate time than the hwclock command. It doesn't fix this issue permanently, and you have to manually run the command if the drift becomes too high again.

A possible alternative workaround to stuartleeks' option (works for me on Ubuntu 18.04):

1. Add the following to your ~/.bashrc:

sync_time(){
    if sudo echo Starting time sync in background
    then
        sudo nohup watch -n 10 ntpdate time.windows.com > /dev/null 2>&1 &
    fi
}

2. Source your bashrc: . ~/.bashrc
3. Run sync_time once and it should rerun the command every 10 seconds in the background indefinitely, without stopping (even after closing your WSL window). You can check this is running with the command jobs, or if you check WSL's processes with top -d 1 you can see ntpdate being run automatically every 10s. I haven't tested this thoroughly but my WSL is keeping accurate time even after sleep/hibernate.

You do have to rerun this every time you restart your computer. Maybe someone else can figure out how to make this fully automatic.

[raydnichols]

I think this WSL clock problem must stem from Windows needing to aggressively sleep to save power. I wonder if we could just have an option to not do that? I notice, compared to my Ubuntu desktop machine on the same network, that if I lock my computer: my WSL clock will drift, but also no Outlook mails will be retrieved (have to wait for a few minutes after unlock), and any SSH sessions I have are lost.
My morning routine every day for the last few months is:

• Try: sudo hwclock -s
• Test the time: date
• That mostly works, but if it doesn't, close all WSL windows and Visual Code editors that use it, then in a Windows Command prompt: wsl.exe --shutdown
• Then log back in to everything
  During the day I have to run my computer with no power saving options to blank the screen. At night I can let it power off the screen to do the whole thing again the next day. I need an accurate clock for git commits and AWS CLI authentication.

[jblang]

I have enabled systemd and timesyncd and I haven't had a problem since. I don't have to run any commands to manually update the clock or futz with power settings. Enabling systemd does have the downside that WSL takes considerably longer to start the first time, but it's a small price to pay.

• To do this, first add this to /etc/wsl.conf:

  [boot]
  systemd=true
  

• Then, in /lib/systemd/system/systemd-timesyncd.service, comment out this line:

  #ConditionVirtualization=!container
  

• Now run wsl --shutdown from a PowerShell tab, and then start a new Ubuntu tab.
• Once it starts up run systemctl status systemd-timesyncd and make sure the status is Active (running)
• Hopefully, your clock now stays in sync.

Even though I have a workaround it's still frustrating that so many people have this issue and nobody from Microsoft has responded...

@craigloewen-msft can you please help here? Is someone looking into this? Thirty people have reported this just since the beginning of the year. If there is another open issue where progress is being made, I haven't found it.

[asampal]

Maybe they don't monitor closed issues so opening a new one might be the way to address this.

[AnimeshRy]

The issue still persists, Can we open this issue again ? @benhillis. Have been manually updating time for the past week

[digoo]

Funny thing. I have just experienced this today. Also the previous "fix" (sudo hwclock -s) also worked.
This impacted my identify server identification, so I kept getting until the fix above:
unable to parse JWT token: Token is not valid yet.

[jblang]

Just a heads up, the previous workaround that I documented stopped working for me this week. Something, I suspect an update to WSL2, reset the systemd-timesyncd.service file to its original configuration, so I had to comment out the ConditionVirtualization=!container line again. After that it works as before...

[raydnichols]

Thanks @jblang for your previous notes. I have been using that. It's probably that we need to edit systemd formally using an approach like:
https://docs.fedoraproject.org/en-US/quick-docs/understanding-and-administering-systemd/#modifying-existing-systemd-services

I have found timesyncd working ... yet still ... sometimes I have to wait for up to 20 minutes for it to actually re-sync first thing on the morning.

[matthewnourse]

This bug has been driving me nuts so I wrote a little daemon to work around it:

https://github.com/matthewnourse/polite-hwclock-hctosys

...and it's been working great for me. Feedback/bug reports/feature requests welcome.

[esumii]

To remind: as far as this WSL bug (delay of time) is concerned, I do not see any problem in automatically running hwclock every time the host Windows resumes.

#5324 (comment)

[matthewnourse]

To remind: as far as this WSL bug (delay of time) is concerned, I do not see any problem in automatically running hwclock every time the host Windows resumes.

#5324 (comment)

Cool, agreed re hwclock & delay of time on resume, in that case polite-hwclock-hctosys acts the same as hwclock anyway. However on my system I noticed that sometimes the system clock also wobbles ahead of the hardware clock, rarely more than a second, but enough to trouble me...in that case I would rather a gradual-adjustment solution. Also some of my clock drifts happen when the system is just running normally and there's been no resume. In that case the drift is only 1-2 seconds, but still, it irks me. In any case, if a solution is working for someone else then far be it from me to suggest that they change :-).

[juhap]

I've been running "hwclock -s" for a while with no problems. Now I noticed the hardware clock is also wrong for some reason (windows time is right):

PS> date; wsl date; wsl sudo hwclock

Sunday, March 19, 2023 5:02:19 PM
Sun Mar 19 14:10:47 EET 2023
2023-03-19 14:10:47.906726+02:00

(Windows 11, 22H2, Build 22621.1413, kernel 5.15.90.1-microsoft-standard-WSL2)

[duaneking]

An Absolutely Terrible Solution that still works better than anything presented so far:

In your .bashrc and (and hourly via crontab because it still cant be trusted):

sudo ntpdate time.windows.com && sudo hwclock -s

[esumii]

Unfortunately, it does not work unless the .bashrc or the hourly cron runs.

[ke1ewang]

Same error.

[duaneking]

It really worries me that Microsoft has apparently decided that this critical security issue is a low priority, given the amount of money that Microsoft customers have spent and invested on WSL based solutions to integrate windows with their Linux platforms.

Would anybody from a cybersecurity company be willing to help create a CVE about this issue? Because it does create security issues when TLS wont connect due to time differences, and Time is as a critical security component... so if it's not working then you can't trust the regulatory and security compliance of the systems involved.

This is creating auditing nightmares in the real world; I can't go into detail but I can say that it's costing companies a lot of money and it's creating a lot of risk for a lot of people. Fix this asap please.

[dimo414]

You might want to weigh in on #8204 which is still open. Unfortunately there's more than a dozen related bugs, seemingly because this issue has been fixed and regressed multiple times, which makes tracking down the current status very hard.

[levrik]

A new megathread to track this issue has been opened. MS employees have linked it from all other open issues but not from this already closed one. So I thought I'll add this here in case someone stumbles upon this issue.

#10006

[kumkee]

The following comment by @jblang is easily the best one.
By the way, can we achieve # ConditionVirtualization=!container by editing the less invasive /etc/systemd/timesyncd.conf?

I have enabled systemd and timesyncd and I haven't had a problem since. I don't have to run any commands to manually update the clock or futz with power settings. Enabling systemd does have the downside that WSL takes considerably longer to start the first time, but it's a small price to pay.

• To do this, first add this to /etc/wsl.conf:

  [boot]
  systemd=true
  

• Then, in /lib/systemd/system/systemd-timesyncd.service, comment out this line:

  #ConditionVirtualization=!container
  

• Now run wsl --shutdown from a PowerShell tab, and then start a new Ubuntu tab.
• Once it starts up run systemctl status systemd-timesyncd and make sure the status is Active (running)
• Hopefully, your clock now stays in sync.

Originally posted by @jblang in #5324 (comment)

[gnantel]

By the way, can we achieve # ConditionVirtualization=!container by editing the less invasive /etc/systemd/timesyncd.conf?

sudo systemctl edit systemd-timesyncd with the following contents:

[Unit]
ConditionVirtualization=
ConditionVirtualization=wsl

References: https://unix.stackexchange.com/a/737366/272577 and #8204 (comment)

[kumkee]

By the way, can we achieve # ConditionVirtualization=!container by editing the less invasive /etc/systemd/timesyncd.conf?

sudo systemctl edit systemd-timesyncd with the following contents:

[Unit]
ConditionVirtualization=
ConditionVirtualization=wsl

References: https://unix.stackexchange.com/a/737366/272577 and #8204 (comment)

Thank you. That's exactly what I am after.

[webstean]

Still an issue, it's a bug!!!!!

[lindhe]

@webstean Please see #10006

[silvio2402]

I tried the listed methods to fix the time not being synced, but none of them worked.
After doing sudo ntpdate time.windows.com (you have to install ntpdate first) and then sudo service hwclock.sh reload the clock was synced successfully.

[kumkee]

For those who are still struggling in October 2023, doing both of the following will solve the problem and save you from manually syncing every time.

1. Set up Windows Task Scheduler - schtasks
2. Set up systemd-timesyncd

But I agree it is still a bug.

Update in March 2024: the kernel update mentioned in @masud-abdulkadir's reply 2 posts below does seem to fix the bug. Tested on WSL 2.1.4.0 with Kernel 5.15.146.1-2.

[lzcmian]

same bug

[masud-abdulkadir]

Newer people, it's finally being fixed here in #10006 currently in pre-release so you can update to that or keep your workarounds until its in release and then update, i mean we've waited near 4 years, can wait a little longer eh? :p Here is the link to the comment, and here is the patch itself if you're curious.