Migrate to OsPath

[kokobd]

We have a better alternative to FilePath now. See: https://hasufell.github.io/posts/2022-06-29-fixing-haskell-filepaths.html

What we should do:

1. Change NormalizedFilePath to NormalizedOsPath after the issue in lsp to support OsPath Migrate to OsPath lsp#445 is solved. Profiling it to make sure the performance improves.
2. Change other usages of FilePath to OsPath

However, this can not be done until ghc-9.6 is released (ships the new filepath library), or all ghc shipped libraries are available on Hackage.

Many GHC bundled libraries do not have new versions released to Hackage (for example ghci and hpc, their versions on hackage is very old). Thus, cabal solver fails if I add filepath ^>= 1.4.100.0 to a project that depends on ghc, which then depends on ghci. The installed ghci links against filepath-1.4.2.2 in this case, and cabal can not rebuild a ghci with filepath ^>= 1.4.100.0, as the source code is not found on hackage.

[pepeiborra]

Thus, cabal solver fails if I add filepath ^>= 1.4.100.0 to a project that depends on ghc, which then depends on ghci.

Oh bummer. Well, if we really want to, it should be ok to embed OsPath and combinators in a filepath-compat package.

[hasufell]

https://gitlab.haskell.org/ghc/ghc/-/issues/21887#note_443922

It may be possible to provide updated GHC releases that include new filepath. But this is still in discussion.

[kokobd]

I don't know why, but the overall memory usage decreased from 1.5G to 700M
The data is collected by launching haskell-language-server from the command line in the HLS source repo with profiling RTS options enabled.

[hasufell]

@kokobd you mean between String and OsPath?

[kokobd]

@kokobd you mean between String and OsPath?

@hasufell Yes. But when loading HLS executable into VSCode, I no longer witness such a big difference. I'm still investigating the reason.

[pepeiborra]

Can you run cabal bench ghcide and share the contents of the ghcide/bench-results folder minus the binaries ?

[kokobd]

cabal bench ghcide is failing with master & ghc 9.2.3. Verified by creating a fresh new Gitpod workspace from master.

[pepeiborra]

I'll take a look, but in the meantime CI shows that it works fine on 8.10.7. Would you be able to run it there?

EDIT: nevermind, I see that 8.10.7 won't do
EDIT2: #3069 should help

[pepeiborra]

Ran the edit benchmark on Cabal 3.6.3.0, results seem quite promising:

[kokobd]

Next steps:

1. Upgrade lsp and lsp-types in HLS to 1.5. It has some breaking changes.
2. Either extend filepath-compat to support older versions of GHC (currently it only supports ghc 9.2.3), or release GHC minor versions with new filepath package. Support of some old GHC might need to be dropped. See: https://gitlab.haskell.org/ghc/ghc/-/issues/21887

[kokobd]

Real OsPath will only be available since GHC 9.6. Instead, I use ShortByteString within NormalizedFilePath. The result is still very promising.

Soon, we will benefit from the performance improvement with all versions of GHC supported by HLS.

Next steps:

1. Merge use OsPath in NormalizedFilePath lsp#446 and Improve performance of NormalizedFilePath #3067. (almost done)
2. Eliminate the usage of partial functions introduced by this PR gradually.

[michaelpj]

I think this is mostly done. Was there more to do?

[hasufell]

I haven't looked at it, but I wonder how you do file operations etc. I'm guessing you convert back to filepath somewhere? In that case the conversion is not complete

[michaelpj]

It looks like the situation is:

• lsp is set up for OsPath
• HLS hasn't migrated at all, and in the interests of keeping it simple we probably won't until 9.6 is the oldest version we support, so we don't need conditional logic and can just use OsPath everywhere

[hasufell]

So in ten years, I guess?

[michaelpj]

Two major GHC versions, so yeah, a while. Or I guess if someone is keen and wants to do it with conditional logic they could do!