Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch load multi read #17

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
58b8b68
Refactor session loading to manage pending files so we can batch load…
soulomoon Nov 2, 2024
ea002d7
distribute errors to all pending files are being loading
soulomoon Nov 3, 2024
c78b197
better filter loading files
soulomoon Nov 3, 2024
b879375
fallback to non-batch load
soulomoon Nov 4, 2024
8953aec
typo
soulomoon Nov 4, 2024
4bdc2c8
update inline doc
soulomoon Nov 4, 2024
c4bb53a
update inline doc
soulomoon Nov 4, 2024
112bc95
add LogSessionReloadOnError to log errors during file reloads; cleanu…
soulomoon Nov 4, 2024
9a2aaf1
Merge branch 'master' into batch-load
soulomoon Nov 4, 2024
6e04d28
refactor loadSessionWithOptions to improve error handling and clarify…
soulomoon Nov 4, 2024
67aebc4
refactor loadSessionWithOptions to improve pending file handling and …
soulomoon Nov 4, 2024
98ae446
add doc about limitation
soulomoon Nov 4, 2024
f3eb580
absolute file at the beginning
soulomoon Nov 5, 2024
e7bd3d4
run session loader and worker in sperate
soulomoon Nov 7, 2024
1f97c40
cleanup
soulomoon Nov 7, 2024
4c998bd
rename LogGetSessionRetry to LogLookupSessionCache for clarity in log…
soulomoon Nov 8, 2024
79a43a0
extract attempt to load files from errors
soulomoon Nov 9, 2024
beb1764
refactor session loading to wait for pending files before cache check
soulomoon Nov 12, 2024
6139522
add LogTime to logging for improved time tracking during session loading
soulomoon Nov 18, 2024
7314509
refactor session loading to handle dependency checks more clearly
soulomoon Nov 19, 2024
cddcc55
Refactors session loading logic
soulomoon Nov 19, 2024
c23909d
ci(mergify): upgrade configuration to current format (#4454)
mergify[bot] Nov 24, 2024
fea0135
More tests and better docs for cabal-add (#4455)
VenInf Nov 25, 2024
b8127f7
Update python read-the-docs dependencies to latest (#4457)
fendor Dec 1, 2024
25c5d82
Allow building with GHC 9.8.4 (#4459)
fendor Dec 4, 2024
4c99c47
Merge branch 'master' into batch-load-multi-read
soulomoon Dec 6, 2024
bb78a36
delay the restart
soulomoon Dec 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 9 additions & 16 deletions .github/mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,19 @@ queue_rules:
- name: default
# Mergify always respects the branch protection settings
# so we can left empty mergify own ones
conditions: []

pull_request_rules:
- name: Automatically merge pull requests
conditions:
queue_conditions:
- label=merge me
- '#approved-reviews-by>=1'
actions:
queue:
method: squash
name: default
# The queue action automatically updates PRs that
# have entered the queue, but in order to do that
# they must have passed CI. Since our CI is a bit
# flaky, PRs can fail to get in, which then means
# they don't get updated, which is extra annoying.
# This just adds the updating as an independent
# step.
merge_conditions: []
merge_method: squash

pull_request_rules:
- name: Automatically update pull requests
conditions:
- label=merge me
actions:
update:
- name: refactored queue action rule
conditions: []
actions:
queue:
17 changes: 16 additions & 1 deletion cabal.project
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ packages:
./hls-test-utils


index-state: 2024-11-02T00:00:00Z
index-state: 2024-12-02T00:00:00Z

tests: True
test-show-details: direct
Expand Down Expand Up @@ -46,3 +46,18 @@ constraints:
if impl(ghc >= 9.9)
-- https://github.com/haskell/haskell-language-server/issues/4324
benchmarks: False

if impl(ghc >= 9.8.4) && impl(ghc < 9.8.5)
-- By depending on ghc-lib-parser and ghc, we are encountering
-- a constraint conflict, ghc-9.8.4 comes with `filepath-1.4.301.0`,
-- and `ghc-lib-parser-9.8.4.20241130` specifies `filepath >=1.5 && < 1.6.
-- See https://github.com/digital-asset/ghc-lib/issues/572 for details.
allow-older:
ghc-lib-parser:filepath
constraints:
ghc-lib-parser==9.8.4.20241130

source-repository-package
type: git
location: https://github.com/soulomoon/hie-bios.git
tag: 93582c21372af573e5103bad198777a3317a2df2
8 changes: 4 additions & 4 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Sphinx~=5.3.0
sphinx-rtd-theme~=1.1.0
myst-parser~=1.0.0
docutils<0.19
Sphinx~=8.1.3
sphinx-rtd-theme~=3.0.2
myst-parser~=4.0.0
docutils~=0.21.2
2 changes: 2 additions & 0 deletions ghcide/ghcide.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ library
, unliftio-core
, unordered-containers >=0.2.10.0
, vector
, ListT

if os(windows)
build-depends: Win32
Expand Down Expand Up @@ -204,6 +205,7 @@ library
Development.IDE.GHC.CPP
Development.IDE.GHC.Warnings
Development.IDE.Types.Action
Development.IDE.Session.OrderedSet

if flag(pedantic)
ghc-options:
Expand Down
254 changes: 188 additions & 66 deletions ghcide/session-loader/Development/IDE/Session.hs

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ data CradleErrorDetails =
Depicts the cradle error in a user-friendly way.
-}
renderCradleError :: CradleError -> Cradle a -> NormalizedFilePath -> FileDiagnostic
renderCradleError (CradleError deps _ec ms) cradle nfp
renderCradleError (CradleError deps _ec ms _attemptToLoadFiles) cradle nfp
| HieBios.isCabalCradle cradle =
let (fp, showDiag, diag) = ideErrorWithSource (Just "cradle") (Just DiagnosticSeverity_Error) nfp $ T.unlines $ map T.pack userFriendlyMessage in
(fp, showDiag, diag{_data_ = Just $ Aeson.toJSON CradleErrorDetails{cabalProjectFiles=absDeps}})
Expand Down
40 changes: 40 additions & 0 deletions ghcide/session-loader/Development/IDE/Session/OrderedSet.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
module Development.IDE.Session.OrderedSet where

import Control.Concurrent.STM (STM, TQueue, newTQueueIO)
import Control.Concurrent.STM.TQueue (readTQueue, writeTQueue)
import Control.Monad (when)
import Data.Hashable (Hashable)
import qualified Focus
import qualified ListT as LT
import qualified StmContainers.Set as S
import StmContainers.Set (Set)


type OrderedSet a = (TQueue a, Set a)

insert :: Hashable a => a -> OrderedSet a -> STM ()
insert a (que, s) = do
(_, inserted) <- S.focus (Focus.testingIfInserts $ Focus.insert ()) a s
when inserted $ writeTQueue que a

newIO :: Hashable a => IO (OrderedSet a)
newIO = do
que <- newTQueueIO
s <- S.newIO
return (que, s)

readQueue :: Hashable a => OrderedSet a -> STM a
readQueue rs@(que, s) = do
f <- readTQueue que
b <- S.lookup f s
-- retry if the file is already in done
if b then return f else readQueue rs

lookup :: Hashable a => a -> OrderedSet a -> STM Bool
lookup a (_, s) = S.lookup a s

delete :: Hashable a => a -> OrderedSet a -> STM ()
delete a (_, s) = S.delete a s

toUnOrderedList :: Hashable a => OrderedSet a -> STM [a]
toUnOrderedList (_, s) = LT.toList $ S.listT s
9 changes: 4 additions & 5 deletions plugins/hls-cabal-plugin/src/Ide/Plugin/Cabal/CabalAdd.hs
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,8 @@ addDependencySuggestCodeAction plId verTxtDocId suggestions haskellFilePath caba
-- | Gives the build targets that are used in the `CabalAdd`.
-- Note the unorthodox usage of `readBuildTargets`:
-- If the relative path to the haskell file is provided,
-- the `readBuildTargets` will return a main build target.
-- This behaviour is acceptable for now, but changing to a way of getting
-- all build targets in a file is advised.
-- the `readBuildTargets` will return build targets, where this
-- module is mentioned (in exposed-modules or other-modules).
getBuildTargets :: GenericPackageDescription -> FilePath -> FilePath -> IO [BuildTarget]
getBuildTargets gpd cabalFilePath haskellFilePath = do
let haskellFileRelativePath = makeRelative (dropFileName cabalFilePath) haskellFilePath
Expand All @@ -167,10 +166,10 @@ addDependencySuggestCodeAction plId verTxtDocId suggestions haskellFilePath caba
mkCodeAction :: FilePath -> Maybe String -> (T.Text, T.Text) -> CodeAction
mkCodeAction cabalFilePath target (suggestedDep, suggestedVersion) =
let
versionTitle = if T.null suggestedVersion then T.empty else " version " <> suggestedVersion
versionTitle = if T.null suggestedVersion then T.empty else "-" <> suggestedVersion
targetTitle = case target of
Nothing -> T.empty
Just t -> " target " <> T.pack t
Just t -> " at " <> T.pack t
title = "Add dependency " <> suggestedDep <> versionTitle <> targetTitle
version = if T.null suggestedVersion then Nothing else Just suggestedVersion

Expand Down
20 changes: 17 additions & 3 deletions plugins/hls-cabal-plugin/test/CabalAdd.hs
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,30 @@ cabalAddTests :: TestTree
cabalAddTests =
testGroup
"CabalAdd Tests"
[ runHaskellTestCaseSession "Code Actions - Can add hidden package" ("cabal-add-testdata" </> "cabal-add-exe")
[ runHaskellTestCaseSession "Code Actions - Can add hidden package to an executable" ("cabal-add-testdata" </> "cabal-add-exe")
(generateAddDependencyTestSession "cabal-add-exe.cabal" ("src" </> "Main.hs") "split" [253])
, runHaskellTestCaseSession "Code Actions - Guard against HPack" ("cabal-add-testdata" </> "cabal-add-packageYaml")
(generatePackageYAMLTestSession ("src" </> "Main.hs"))
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a library" ("cabal-add-testdata" </> "cabal-add-lib")
(generateAddDependencyTestSession "cabal-add-lib.cabal" ("src" </> "MyLib.hs") "split" [348])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a test" ("cabal-add-testdata" </> "cabal-add-tests")
(generateAddDependencyTestSession "cabal-add-tests.cabal" ("test" </> "Main.hs") "split" [478])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a benchmark" ("cabal-add-testdata" </> "cabal-add-bench")
(generateAddDependencyTestSession "cabal-add-bench.cabal" ("bench" </> "Main.hs") "split" [403])

, runHaskellTestCaseSession "Code Actions - Can add hidden package to an executable, multiple targets" ("cabal-add-testdata" </> "cabal-add-multitarget")
(generateAddDependencyTestSession "cabal-add-multitarget.cabal" ("src" </> "Main.hs") "split" [269])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a library, multiple targets" ("cabal-add-testdata" </> "cabal-add-multitarget")
(generateAddDependencyTestSession "cabal-add-multitarget.cabal" ("lib" </> "MyLib.hs") "split" [413])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to an internal library, multiple targets" ("cabal-add-testdata" </> "cabal-add-multitarget")
(generateAddDependencyTestSession "cabal-add-multitarget.cabal" ("lib" </> "InternalLib.hs") "split" [413])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a test, multiple targets" ("cabal-add-testdata" </> "cabal-add-multitarget")
(generateAddDependencyTestSession "cabal-add-multitarget.cabal" ("test" </> "Main.hs") "split" [655])
, runHaskellTestCaseSession "Code Actions - Can add hidden package to a benchmark, multiple targets" ("cabal-add-testdata" </> "cabal-add-multitarget")
(generateAddDependencyTestSession "cabal-add-multitarget.cabal" ("bench" </> "Main.hs") "split" [776])


, runHaskellTestCaseSession "Code Actions - Guard against HPack" ("cabal-add-testdata" </> "cabal-add-packageYaml")
(generatePackageYAMLTestSession ("src" </> "Main.hs"))

, testHiddenPackageSuggestions "Check CabalAdd's parser, no version"
[ "It is a member of the hidden package 'base'"
, "It is a member of the hidden package 'Blammo-wai'"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
module Main (main) where

import Data.List.Split

main :: IO ()
main = putStrLn "Test suite not yet implemented."
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
cabal-version: 2.4
name: cabal-add-multitarget
version: 0.1.0.0
build-type: Simple

executable cabal-add-exe
main-is: Main.hs
hs-source-dirs: src
ghc-options: -Wall
build-depends: base
default-language: Haskell2010

library
exposed-modules: MyLib
other-modules: InternalLib
build-depends: base >= 4 && < 5
hs-source-dirs: lib
ghc-options: -Wall

test-suite cabal-add-tests-test
main-is: Main.hs
hs-source-dirs: test
type: exitcode-stdio-1.0
build-depends: base
default-language: Haskell2010

benchmark benchmark
main-is: Main.hs
build-depends: base
hs-source-dirs: bench
type: exitcode-stdio-1.0
ghc-options: -threaded

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
module InternalLib (internalFunc) where

import Data.List.Split

internalFunc :: IO ()
internalFunc = putStrLn "internalFunc"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
module MyLib (someFunc) where

import Data.List.Split

someFunc :: IO ()
someFunc = putStrLn "someFunc"
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module Main where

import Data.List.Split

main = putStrLn "Hello, Haskell!"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
module Main (main) where

import Data.List.Split

main :: IO ()
main = putStrLn "Test suite not yet implemented."
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ packages: cabal-add-exe
cabal-add-lib
cabal-add-tests
cabal-add-bench
cabal-add-multitarget
cabal-add-packageYaml
Loading