Deriving classes is slow #127
Comments
|
I wrote a benchmark. Here are the raw results:
|
|
Although I ran the benchmark against many versions of the compiler, the output is difficult to understand when graphed.
8.0.x is generally a little slower than 7.10.x, but the difference is both inconsistent and not that big. The thing I'm actually trying to investigate here is the impact on compile time of deriving type classes. It was important to me to show that GHC 8.0.2 isn't drastically different than other recent versions of GHC. (I could test with even older versions of GHC, but I'd have to exclude
This shows that deriving |
|
Compiling modules with types always takes the longest … I wonder why that is? |
|
This is something pointed out as something to work on for Summer of Haskell, there's one more day for applications! https://summer.haskell.org/ideas.html#galois-ghc https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler#Derivinginstances |
|
For Read, I think this ticket is applicable: |
|
And Ord should hopefully be a bit better in 8.2.1: |
|
I ran my benchmark against GHC 8.2.1-rc1 (https://gist.github.com/tfausak/a36862c53a2cc53029cab18a05788b95). Here are the raw results:
And here it is graphed compared to 8.0.2:
|
|
To get a feel for the generated instances, here they are (via Data (implies Typeable)instance Data.Data.Data body_a4Wo =>
Data.Data.Data (Deriving.Data.Section body_a4Wo) where
Data.Data.gfoldl
k_a4YQ
z_a4YR
(Deriving.Data.Section a1_a4YS a2_a4YT a3_a4YU)
= (((z_a4YR Deriving.Data.Section `k_a4YQ` a1_a4YS)
`k_a4YQ` a2_a4YT)
`k_a4YQ` a3_a4YU)
Data.Data.gunfold k_a4YV z_a4YW _
= k_a4YV (k_a4YV (k_a4YV (z_a4YW Deriving.Data.Section)))
Data.Data.toConstr (Deriving.Data.Section _ _ _)
= Deriving.Data.$cCOLrOXxQLtLE5asoBRhrjA
Data.Data.dataTypeOf _ = Deriving.Data.$tCOLrOXxQLtLE5asoBRhrjA
Data.Data.dataCast1 f_a4YX = Data.Typeable.gcast1 f_a4YXEqinstance GHC.Classes.Eq body_a2PO =>
GHC.Classes.Eq (Deriving.Eq.Section body_a2PO) where
(GHC.Classes.==)
(Deriving.Eq.Section a1_a2PS a2_a2PT a3_a2PU)
(Deriving.Eq.Section b1_a2PV b2_a2PW b3_a2PX)
= ((((a1_a2PS GHC.Classes.== b1_a2PV))
GHC.Classes.&& ((a2_a2PT GHC.Classes.== b2_a2PW)))
GHC.Classes.&& ((a3_a2PU GHC.Classes.== b3_a2PX)))
(GHC.Classes./=) a_a2PY b_a2PZ
= GHC.Classes.not ((GHC.Classes.==) a_a2PY b_a2PZ)Genericinstance GHC.Generics.Generic
(Deriving.Generic.Section body_a2ww) where
GHC.Generics.from x_a2xu
= GHC.Generics.M1
(case x_a2xu of {
Deriving.Generic.Section g1_a2xv g2_a2xw g3_a2xx
-> GHC.Generics.M1
((GHC.Generics.:*:)
(GHC.Generics.M1 (GHC.Generics.K1 g1_a2xv))
((GHC.Generics.:*:)
(GHC.Generics.M1 (GHC.Generics.K1 g2_a2xw))
(GHC.Generics.M1 (GHC.Generics.K1 g3_a2xx)))) })
GHC.Generics.to (GHC.Generics.M1 x_a2xy)
= case x_a2xy of {
GHC.Generics.M1 ((GHC.Generics.:*:) (GHC.Generics.M1 (GHC.Generics.K1 g1_a2xz))
((GHC.Generics.:*:) (GHC.Generics.M1 (GHC.Generics.K1 g2_a2xA))
(GHC.Generics.M1 (GHC.Generics.K1 g3_a2xB))))
-> Deriving.Generic.Section g1_a2xz g2_a2xA g3_a2xB }NFData (implies Generic)instance Control.DeepSeq.NFData body_a7gu =>
Control.DeepSeq.NFData (Deriving.NFData.Section body_a7gu) whereOrd (implies Eq)instance GHC.Classes.Ord body_a3uw =>
GHC.Classes.Ord (Deriving.Ord.Section body_a3uw) where
GHC.Classes.compare a_a3uQ b_a3uR
= case a_a3uQ of {
Deriving.Ord.Section a1_a3uS a2_a3uT a3_a3uU
-> case b_a3uR of {
Deriving.Ord.Section b1_a3uV b2_a3uW b3_a3uX
-> case (GHC.Classes.compare a1_a3uS b1_a3uV) of {
GHC.Types.LT -> GHC.Types.LT
GHC.Types.EQ
-> case (GHC.Classes.compare a2_a3uT b2_a3uW) of {
GHC.Types.LT -> GHC.Types.LT
GHC.Types.EQ -> (a3_a3uU `GHC.Classes.compare` b3_a3uX)
GHC.Types.GT -> GHC.Types.GT }
GHC.Types.GT -> GHC.Types.GT } } }
(GHC.Classes.<) a_a3uY b_a3uZ
= case a_a3uY of {
Deriving.Ord.Section a1_a3v0 a2_a3v1 a3_a3v2
-> case b_a3uZ of {
Deriving.Ord.Section b1_a3v3 b2_a3v4 b3_a3v5
-> case (GHC.Classes.compare a1_a3v0 b1_a3v3) of {
GHC.Types.LT -> GHC.Types.True
GHC.Types.EQ
-> case (GHC.Classes.compare a2_a3v1 b2_a3v4) of {
GHC.Types.LT -> GHC.Types.True
GHC.Types.EQ -> (a3_a3v2 GHC.Classes.< b3_a3v5)
GHC.Types.GT -> GHC.Types.False }
GHC.Types.GT -> GHC.Types.False } } }
(GHC.Classes.<=) a_a3v6 b_a3v7
= case a_a3v6 of {
Deriving.Ord.Section a1_a3v8 a2_a3v9 a3_a3va
-> case b_a3v7 of {
Deriving.Ord.Section b1_a3vb b2_a3vc b3_a3vd
-> case (GHC.Classes.compare a1_a3v8 b1_a3vb) of {
GHC.Types.LT -> GHC.Types.True
GHC.Types.EQ
-> case (GHC.Classes.compare a2_a3v9 b2_a3vc) of {
GHC.Types.LT -> GHC.Types.True
GHC.Types.EQ -> (a3_a3va GHC.Classes.<= b3_a3vd)
GHC.Types.GT -> GHC.Types.False }
GHC.Types.GT -> GHC.Types.False } } }
(GHC.Classes.>=) a_a3ve b_a3vf
= case a_a3ve of {
Deriving.Ord.Section a1_a3vg a2_a3vh a3_a3vi
-> case b_a3vf of {
Deriving.Ord.Section b1_a3vj b2_a3vk b3_a3vl
-> case (GHC.Classes.compare a1_a3vg b1_a3vj) of {
GHC.Types.LT -> GHC.Types.False
GHC.Types.EQ
-> case (GHC.Classes.compare a2_a3vh b2_a3vk) of {
GHC.Types.LT -> GHC.Types.False
GHC.Types.EQ -> (a3_a3vi GHC.Classes.>= b3_a3vl)
GHC.Types.GT -> GHC.Types.True }
GHC.Types.GT -> GHC.Types.True } } }
(GHC.Classes.>) a_a3vm b_a3vn
= case a_a3vm of {
Deriving.Ord.Section a1_a3vo a2_a3vp a3_a3vq
-> case b_a3vn of {
Deriving.Ord.Section b1_a3vr b2_a3vs b3_a3vt
-> case (GHC.Classes.compare a1_a3vo b1_a3vr) of {
GHC.Types.LT -> GHC.Types.False
GHC.Types.EQ
-> case (GHC.Classes.compare a2_a3vp b2_a3vs) of {
GHC.Types.LT -> GHC.Types.False
GHC.Types.EQ -> (a3_a3vq GHC.Classes.> b3_a3vt)
GHC.Types.GT -> GHC.Types.True }
GHC.Types.GT -> GHC.Types.True } } }Readinstance GHC.Read.Read body_a2E5 =>
GHC.Read.Read (Deriving.Read.Section body_a2E5) where
GHC.Read.readPrec
= GHC.Read.parens
(Text.ParserCombinators.ReadPrec.prec
11
(do { GHC.Read.expectP (Text.Read.Lex.Ident "Section");
GHC.Read.expectP (Text.Read.Lex.Punc "{");
GHC.Read.expectP (Text.Read.Lex.Ident "sectionSize");
GHC.Read.expectP (Text.Read.Lex.Punc "=");
a1_a2Eb <- Text.ParserCombinators.ReadPrec.reset GHC.Read.readPrec;
GHC.Read.expectP (Text.Read.Lex.Punc ",");
GHC.Read.expectP (Text.Read.Lex.Ident "sectionCrc");
GHC.Read.expectP (Text.Read.Lex.Punc "=");
a2_a2Ec <- Text.ParserCombinators.ReadPrec.reset GHC.Read.readPrec;
GHC.Read.expectP (Text.Read.Lex.Punc ",");
GHC.Read.expectP (Text.Read.Lex.Ident "sectionBody");
GHC.Read.expectP (Text.Read.Lex.Punc "=");
a3_a2Ed <- Text.ParserCombinators.ReadPrec.reset GHC.Read.readPrec;
GHC.Read.expectP (Text.Read.Lex.Punc "}");
GHC.Base.return (Deriving.Read.Section a1_a2Eb a2_a2Ec a3_a2Ed) }))
GHC.Read.readList = GHC.Read.readListDefault
GHC.Read.readListPrec = GHC.Read.readListPrecDefaultShowinstance GHC.Show.Show body_a2aI =>
GHC.Show.Show (Deriving.Show.Section body_a2aI) where
GHC.Show.showsPrec
a_a2aM
(Deriving.Show.Section b1_a2aN b2_a2aO b3_a2aP)
= GHC.Show.showParen
(a_a2aM GHC.Classes.>= 11)
((GHC.Base..)
(GHC.Show.showString "Section {")
((GHC.Base..)
(GHC.Show.showString "sectionSize = ")
((GHC.Base..)
(GHC.Show.showsPrec 0 b1_a2aN)
((GHC.Base..)
(GHC.Show.showString ", ")
((GHC.Base..)
(GHC.Show.showString "sectionCrc = ")
((GHC.Base..)
(GHC.Show.showsPrec 0 b2_a2aO)
((GHC.Base..)
(GHC.Show.showString ", ")
((GHC.Base..)
(GHC.Show.showString "sectionBody = ")
((GHC.Base..)
(GHC.Show.showsPrec 0 b3_a2aP)
(GHC.Show.showString "}"))))))))))
GHC.Show.showList = GHC.Show.showList__ (GHC.Show.showsPrec 0)TypeableNothing :)I would like to see how non-derived instances affect compile times. Unfortunately these dumped instances aren't clean enough to put as-is in a module. Also even copy-pasting them would be tedious. I'll see if I can come up with something to replace |
|
I was curious if just having the instances affected compile times at all. So I tested manually adding instances like this: instance Eq t where (==) = undefined
instance Ord t where compare = undefined
instance Read t where readsPrec = undefined
instance Show t where show = undefinedHere are the raw results (from a faster machine than previous benchmarks):
Graphed:
Providing empty instances for |
|
I compared
So providing a |
|
GHC 8.2.1-rc2 is moderately faster than 8.0.2.
|
|
I was curious how
It's crazy how deriving all these classes slows compilation down by 19x.
The good news is that the actual build time for all the type classes is lower than the expected build time (made by summing the deltas for each type class).
Anyway, I still want to see how much of this time is code generation. That might be easier with |
|
Oh, also: I think I'll wait until GHC 8.2.1 is out before actually writing this blog post.It should be out relatively recently.
|
|
I should test different optimization levels too. |
|
GHC 8.2.1 has been officially released, so I should re-run my benchmarks against it. |
|
I ran the benchmarks against GHC 8.2.1:
|
|
I massively reworked the benchmark to test more versions of GHC and more easily change the list of derived classes via CPP: https://gist.github.com/tfausak/0c90ed1b450ae84136c110f026e16bc6 The new benchmark takes a long time to run. Setting up the Docker container takes at least an hour. Running the actual benchmark takes about an hour. I ran the benchmark on my laptop and saved the results (raw, text, CSV, JSON, and HTML): report.zip I also uploaded the HTML results to my blog so that you can click around: http://taylor.fausak.me/static/pages/2017-07-25-haskell-type-class-deriving-performance.html The Criterion report isn't great for visualizing the results, so I'll need to make some better graphs from the CSV or JSON. Regardless, here are the results for deriving all classes (Data, Eq, Ord, Read, Show, Typeable) across GHC versions 7.0.1 to 8.2.1:
Bars show compile time in seconds. Lower is better. The fastest was GHC 7.8.4 at 3.84 seconds. The slowest was GHC 7.8.1 at 4.65 seconds. Broadly speaking, GHC 7.10 and 8.0 were slow, but GHC 8.2 seems to have returned us to GHC 7.8-level speeds. I also have this data broken down by class, but it's a lot to analyze. Glancing over the graphs, For the base case of deriving no classes, the Criterion graph makes the bars pretty short.
(Same as before: Compile time in seconds. Lower is better.) The fastest is GHC 7.0.4 at 172 milliseconds. The slowest is GHC 7.10.2 at 475 milliseconds. Things got slower with GHC 7.8 (304 milliseconds) and haven't recovered, even with GHC 8.2.1 (328 milliseconds). |
I was curious about how long it takes to derive type classes in Haskell. So I ripped out all the data types from Rattletrap, put them in a single module, and did some benchmarks.
TL;DR:
deriving (): 1.5 secondsderiving (Typeable, Eq, Generic, Show, Ord, Read, Data): 33.9 secondsThe text was updated successfully, but these errors were encountered: