Loading modules with large literals is very slow

[brianhuffman]

If I create and load a module containing a single very large hex literal, it takes a very long time to load: about 5 minutes on my new fast machine for a size on the order of half a million bits. The running time seems to be (at least) quadratic in the length.

module Test where
x : [500000]
x = 0x3a1c27d9ef018b....

(Why do I want to load such files? The idea is to generate modules like this from binary files, as a workaround for issue #137.)

[brianhuffman]

Profiling indicates that the majority of time is taken up by the lexer:

total time  =      231.52 secs   (231524 ticks @ 1000 us, 1 processor)
total alloc = 116,494,851,712 bytes  (excludes profiling overheads)

COST CENTRE      MODULE                          %time %alloc

numToken.toVal.\ Cryptol.Parser.LexerUtils        82.8   25.0
genLog.underLoop Cryptol.TypeCheck.Solver.InfNat  16.5   68.5
numToken.toVal   Cryptol.Parser.LexerUtils         0.3    5.7

[brianhuffman]

After 7adf5b2, the bottleneck is now genLog, which is used to evaluate the width type function.

    total time  =       39.73 secs   (39729 ticks @ 1000 us, 1 processor)
    total alloc = 93,921,624,792 bytes  (excludes profiling overheads)

COST CENTRE      MODULE                          %time %alloc

genLog.underLoop Cryptol.TypeCheck.Solver.InfNat  96.0   85.0
numToken.toVal.\ Cryptol.Parser.LexerUtils         2.5   14.1
fromDecDigit     Cryptol.Parser.LexerUtils         1.2    0.7