Human readable formatting for numbers

[straight-shoota]

This PR adds three methods for formatting numbers in human-readable formats:

• Number#format: Similar to #to_s but allows customized separator and delimiter (useful for localization) and specifying the number of printed decimal places.
• Number#humanize: Formats the most significant digits of the number with a prefix indicating the magnitude. Output parameters, prefixes etc. can be fully customized.
• Int#humanize_bytes: Specialization of Number#humanize for binary units

123.4567.format(decimal_places: 3) # => "123.457"
1_200_000_000.humanize # => "1.2G"
1536.humanize_bytes # => "1.5KB"

Number#humanize is already put to use in the benchmark module.

[asterite]

Can this be in a number/humanize require? The std keeps growing and base compile times are getting slower and slower.

[RX14]

@asterite methods dont affect compile times at all unless they are used. This method is unused unless you use the benchmark harness then you're doing a release build anyway and you don't care.

[asterite]

@RX14 Almost: the parser still has to parse it and build an AST for it. Every bit counts.

[straight-shoota]

@asterite I don't think that extra tiny bit is worth the nuisance to have to require number/humanize. I'd like to encourage people to use these formatter methods for human-readable output and hope they'll be used in many Crystal applications 👍

[wooster0]

The prelude.cr/std should be shrinked generally I think. Every single time you compile some crystal file, all the methods in every require in prelude.cr get parsed. Even if you only use 3 or no methods of all these thousand methods.

time(crystal emptyfile.cr --prelude=empty); time(crystal emptyfile.cr)

real    0m0.349s
user    0m0.156s
sys     0m0.094s

real    0m2.157s
user    0m1.594s
sys     0m1.219s

[straight-shoota]

@r00ster91 this is not the place to discuss prelude.cr.

[asterite]

@straight-shoota Could you use descriptive commit messages for fixups? Just having fixup! with no context makes it really hard to review commits.

[straight-shoota]

@asterite The first line of the commit message needs to reference the targeted commit, otherwise !fixup doesn't work. Descriptions are only visible in the second line.
Maybe I'm missing something but I can't find a way to prevent that.

[asterite]

Is fixup! a convention or something?

[yxhuvud]

@asterite if you use git rebase --autosquash, then it is magic that automatically make git rebase do the right thing.

@straight-shoota You can refer to refspecs instead. see https://robots.thoughtbot.com/autosquashing-git-commits

[straight-shoota]

@yxhuvud That still doesn't allow custom descriptions in the first line of the commit message, does it?

[yxhuvud]

@straight-shoota Ah, no. That is a pity.

[RX14]

@asterite it's not just a convention, it's an official part of git. git commit --fixup=sha1 and then git commit --autosquash master.

[RX14]

@asterite judging by --stats, parsing uses less than 1% of the time in any program.

I'm entirely unconcerned by parse time. By the time parse time is the compiler bottleneck, we will have won.

[asterite]

That's because parse stats just show the time for parsing the main file. When a require is processed, that parsing time happens in top-level, I think.

At one point all parsing was done in parsing, but since that has changed the parsing step has very little use now as stats info.

Regardless, it's not just parsing time, it's holding the AST nodes in memory for the entire compilation.

But yeah, let's not care. Let's wait for the magical solution that will enable incremental compilation :-)

[RX14]

@asterite Parse and top level together for the compiler use 473 milliseconds out of 28.618 seconds. That's 1.65%. It's still not a bottleneck. For cached builds, that increases to about 2.5% of time.

I'm not saying compiler performance isn't useful, but this is picking the wrong thing to worry about.

[asterite]

Sounds good.

[RX14]

I think this is good, but I still think the IEC standard should be the default.

Another place where it would be possible to use this method is in HTTP::LogHandler.

[straight-shoota]

What do others think about IEC vs JEDEC? (see thread #6314 (comment))

[Blacksmoke16]

Will this make it into 0.27.1?

[bcardiff]

I would prefer the number and string code to be in a humanize.cr file in the std lib (even if it's included in the prelude). That would help to keep related code together.

[bcardiff]

@straight-shoota what do you think about organizing all the humanize code in a separate file/folder?

[RX14]

@bcardiff he's on holiday...

[straight-shoota]

@bcardiff done

[Sija]

Hmm, seems that lately CI bugs increased in frequency.

[bcardiff]

The build error should go out after a rebase. But I would merge this with squash though. So GTG on my side.

The code looks better in the current layout. Thanks, @straight-shoota.

[straight-shoota]

CI is still broken @bcardiff

[bcardiff]

hmm, it's the same issue as in #7179.

Lot's of cc: error: ****.o: No such file or directory

[asterite]

Can those failing builds be reproduced? Are they using docker? If we can't reproduce these locally it'll be very hard to figure out what has gone wrong. And I'm almost sure it has nothing to do with a specific PR, I'm almost sure it's something about the CI (disk, memory?)

[bcardiff]

The CRYSTAL_CACHE_DIR in the CI uses the /tmp inside docker. That was done a long time ago maybe there is something special about /tmp (a cleanup could be triggered?) although its writes are performed over the container as any other folder.

Let's see if using the default location helps at both PRs.

---

spoiler: it didn't work

[j8r]

Usually /tmp is mounted as tmps, which is RAM. If both the storage and the compilation use RAM, we can have issues if the RAM is full. I haven't found anything more.

[bcardiff]

It seems that the issue is not /tmp but somewhere when emitting the .o.tmp files the process stops. Leaving partial .o.tmp files and some missing .o oddly enough the missing files seem to be consistent.

Running in CircleCI make std_spec FLAGS=--no-debug seems to make things pass. Always.

I don't know if it is something about the debug information itself or the file size or something with the memory buffers.

That's all I could find.

[asterite]

I suggest trying to run a debug build if the compiler with debug output (puts, after each .o is generated, check if the file is there later, etc.). Otherwise I don't know how can we debug this.

[straight-shoota]

Rebased and ready to ship.