Propose increasing number of columns to 800

[bkamins]

80 columns in HTML is a very small number given current standards. Especially that browser creates horizontal scroll bar when displaying an object. This should not have a big impact on performance either.

I think leaving 30 lines is OK.

[bkamins]

CC @nalimilan @ronisbr

[bkamins]

@stevengj - do you have an opinion on this proposal? Thank you!

[bkamins]

It would be good to have a decision on this PR before JuliaCon so that Jupyter based materials presented there could take advantage of this change (if it were approved). Thank you!

[fredrikekre]

I'm happy to merge it, but do you have a MWE just so I can try it out?

[fredrikekre]

For something like a rand(1000, 1000) this doesn't look very nice (text/plain output) and doesn't give a scrollbar. For a DataFrame it looks ok though (text/html I suppose?)

[bkamins]

A good point - we checked the same thing in parallel.

The MWE is rand(100, 100) and it shows we have a problem.

The issue is that if I e.g. do:

using DataFrames
DataFrame(rand(100, 100), :auto)

all is good and horizontal scroll bar is automatically added.

However, if I do:

rand(100, 100)

I get (after setting "COLUMNS" to 800):

This is bad and the problem is that for some reason in this case Jupyter Notebook does not insert a horizontal scroll bar. Most likely (I am not 100% about the reason) is that when we do rand(100, 100) the display is always plain text.

This means that before merging this PR we should first understand in what circumstances Jupyter Notebook inserts a horizontal scroll bar.

This would probably also affect display of things like:

which currently use whole window width and "wrap around".

The issue is discussed on SO.

---

In summary: things are more complex than they seemed :( and this PR should not be merged as is.

@nalimilan @ronisbr : what do you think we should do for DataFrames.jl given this context?

In essence the problem is:

user cannot have both wide Matrix and DataFrame nicely displayed. If we set "COLUMNS" to a large value DataFrame will be displayed nicely, but Matrix will be a mess. If we keep "COLUMNS" as it is now (80) then Matrix is OK, but DataFrame is not good

So it seems that the only solution is to have our own configuration parameter for HTML display width/height?

[fredrikekre]

You could perhaps insert this change more surgically for just for text/html

[bkamins]

My feeling was that if we anyway need two separate values then the least problematic approach is to have a custom DataFrames.jl specific configuration option. In DataFrames.jl we could just use get(ENV, "TABLE_COLUMNS", 800) instead of using ENV["COLUMNS"]. Or maybe this should be PrettyTables.jl setting? @ronisbr - what do you think?

[nalimilan]

AFAICT the option should be HTML-specific rather than DataFrames/PrettyTables-specific? Or maybe let's just hardcode the value of 800 in DataFrames? I mean, scrolling until the 800th column is totally impractical anyway. Does IJulia set the :limit=>true IOContext property when printing HTML?

It's hard to find a good solution because none of these choices is ideal IMO. What should ideally happen is that only the columns that fit on screen should be visible, but users would be able to see other columns by clicking on scroll buttons that would load the next block of columns (or by scrolling indefinitely). Other approaches are just stopgaps.

[bkamins]

Other approaches are just stopgaps.

I agree, but this would be a big project to do it right (if practically possible at all).
What you propose is roughly the target functionality of VSCode extension for viewing tables AFAICT.

Or maybe let's just hardcode the value of 800 in DataFrames?

So this is my proposal essentially. But I want to allow this 800 to be changed by setting a proper ENV entry.
I would name property (in order from more specific to more general):

• "DATAFRAME_COLUMNS" if we want it to be DataFrames.jl specific (then only DataFrames.jl needs to be aware of it)
• "TABLE_COLUMNS" if we want it to be PrettyTables.jl specific (then PrettyTables.jl has to be aware of it and in consequence also DataFrames.jl)
• "HTML_COLUMNS" if we want it to be any HTML output specific (then it should be mentioned in IJulia.jl developer documentation)

[nalimilan]

OK. Then could IJulia set the displaysize IOContext property for HTML output?

[ronisbr]

Hi! Sorry for the delay.

In PrettyTables, the maximum number of rows or columns that will be printed is just two keyword parameters. Hence, DataFrames can select an environment variable and we use this value when calling pretty_table.

[bkamins]

Hence, DataFrames can select an environment variable and we use this value when calling pretty_table.

OK - let us discuss it then when you open a PR to DataFrames.jl. Thank you!

[bkamins]

Closing as this solution is problematic per the comments above. In DataFrames.jl we will use separate values to control display size.