WIP: Add summary statistics to the dataresource

[alexandercbooth]

• closes
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Following the discussion in https://github.com/nteract/nteract/issues/3177, this adds summary statistics for folks using the dataresource. Our main concern is how this might affect performance.

I wouldn't have put this up before the tests are passing, but as @rgbkrk and I are currently attending a Numfocus event, we hoped we might catch up with @jreback and/or others in person if anyone is attending to discuss and wanted to get the ball rolling 🙂.

cc @TomAugspurger

[pep8speaks]

Hello @alexandercbooth! Thanks for submitting the PR.

• There are no PEP8 issues in the file pandas/io/json/table_schema.py !

• In the file pandas/tests/io/json/test_json_table_schema.py, following are the PEP8 issues :

Line 35:9: E128 continuation line under-indented for visual indent
Line 36:9: E128 continuation line under-indented for visual indent
Line 37:13: E128 continuation line under-indented for visual indent
Line 38:13: E128 continuation line under-indented for visual indent
Line 39:13: E128 continuation line under-indented for visual indent
Line 40:13: E128 continuation line under-indented for visual indent
Line 41:13: E128 continuation line under-indented for visual indent
Line 42:13: E128 continuation line under-indented for visual indent
Line 43:13: E128 continuation line under-indented for visual indent
Line 45:9: E122 continuation line missing indentation or outdented
Line 46:9: E128 continuation line under-indented for visual indent
Line 47:9: E128 continuation line under-indented for visual indent
Line 48:9: E122 continuation line missing indentation or outdented
Line 49:9: E128 continuation line under-indented for visual indent
Line 50:13: E128 continuation line under-indented for visual indent
Line 51:13: E128 continuation line under-indented for visual indent
Line 52:13: E128 continuation line under-indented for visual indent
Line 53:13: E128 continuation line under-indented for visual indent
Line 54:13: E128 continuation line under-indented for visual indent
Line 56:9: E122 continuation line missing indentation or outdented
Line 57:9: E128 continuation line under-indented for visual indent
Line 58:13: E128 continuation line under-indented for visual indent
Line 59:13: E128 continuation line under-indented for visual indent
Line 60:13: E128 continuation line under-indented for visual indent
Line 61:13: E128 continuation line under-indented for visual indent
Line 62:13: E128 continuation line under-indented for visual indent
Line 63:13: E128 continuation line under-indented for visual indent
Line 64:13: E128 continuation line under-indented for visual indent
Line 66:9: E122 continuation line missing indentation or outdented
Line 66:10: E231 missing whitespace after ','
Line 76:9: E128 continuation line under-indented for visual indent
Line 77:9: E128 continuation line under-indented for visual indent
Line 78:9: E128 continuation line under-indented for visual indent
Line 79:13: E128 continuation line under-indented for visual indent
Line 80:13: E128 continuation line under-indented for visual indent
Line 81:13: E128 continuation line under-indented for visual indent
Line 82:13: E128 continuation line under-indented for visual indent
Line 83:13: E128 continuation line under-indented for visual indent
Line 84:13: E128 continuation line under-indented for visual indent
Line 85:13: E128 continuation line under-indented for visual indent
Line 86:9: E122 continuation line missing indentation or outdented
Line 94:9: E128 continuation line under-indented for visual indent
Line 95:9: E128 continuation line under-indented for visual indent
Line 96:9: E128 continuation line under-indented for visual indent
Line 97:13: E128 continuation line under-indented for visual indent
Line 98:13: E128 continuation line under-indented for visual indent
Line 99:13: E128 continuation line under-indented for visual indent
Line 100:13: E128 continuation line under-indented for visual indent
Line 101:13: E128 continuation line under-indented for visual indent
Line 102:13: E128 continuation line under-indented for visual indent
Line 103:13: E128 continuation line under-indented for visual indent
Line 104:9: E122 continuation line missing indentation or outdented
Line 114:9: E128 continuation line under-indented for visual indent
Line 115:9: E122 continuation line missing indentation or outdented
Line 116:9: E128 continuation line under-indented for visual indent
Line 117:9: E128 continuation line under-indented for visual indent
Line 118:13: E128 continuation line under-indented for visual indent
Line 119:13: E128 continuation line under-indented for visual indent
Line 120:13: E128 continuation line under-indented for visual indent
Line 121:13: E128 continuation line under-indented for visual indent
Line 122:13: E128 continuation line under-indented for visual indent
Line 123:13: E128 continuation line under-indented for visual indent
Line 124:13: E128 continuation line under-indented for visual indent
Line 125:9: E122 continuation line missing indentation or outdented
Line 126:9: E128 continuation line under-indented for visual indent
Line 127:9: E128 continuation line under-indented for visual indent
Line 128:9: E122 continuation line missing indentation or outdented
Line 129:9: E128 continuation line under-indented for visual indent
Line 130:9: E128 continuation line under-indented for visual indent
Line 131:13: E128 continuation line under-indented for visual indent
Line 132:13: E128 continuation line under-indented for visual indent
Line 133:13: E128 continuation line under-indented for visual indent
Line 134:13: E128 continuation line under-indented for visual indent
Line 135:13: E128 continuation line under-indented for visual indent
Line 136:9: E122 continuation line missing indentation or outdented
Line 137:9: E128 continuation line under-indented for visual indent
Line 138:9: E128 continuation line under-indented for visual indent
Line 139:13: E128 continuation line under-indented for visual indent
Line 140:13: E128 continuation line under-indented for visual indent
Line 141:13: E128 continuation line under-indented for visual indent
Line 142:13: E128 continuation line under-indented for visual indent
Line 143:13: E128 continuation line under-indented for visual indent
Line 144:13: E128 continuation line under-indented for visual indent
Line 145:13: E128 continuation line under-indented for visual indent
Line 146:9: E122 continuation line missing indentation or outdented
Line 363:16: E128 continuation line under-indented for visual indent
Line 368:16: E128 continuation line under-indented for visual indent
Line 373:16: E128 continuation line under-indented for visual indent
Line 389:35: E231 missing whitespace after ':'
Line 389:43: E231 missing whitespace after ','
Line 389:50: E231 missing whitespace after ':'
Line 390:17: E128 continuation line under-indented for visual indent
Line 390:24: E231 missing whitespace after ':'
Line 390:33: E231 missing whitespace after ','
Line 390:40: E231 missing whitespace after ':'
Line 390:50: E231 missing whitespace after ','
Line 391:17: E128 continuation line under-indented for visual indent
Line 391:25: E231 missing whitespace after ':'
Line 391:29: E231 missing whitespace after ','
Line 391:36: E231 missing whitespace after ':'
Line 391:40: E231 missing whitespace after ','
Line 391:46: E231 missing whitespace after ':'
Line 391:50: E231 missing whitespace after ','
Line 391:56: E231 missing whitespace after ':'
Line 391:60: E231 missing whitespace after ','
Line 391:66: E231 missing whitespace after ':'
Line 391:70: E231 missing whitespace after ','
Line 391:76: E231 missing whitespace after ':'
Line 391:80: E231 missing whitespace after ','
Line 391:80: E501 line too long (103 > 79 characters)
Line 391:86: E231 missing whitespace after ':'
Line 391:90: E231 missing whitespace after ','
Line 391:96: E231 missing whitespace after ':'
Line 406:26: E231 missing whitespace after ':'
Line 406:34: E231 missing whitespace after ','
Line 406:41: E231 missing whitespace after ':'
Line 406:52: E231 missing whitespace after ','
Line 406:59: E231 missing whitespace after ':'
Line 407:9: E128 continuation line under-indented for visual indent
Line 407:16: E231 missing whitespace after ':'
Line 407:25: E231 missing whitespace after ','
Line 407:32: E231 missing whitespace after ':'
Line 407:42: E231 missing whitespace after ','
Line 408:9: E128 continuation line under-indented for visual indent
Line 408:17: E231 missing whitespace after ':'
Line 408:21: E231 missing whitespace after ','
Line 408:28: E231 missing whitespace after ':'
Line 408:32: E231 missing whitespace after ','
Line 408:38: E231 missing whitespace after ':'
Line 408:42: E231 missing whitespace after ','
Line 408:48: E231 missing whitespace after ':'
Line 408:52: E231 missing whitespace after ','
Line 408:58: E231 missing whitespace after ':'
Line 408:62: E231 missing whitespace after ','
Line 408:68: E231 missing whitespace after ':'
Line 408:72: E231 missing whitespace after ','
Line 408:78: E231 missing whitespace after ':'
Line 408:80: E501 line too long (94 > 79 characters)
Line 408:82: E231 missing whitespace after ','
Line 408:88: E231 missing whitespace after ':'
Line 427:41: E126 continuation line over-indented for hanging indent
Line 430:80: E501 line too long (80 > 79 characters)
Line 447:37: E126 continuation line over-indented for hanging indent
Line 466:80: E501 line too long (80 > 79 characters)
Line 470:21: E128 continuation line under-indented for visual indent
Line 471:21: E128 continuation line under-indented for visual indent
Line 472:21: E128 continuation line under-indented for visual indent
Line 473:21: E128 continuation line under-indented for visual indent
Line 474:21: E128 continuation line under-indented for visual indent
Line 475:21: E128 continuation line under-indented for visual indent
Line 476:21: E128 continuation line under-indented for visual indent
Line 478:1: W293 blank line contains whitespace
Line 489:80: E501 line too long (80 > 79 characters)
Line 493:21: E128 continuation line under-indented for visual indent
Line 494:21: E128 continuation line under-indented for visual indent
Line 495:21: E128 continuation line under-indented for visual indent
Line 496:21: E128 continuation line under-indented for visual indent
Line 497:21: E128 continuation line under-indented for visual indent
Line 498:21: E128 continuation line under-indented for visual indent
Line 499:21: E128 continuation line under-indented for visual indent
Line 524:21: E128 continuation line under-indented for visual indent
Line 525:21: E128 continuation line under-indented for visual indent
Line 525:80: E501 line too long (88 > 79 characters)
Line 526:21: E128 continuation line under-indented for visual indent
Line 527:21: E128 continuation line under-indented for visual indent
Line 527:80: E501 line too long (90 > 79 characters)
Line 528:21: E128 continuation line under-indented for visual indent
Line 528:80: E501 line too long (90 > 79 characters)

[jorisvandenbossche]

With this PR, it relies on the specific output of pandas' describe method, but shouldn't this be some standardized fields in the table schema spec?

[rgbkrk]

@rufuspollock @pwalsh -- is there a standardized section for summary statistics (similar to what df.describe() does) for the data or could we add it to the spec for data resource?

[jreback]

closing as stale. if you'd like to continue, pls ping.