BUG: fix reading multi-index data in python parser

[mcwitt]

partial fix for #6893

The python parser has a problem reading data with a multi-index specified in the row following the header, for example

In [3]: text = """                      A       B       C       D        E
one two three   four
a   b   10.0032 5    -0.5109 -2.3358 -0.4645  0.05076  0.3640
a   q   20      4     0.4473  1.4152  0.2834  1.00661  0.1744
x   q   30      3    -0.6662 -0.5243 -0.3580  0.89145  2.5838"""

In [4]: pd.read_table(StringIO(text), sep='\s+', engine='python')
Out[4]: 
                           E
one two three   four        
a   b   10.0032 5     0.3640
    q   20.0000 4     0.1744
x   q   30.0000 3     2.5838

[3 rows x 1 columns]

(the C parser doesn't make it this far, see #6893)

This PR fixes the bug in the python parser:

In [4]: pd.read_table(StringIO(text), sep='\s+', engine='python')
Out[4]: 
                           A       B       C        D       E
one two three   four                                         
a   b   10.0032 5    -0.5109 -2.3358 -0.4645  0.05076  0.3640
    q   20.0000 4     0.4473  1.4152  0.2834  1.00661  0.1744
x   q   30.0000 3    -0.6662 -0.5243 -0.3580  0.89145  2.5838

[3 rows x 5 columns]

[mcwitt]

Don't ignore columns here, we will lose the columns added from the row specifying the index...

[jreback]

ok, pls add a release note (even if it doesn't completely solve)

[mcwitt]

@jreback OK, release note added. OK to open another PR for the C parser fix or at least to detect the condition?

[jreback]

go ahead and open a new issue for the c-parser (reference the original issue too and this fix)

put the release note with the other changes for the parser (just move right below)
are their any tests for this? or were the others just disabled?

[mcwitt]

go ahead and open a new issue for the c-parser (reference the original issue too and this fix)

would it make sense just to leave the original issue open, since this is only a partial fix?

are their any tests for this? or were the others just disabled?

sort of... test_read_table_buglet_4x_multiindex was previously in ParserTests but I copied it to TestPythonParser in #6889 since it had been falling back to python. For the C engine we currently just test that it produces a CParserError. However the test only checks that the index is read correctly and so it didn't expose the bug fixed in this PR. Now I've added a bit to the bottom that does cover this (testing now, will commit soon).

[jreback]

ok...we can leave the original issue open, no prob

let me know when you have rebased and green

[mcwitt]

@jreback green!

[jreback]

thanks!

ok....so come back to the open issue at some point (its in 0.14.1), the projected bug-fix release
so have some time (prob 1 mo after 0.14)