BUG: SparseDataFrame from scipy.sparse.dia.dia_matrix returns incorrect dataframe

[keitakurita]

Code Sample, a copy-pastable example if possible

>>> import numpy as np
>>> import scipy.sparse
>>> A = np.arange(1, 5).reshape(2, 2)
>>> A
array([[1, 2],
       [3, 4]])
>>> spm = scipy.sparse.bsr.bsr_matrix(A)
>>> pd.SparseDataFrame(spm)
   0  1
0  1  2
1  3  4
>>> spm = scipy.sparse.dia.dia_matrix(A)
>>> pd.SparseDataFrame(spm)
   0  1
0  3  4
1  1  2
>>> pd.SparseDataFrame(A)
   0  1
0  1  2
1  3  4

Problem description

Initialization of SparseDataFrame with scipy.sparse.dia.dia_matrix returns a different result from direct initialization from np.ndarray and other sparse matrix formats. This is unexpected behavior since the user would expect the output DataFrame to be consistent with the initial array.

Expected Output

>>> pd.SparseDataFrame(spm)
   0  1
0  1  2
1  3  4

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: 075eca1 python: 3.6.1.final.0 python-bits: 64 OS: Darwin OS-release: 16.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: ja_JP.UTF-8 LOCALE: ja_JP.UTF-8

pandas: 0.20.0rc1+29.g075eca1
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.18.1
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

[jreback]

cc @kernc

[jreback]

this is a dupe of #16179 it appears.

[keitakurita]

@jreback
If this issue is going to be closed, I think it would be a good idea to add the error for the dok format sparse matrix to issue #16179 , since these are two distinct sparse matrix formats with slightly different buggy behaviors.

[jreback]

you can update the other issue if you like, this is just another case