Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: assert_series_equal not raising on unequal series? #55882

Closed
3 tasks done
MarcoGorelli opened this issue Nov 8, 2023 · 7 comments · Fixed by #55934
Closed
3 tasks done

BUG: assert_series_equal not raising on unequal series? #55882

MarcoGorelli opened this issue Nov 8, 2023 · 7 comments · Fixed by #55934
Labels
Blocker for rc Blocking issue or pull request for release candidate Bug Testing pandas testing functions or related to the test suite
Milestone

Comments

@MarcoGorelli
Copy link
Member

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

left = pd.Series([1577840521123000])
right = pd.Series([1577840521123543])
pd.testing.assert_series_equal(left, right)

Issue Description

The above doesn't raise

Expected Behavior

It should raise

Installed Versions

In [4]: pd.show_versions()
/home/marcogorelli/tmp/.venv/lib/python3.10/site-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")

INSTALLED VERSIONS

commit : 51f3d03
python : 3.10.12.final.0
python-bits : 64
OS : Linux
OS-release : 5.10.102.1-microsoft-standard-WSL2
Version : #1 SMP Wed Mar 2 00:30:59 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 2.2.0.dev0+541.g51f3d03087
numpy : 1.25.1
pytz : 2023.3
dateutil : 2.8.2
setuptools : 67.6.1
pip : 23.1.2
Cython : None
pytest : 7.3.1
hypothesis : 6.82.4
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.16.1
pandas_datareader : None
bs4 : 4.12.2
bottleneck : None
dataframe-api-compat: None
fastparquet : 2023.7.0
fsspec : 2023.6.0
gcsfs : None
matplotlib : 3.7.1
numba : 0.58.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 12.0.1
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.11.0
sqlalchemy : 2.0.20
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

@MarcoGorelli MarcoGorelli added Bug Needs Triage Issue that has not been reviewed by a pandas team member Testing pandas testing functions or related to the test suite labels Nov 8, 2023
@EricJeannotte
Copy link

By default, the parameter check_exact=False, which means that default relative(1e-5) and absolute tolerances(1e-8) are applied according to documentation. Considering that the numbers are very large in your example, I believe the output is correct.

@MarcoGorelli
Copy link
Member Author

from today's call, this check_exact / rtol / atol should not take effect for ints https://docs.google.com/document/d/1tGbTiYORHiSPgVMXawiweGJlBw5dOkVJLY-licoBmBU/edit

@lithomas1 lithomas1 removed the Needs Triage Issue that has not been reviewed by a pandas team member label Nov 11, 2023
@lithomas1 lithomas1 added this to the 2.2 milestone Nov 11, 2023
@lithomas1 lithomas1 added the Blocker for rc Blocking issue or pull request for release candidate label Nov 11, 2023
@lithomas1
Copy link
Member

Labelling as blocker for 2.2, not because it is a blocker, but because it looks pretty nasty and we should definitely fix this.

@parthi-siva
Copy link
Contributor

take

@parthi-siva
Copy link
Contributor

parthi-siva commented Nov 13, 2023

Hi @MarcoGorelli
can we add a check like this in pandas/_testing/asserters.py

    if left.dtype == np.int64 and right.dtype == np.int64:
        check_exact = True

after adding it's getting raised

In [1]: import pandas as pd

In [2]: left = pd.Series([1577840521123000])
   ...: right = pd.Series([1577840521123543])

In [3]: pd.testing.assert_series_equal(left, right)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 pd.testing.assert_series_equal(left, right)

    [... skipping hidden 2 frame]

File ~/Documents/pandas/pandas/_testing/asserters.py:672, in assert_numpy_array_equal.<locals>._raise(left, right, err_msg)
    670     diff = diff * 100.0 / left.size
    671     msg = f"{obj} values are different ({np.round(diff, 5)} %)"
--> 672     raise_assert_detail(obj, msg, left, right, index_values=index_values)
    674 raise AssertionError(err_msg)

File ~/Documents/pandas/pandas/_testing/asserters.py:602, in raise_assert_detail(obj, message, left, right, diff, first_diff, index_values)
    599 if first_diff is not None:
    600     msg += f"\n{first_diff}"
--> 602 raise AssertionError(msg)

AssertionError: Series are different

Series values are different (100.0 %)
[index]: [0]
[left]:  [1577840521123000]
[right]: [1577840521123543]

@MarcoGorelli
Copy link
Member Author

nice - it would need to be more generic than if left.dtype == np.int64 and right.dtype == np.int64:, I think there's a is_integer_dtype function (or something like that) you might be able to use

@parthi-siva
Copy link
Contributor

Yeah right. Thanks @MarcoGorelli create a PR. Once pls review

@parthi-siva parthi-siva removed their assignment Nov 23, 2023
rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Feb 21, 2024
`test_order_nested_json_reader` was refactored to use `assert_eq` instead of comparing via pyarrow. This was failing in pandas 2.2 due to pandas-dev/pandas#57429

`test_orc_reader_trailing_nulls` I believe was failing due to a change in how integers are compared with `assert_series_equal`: pandas-dev/pandas#55882. The "casting workaround" doesn't seem necessary in pandas 2.2 so just avoiding it all together

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #15062
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Blocker for rc Blocking issue or pull request for release candidate Bug Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants