Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2024 Data RelVals and InputInfo Events Skimming (actually) Working #45055

Merged
merged 3 commits into from
Aug 20, 2024

Conversation

AdrianoDee
Copy link
Contributor

@AdrianoDee AdrianoDee commented May 27, 2024

PR description:

This PR proposes few things:

  1. the introduction of a new script das-up-to-nevents.py that, given in input a dataset name, allows to get the list of files corresponding to N events. The dataset is also automatically skimmed to include only the run and the lumis from the latest and greatest Golden data certification json;

  2. a fix to the fact that the InputInfo class, while accepting as input a variable events does not really use it to skim the result of the das query. It's only really used in a log. I find it ambiguous and a bit confusing. In practice it also means that for 2023 and 2022 RelVals we have been running on O(few millions) events per pre-release (see e.g. CMSSW_14_1_0_pre3__Data_Run3_2023_Data-RunEGamma2023D). While this is nice in terms of being able to do that (kudos on computing!) it usually slows down a lot the RelVal production. This leverages on the script introduced in (1).

  3. the addition of 2024 data RelVals for Eras from B to F as standard wfs with 2024.* prefix. These are run with:

    • Run3_2024 era;
    • HLT:@relval2024 (currently pointing at GRun);
    • running on 50k events using the script in (1) and the script in (2)
  4. a new set of RelVals under relvals_data_highstats.py that should include all the higher statistics data RelVals workflows with 150k,250k,500k and 1M events. For the moment it includes only the 2024 wfs. This set is not turned on by default to avoid to have an over crowded matrix.

All the needed input RAW samples have been copied via Rucio at T2_CH_CERN.

PR validation:

Running (e.g.) runTheMatrix.py -w data,standard -l 12024.003015, 2024.00405, 2024.007005, 2024.0091, 2024.012005, 2024.013025, 2024.10305, 2024.108005, 2024.108015, 2024.210025, 2024.211015, 2024.2111, 2024.212025, 2024.213015, 2024.214025, 2024.21405, 2024.303015, 2024.310025, 2024.31305, 2024.3141, 2024.400005, 2024.402015

@cmsbuild
Copy link
Contributor

cmsbuild commented May 27, 2024

cms-bot internal usage

@AdrianoDee
Copy link
Contributor Author

test parameters:

  • workflows = 143.0010,143.0011,143.0020,143.0021,143.0030,143.0031,143.0040,143.0041,143.0050,143.0051,143.0060,143.0061,143.0070,143.0071,143.0080,143.0081,143.0090,143.0091,143.0100,143.0101,143.0110,143.0111,143.0120,143.0121,143.0130,143.0131,143.0140,143.0141,143.0150,143.0151,143.0161,143.0171,143.0181,143.0191,143.0201,143.0211,143.0221,143.0231,143.0241,143.0251,143.0261,143.0271,143.0281,143.0291,143.0301

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45055/40365

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

  • Configuration/PyReleaseValidation (pdmv, upgrade)

@sunilUIET, @AdrianoDee, @subirsarkar, @srimanob, @miquork, @cmsbuild can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @slomeo, @makortel, @fabiocos, @missirol this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@AdrianoDee
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-016c90/39543/summary.html
COMMIT: d040c0f
CMSSW: CMSSW_14_1_X_2024-05-27-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45055/39543/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

The relvals timed out after 4 hours.

----- Begin Fatal Exception 27-May-2024 17:18:31 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024C/MuonEG/RAW/v1/000/380/049/00000/02ccb064-fb1b-4042-9449-0ebf9b2118ec.root?tried=+1213llrxrd-redir.in2p3.fr,&xrdcl.requuid=6501fbba-46b2-40bb-a3ad-e500e516a3b8
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 27-May-2024 17:18:33 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024C/BTagMu/RAW/v1/000/380/049/00000/0023904f-d84c-4d4d-b230-9858791afb81.root?tried=+1213xrootd-cms-redir-int.cr.cnaf.infn.it,&xrdcl.requuid=d8e00365-3d23-40da-b5f8-0816ee8d4f15
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 27-May-2024 17:18:34 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024C/DisplacedJet/RAW/v1/000/380/049/00000/009b79a4-1719-4984-9e56-75c51e500694.root?tried=+1213xrootd-cms-redir-int.cr.cnaf.infn.it,&xrdcl.requuid=003f9d3b-3f9d-4d08-a69a-654a395ec995
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 143.001143.001_RunBTagMu2024C/step2_RunBTagMu2024C.log
  • 143.003143.003_RunDisplacedJet2024C/step2_RunDisplacedJet2024C.log
  • 143.007143.007_RunMuonEG2024C/step2_RunMuonEG2024C.log
Expand to see more relval errors ...

@AdrianoDee
Copy link
Contributor Author

test parameters:

  • workflows = 143.0011,143.0021,143.0031,143.0041,143.0051,143.0061,143.0071,143.0081,143.0091,143.0101,143.0111,143.0121,143.0131,143.0141,143.0151,143.0161,143.0171,143.0181,143.0191,143.0201,143.0211,143.0221,143.0231,143.0241,143.0251,143.0261,143.0271,143.0281,143.0291,143.0301

@AdrianoDee
Copy link
Contributor Author

please test
(running only with the 100k wfs for which I have copied the samples at T2_CH_CERN)

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-016c90/39563/summary.html
COMMIT: d040c0f
CMSSW: CMSSW_14_1_X_2024-05-27-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45055/39563/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals

The relvals timed out after 4 hours.

----- Begin Fatal Exception 28-May-2024 12:10:37 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024C/ParkingLLP/RAW/v1/000/380/049/00000/08bbd238-ac10-41e4-b192-818b4a07e7a3.root?tried=+1213xrootd-redic.pi.infn.it,&xrdcl.requuid=5aea3c1c-c8fa-402b-be6b-329054739bfe
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 28-May-2024 12:10:38 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024D/EGamma0/RAW/v1/000/380/310/00000/112e3f5b-2543-4b35-aca4-fe2d37f3fd40.root?tried=+1213xrootd-redic.pi.infn.it,&xrdcl.requuid=95aabfe7-3b5b-4b6d-b3f1-850e8d7174c6
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 28-May-2024 12:10:38 CEST-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://cms-xrd-global.cern.ch//eos/cms/store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root
      [b] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root
      [c] Input file root://xrootd-cms.infn.it//store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root could not be opened.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.
' (errno=3011, code=400). No additional data servers were found.
      [e] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2024C/ParkingHH/RAW/v1/000/380/049/00000/001f71ed-c672-470d-a202-2c437d6deead.root?tried=+1213xrootd-cms-redir-int.cr.cnaf.infn.it,&xrdcl.requuid=1d1c026a-f637-47e3-96be-1c0393fabb3e
      [f] Problematic data server: cms-xrd-global.cern.ch:1094
      [g] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 143.001143.001_RunBTagMu2024C/step2_RunBTagMu2024C.log
  • 143.003143.003_RunDisplacedJet2024C/step2_RunDisplacedJet2024C.log
  • 143.006143.006_RunMuon12024C/step2_RunMuon12024C.log
Expand to see more relval errors ...

@AdrianoDee
Copy link
Contributor Author

Ok, some files are missing a copy to T2_CH_CERN. Asked for the replicas. Will be done soon.

@AdrianoDee
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-016c90/40921/summary.html
COMMIT: 92aef4d
CMSSW: CMSSW_14_1_X_2024-08-14-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45055/40921/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 4 lines to the logs
  • Reco comparison results: 13 differences found in the comparisons
  • DQMHistoTests: Total files compared: 45
  • DQMHistoTests: Total histograms compared: 3422510
  • DQMHistoTests: Total failures: 15
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3422475
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
  • Checked 196 log files, 165 edm output root files, 45 DQM output files
  • TriggerResults: no differences found

@AdrianoDee
Copy link
Contributor Author

+pdmv

@srimanob
Copy link
Contributor

+Upgrade

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen
Copy link
Contributor

+1

@smuzaffar
Copy link
Contributor

@AdrianoDee , looks like after this update many relvals are failing for 14.1.X Ibs.

2024/08/21 06:36:20 ERROR DAS QL ERROR, query=file , lumi , run dataset = /Muon0/Run2024E-v1/RAW, idx=4, msg=unable to parse DAS query
Traceback (most recent call last):
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 934, in _finalize_columns_and_data
    columns = _validate_or_indexify_columns(contents, columns)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 981, in _validate_or_indexify_columns
    raise AssertionError(
AssertionError: 3 columns passed, passed data had 21 columns

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_14_1_X_2024-08-20-2300/bin/el8_amd64_gcc12/das-up-to-nevents.py", line 136, in <module>
    df = das_lumi_data(dataset).merge(das_file_data(dataset),on="file",how="inner") # merge file informations with run and lumis
  File "/data/cmsbld/jenkins/workspace/ib-run-relvals/CMSSW_14_1_X_2024-08-20-2300/bin/el8_amd64_gcc12/das-up-to-nevents.py", line 58, in das_lumi_data
    df = pd.DataFrame(out,columns=["file","run","lumis"])
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/frame.py", line 782, in __init__
    arrays, columns, index = nested_data_to_arrays(
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 498, in nested_data_to_arrays
    arrays, columns = to_arrays(data, columns, dtype=dtype)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 840, in to_arrays
    content, columns = _finalize_columns_and_data(arr, columns, dtype)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02851/el8_amd64_gcc12/external/py3-pandas/2.0.1-33b108e9d19def8c76b7c0615ce106ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 937, in _finalize_columns_and_data
    raise ValueError(err) from err
ValueError: 3 columns passed, passed data had 21 columns

@AdrianoDee
Copy link
Contributor Author

@smuzaffar let me have a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants