Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save simulation output #145

Merged
merged 8 commits into from
May 23, 2022
Merged

Save simulation output #145

merged 8 commits into from
May 23, 2022

Conversation

wigging
Copy link
Collaborator

@wigging wigging commented Feb 25, 2022

Added some utility functions to save the simulation output to CSV, NumPy .npy, and compressed NumPy .npz files. Also added an example of using the save functions.

I did a comparison of the disk usage for these different file formats (see below). The compressed .npz format can save a lot of storage space for large pack simulations. There are other file formats such as Feather and Parquet but using those formats would introduce more dependencies to liionpack.

Below are disk storage results for a 16p2s single charge/discharge simulation. Simulation outputs written to file are time, terminal voltage, pack terminal voltage, pack current, open circuit voltage, cell resistance, and cell current.

Filetype Disk space
csv 552 KB
npy 188 KB
npz 88 KB

Below are disk storage results for a 400p125s single charge/discharge simulation. Simulation outputs written to file are time, terminal voltage, pack terminal voltage, pack current, open circuit voltage, cell resistance, and cell current.

Filetype Disk space
csv 803 MB
npy 246 MB
npz 33 MB

@wigging wigging linked an issue Feb 25, 2022 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Feb 25, 2022

Codecov Report

Merging #145 (fb555eb) into main (a7d8b02) will increase coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #145      +/-   ##
==========================================
+ Coverage   97.12%   97.18%   +0.05%     
==========================================
  Files          11       11              
  Lines        1184     1206      +22     
==========================================
+ Hits         1150     1172      +22     
  Misses         34       34              

@TomTranter
Copy link
Collaborator

@wigging thanks looks good but can you add tests?

@wigging
Copy link
Collaborator Author

wigging commented Feb 26, 2022

Sure, but how do you make a unit test for a function that creates a folder and writes to files?

@TomTranter
Copy link
Collaborator

Sure, but how do you make a unit test for a function that creates a folder and writes to files?

You can use os.path.isfile() to assert that a file is created

@wigging
Copy link
Collaborator Author

wigging commented Feb 28, 2022

Sir @TomTranter, I uploaded some tests. Let me know what else is needed. Regarding the online documentation, will the function docstrings be used to update the docs or do I need to manually update the documentation?

@wigging
Copy link
Collaborator Author

wigging commented May 18, 2022

If this pull request is not useful for liionpack then please close the request. Otherwise, let me know what else needs to be done before accepting the request.

@TomTranter
Copy link
Collaborator

@wigging sorry I haven't got round to looking at the code yet. Will try to asap.

@TomTranter
Copy link
Collaborator

This is a good start and I'm happy to merge. Some thoughts would be we should add headers and a time column to the csv. Think about using hdf5 as well. Also we should add some options to do journaling for the solve process but this may be better to save the simulation objects themselves rather than output variables. Worth thinking about some more

@wigging
Copy link
Collaborator Author

wigging commented May 20, 2022

The csv function saves each output item to its own csv file into a csv-results folder. This includes the time output which is Time_[s]. Similarly, the npy function saves each output item to its own npy file. The npz function saves the output into one npz file.

For saving to csv, I guess it would be better to save everything into one csv file instead of separate files.

For the NumPy formats, I think I'll just use the npz format and remove the ability to save to npy. I don't see any benefits to use npy instead of npz.

Supporting hdf5 output would add another dependency to the project. Liionpack already uses NumPy so not a big deal to save output in a NumPy format.

@wigging
Copy link
Collaborator Author

wigging commented May 20, 2022

It's been awhile since I ran the code in this pull request. Disregard my comment about saving to a single csv file. The csv function saves to multiple files because some of the outputs are 2D arrays. So it's not feasible to save all the output into a single csv file.

@TomTranter TomTranter merged commit 340d67f into main May 23, 2022
@TomTranter TomTranter deleted the save-output branch May 23, 2022 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Post-processing of the data
2 participants