-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enables production data from CSV file and fixes failing probability distribution test #425
Merged
olelod
merged 9 commits into
equinor:master
from
edubarrosTNO:i382-allow-datasource-file
Aug 6, 2021
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
fdc83fb
Enables production data from CSV file
edubarrosTNO b54e4f0
Adds config_parser changes and fixes black formatting
edubarrosTNO 4c526be
Fixes pylint
edubarrosTNO 8119f3b
Fixes description CSVData class
edubarrosTNO e52c2d7
Fixes probability distributions test
edubarrosTNO b1086fa
Updates documentation and CHANGELOG
edubarrosTNO 506c933
Changes position of observation vectors entry in config file
edubarrosTNO 982acf4
Changes CI testdata branch for CI
edubarrosTNO 18c98a7
Reverts CI back to equinor/master testdata branch and updates CHANGELOG
edubarrosTNO File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| from ..data import from_source | ||
|
|
||
| from ..data.from_flow import FromSource, FlowData | ||
| from ..data.from_csv import CSVData |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| from pathlib import Path | ||
| from typing import Union | ||
|
|
||
| import pandas as pd | ||
|
|
||
|
|
||
| class CSVData: | ||
| """ | ||
| CSV data source class | ||
|
|
||
| Args: | ||
| input_data: Full path to CSV file to load production data from | ||
|
|
||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| input_data: Union[Path, str], | ||
| ): | ||
| super().__init__() | ||
|
|
||
| self._input_data: Path = Path(input_data) | ||
|
|
||
| # pylint: disable=too-many-branches | ||
| def _production_data(self) -> pd.DataFrame: | ||
| """ | ||
| Function to read production data for all producers and injectors from a CSV file. | ||
|
|
||
| Returns: | ||
| A DataFrame with a DateTimeIndex and the following columns: | ||
| - date equal to index | ||
| - WELL_NAME Well name as used in Flow | ||
| - WOPR Well Oil Production Rate | ||
| - WGPR Well Gas Production Rate | ||
| - WWPR Well Water Production Rate | ||
| - WOPT Well Cumulative Oil Production | ||
| - WGPT Well Cumulative Gas Production | ||
| - WWPT Well Cumulative Water Production | ||
| - WBHP Well Bottom Hole Pressure | ||
| - WTHP Well Tubing Head Pressure | ||
| - WGIR Well Gas Injection Rate | ||
| - WWIR Well Water Injection Rate | ||
| - WSPR Well Salt Production Rate | ||
| - WSIR Well Salt Injection Rate | ||
| - WSPT Well Cumulative Salt Production | ||
| - WSIT Well Cumulative Salt Injection | ||
| - WSTAT Well status (OPEN, SHUT, STOP) | ||
| - TYPE Well Type: "OP", "GP", "WI", "GI" | ||
| - PHASE Main producing/injecting phase fluid: "OIL", "GAS", "WATER" | ||
|
|
||
| Todo: | ||
| * Remove depreciation warning suppression when solved in LibEcl. | ||
| * Improve robustness pf setting of Phase and Type. | ||
|
|
||
| """ | ||
| df_production_data = pd.read_csv(self._input_data) | ||
| df_production_data["date"] = pd.to_datetime(df_production_data["date"]).dt.date | ||
| df_production_data = df_production_data.set_index("date", drop=False) | ||
| return df_production_data | ||
|
|
||
| @property | ||
| def production(self) -> pd.DataFrame: | ||
| """dataframe with all production data""" | ||
| return self._production_data() | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now the function will return the columns present in the csv file (if e.g. WGPR is not in the csv file it will not be returned with NaN's as I think it will in from_flow.py). I'm not sure if the code needs all vectors, so we may not need NaN's, but there should the docstring should be consistent with the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if I understand the problem or how to solve it. Are you saying that we should check if all supported summary vectors are present in the CSV file? If so, is there a list of all supported summary vectors anywhere in the code?