Combined Plot of Raw and Cleaned CO₂ Data in the Time Domain #49
Conversation
- Loaded and processed CO₂ data from Mauna Loa Observatory (flask_monthly.json). - Applied waveform modifications including: - Padding with reflective boundaries to minimize edge effects. - Windowing using a Hann window to smooth transition edges. - FFT to transform the time-series data to the frequency domain. - Noise removal by filtering low-magnitude frequencies based on a set threshold. - Inverse FFT to revert data back to the time domain after noise reduction. - Undoing the windowing and padding effects to maintain data integrity. - Generated a single combined plot displaying: - Raw CO₂ data with seasonal oscillations (gray line). - Smoothed, cleaned CO₂ data highlighting long-term trends (blue line). - Verified that the cleaned data curve retains the overall upward trend in CO₂ concentration while reducing noise and oscillatory components, per task requirements. This commit completes the time domain analysis and visualization task as specified.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
@dnxjay As per homework description only one Jupyter notebook should be used since you are the first one you can choose the name, so please choose something more generic that fits for all plotting tasks. |
|
@ubsuny/cp1-24 Who wants to review this? |
dnxjay
modified the milestones:
Plot power spectrum C02 Continents,
Plot of power spectrum Mount Lua
Okay will do |
|
I can review this. |
There was a problem hiding this comment.
The plot is clean! And the notebook used to produce it looks nice as well.
When I looked at your PR in a codespace, it looked like the preparation.py was an old version. I'm not sure if that will impact the current version should we merge this PR. Is your codespace that you're working with up to date with the main repository?
The only minor addition I might suggest is some documentation on how the data cleaning works and some benefits of using the cleaned data rather than the raw data.
For instance, we remove the noise by removing low frequencies from the FFT, but that also removes the seasonal oscillations. Is that something we want? Is that something we can adjust for by rescaling the threshold for low frequencies?
Overall it looks good! My main concern is the old version of the preparation.py file, but this might be a non-issue.
| "# Load the JSON data with the correct path\n", | ||
| "data = pd.read_json('/workspaces/CP1-24-HW5/mauna-loa-data/flask_monthly.json')\n", | ||
| "\n", | ||
| "# Correct column names to match 'Year' and 'Month' from the JSON data\n", |
There was a problem hiding this comment.
Good flow of correcting columns --> padding and windowing --> FFT --> Noise Removal --> invFFT as laid out in the task description.
|
@iglesias-cardinale please mark as approved or rejected, not just comment since otherwise it is unclear if this can gets merged. @dnxjay Filename is still an issue |
|
@iglesias-cardinale @dnxjay Also I just noticed your task was to plot the power spectrum as you mentioned yourself in #51 |
There was a problem hiding this comment.
I'm approving so we can get this merged, but we should update the ipynb filename to something more general. The most general name I can think of would be something like CP1-24-HW5-notebook.ipynb. That way there's no confusion about the purpose of the file.
I'm also still concerned about merging since the preparation.py file seems out of date. It's possible that I'm missing something, but I think the codespace you're working on should be synced before merging.
|
It would be nice to generalize the filename |
The first PR that is merged defines it, but it would be good to have a generic one as suggested in PR #52 |
yes i had issues using the preparation.py which is why it wasn't implemented, my codespace was updated yesterday when i did the PR |
|
what am i missing to get this merged? @laserlab @iglesias-cardinale |
This pull request includes the time domain analysis and visualization of CO₂ data from the Mauna Loa Observatory. The analysis involves data processing techniques to clean and plot the data, showing both the raw and smoothed trends.
The code passes linting checks.
Files Changed:
co2_analysis_notebook.ipynb (Updated with data processing code and plot)
/workspaces/CP1-24-HW5/mauna-loa-data/flask_monthly.json (Read for CO₂ data analysis)
This assignment task required the creation of a single plot combining the raw and cleaned CO₂ data in the time domain, with data processed using waveform modification techniques. Techniques used include padding, windowing, FFT, noise removal, inverse FFT, and undoing the window+padding.