Automatic list with overviews of inlcuded area definitions for the documentation #2167

BENR0 · 2022-08-05T18:18:48Z

Based on pytroll/pyresample#450 this generates overviews for all AreaDefinitions included with Satpy and adds them to the resampling section.

I noticed that a quite large number of AreaDefinitions are included which make the section really long now.
Some areas are not included because only AreaDefinitions currently have _repr_html_ (see PR above).

Closes Add gallery of areas to documentation #1206
Fully documented

codecov · 2022-08-05T18:31:37Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.94%. Comparing base (ab55c4a) to head (269315c).
Report is 461 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2167      +/-   ##
==========================================
+ Coverage   95.91%   95.94%   +0.02%     
==========================================
  Files         366      366              
  Lines       53504    53504              
==========================================
+ Hits        51321    51332      +11     
+ Misses       2183     2172      -11

Flag	Coverage Δ
behaviourtests	`4.04% <ø> (+<0.01%)`	⬆️
unittests	`96.03% <ø> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

gerritholl · 2022-08-11T10:26:31Z

This looks interesting. How easy would it be for users to use this feature to generate overviews of areas from their own local YAML files?

mraspaud · 2022-08-11T11:49:16Z

doc/source/area_def_list.py

+def generate_area_def_list():
+    """Create list of available area definitions with overview plot.
+
+    Returns:
+        str
+    """
+    area_list = []
+
+    template = ("{area_name}\n"
+                "{n:->{header_title_length}}\n\n"
+                ".. raw:: html\n\n"
+                "{content}\n\n"
+                "     <hr>\n\n")
+
+    area_file = get_area_file()[0]
+    for aname in list(_read_yaml_area_file_content(area_file).keys()):
+        area = get_area_def(aname)
+        if hasattr(area, "_repr_html_"):
+            content = "\n".join([x.rjust(len(x) + 5) for x in area_repr(area, include_header=False).split("\n")])
+            area_list.append(template.format(area_name=aname, n="", header_title_length=len(aname),
+                                             content=content))
+        else:
+            pass


Doesn't this function rather belong to pyresample?

I think it doesn't really make sense to include in in pyresample because there is no area definition yaml included and this is only useful to generate a rst with a list of area definitions for the documentation.

It is true that pyresample does not have a default yaml file, but it is pyresample that contains all the functions for reading such yaml files. And also, I would argue that for the sake of separation of concerns, it should be included there.
In practice, I wouldn't be against having a default yaml file with a few generic areas in pyresample by the way, that would probably help potential pyresample users in the future...

Wouldn't areas.yaml fit better with pyresample entirely? But pyresample does not have a concept of SATPY_CONFIG_PATH or a config directory search, so it wouldn't know where to find areas.yaml anyway?

Not that anyone asked for my opinion, but I agree with wanting this in pyresample. I'm not a big fan of a default set of areas in pyresample except only as a few examples (like global lon/lat, one LCC, one polar-stereographic, etc)...generic areas. Pyresample is the low-level functionality, but it doesn't know anything about satellite-specific areas or that it is used for "science" at all so to me it is a fine line.

A while ago, I found a small list of generic areas, like europe, north and south america, etc. I'll try to find it again, so we can use that.

would this work? https://cfconventions.org/Data/standardized-region-list/standardized-region-list.current.html

As far as I see, those are just area names without defining anything AreaDefinitiony that we could use.

BENR0 · 2022-08-11T14:19:16Z

@gerritholl Curently not that easy I guess. But could easily be refactored to be available to users I think.

mraspaud · 2022-08-12T07:13:18Z

@gerritholl Curently not that easy I guess. But could easily be refactored to be available to users I think.

Especially if it was in pyresample 😉

BENR0 · 2022-08-12T10:48:57Z

@mraspaud @gerritholl ok so where should this go in pyresample? Should I include it in pytroll/pyresample#450 or make a separate PR?

coveralls · 2022-11-15T09:28:13Z

Coverage increased (+0.2%) to 94.948% when pulling 6edf540 on BENR0:feat_automatic_area_def_overview into 5cc09e0 on pytroll:main.

pnuu

LGTM. A screenshot would be nice :-)

BENR0 · 2022-11-18T10:12:03Z

Since this is based on the area html representation is has the same features (properties list/ foldable map).
So this is how it looks like:

Folded

Unfolded

pnuu · 2022-11-23T08:12:16Z

There are some problems with building the documentation, but they seem to be because pytroll/pyresample#450 needs to be merged and released first.

BENR0 · 2022-11-28T12:21:23Z

There are some problems with building the documentation, but they seem to be because pytroll/pyresample#450 needs to be merged and released first.

Yes correct. Based on some discussions and ideas I had I am working on making this more useable/pretty so please wait with merging this even if pytroll/pyresample#450 gets merged in the meantime.

BENR0 · 2022-12-05T14:03:10Z

Since there are quite a few AreaDefinitions in the yaml file I added a searchable table similar to the reader table with links to the definitions so the user does not have to scroll through the whole document. Currently this is still a bit messy since I just included the generation in the conf.py of the documentation for demonstration purposes. If this is wanted I will refactor this. It could be integrated into the generation function I already moved to pyresample but there on its own it would just generated a static table because the searching and sorting depends on the datatable css and js which is integrated in the satpy documentation.

Below you can see how it would look like. One think to note is that all the area definitions also appear in the toc tree which is not a huge deal but not that nice I guess. I think that has something to do with the toc maxdepth setting but I couldn't get that fixed somehow. So if somebody has an idea about that please share.

Apart from that I notice that there are two area definitions I think belong to DynamicAreaDefinitions?:

omerc_bb
laea_bb
Right now these produce broken links in the table because obviously there can be no overview generated for those.

mraspaud · 2023-02-02T09:46:41Z

Just as a reminder for whoever is looking for a status update, this is waiting for completion of pytroll/pyresample#450

coveralls · 2023-06-05T12:14:17Z

Pull Request Test Coverage Report for Build 9483462883

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.04%

Totals
Change from base Build 9418285216:	0.0%
Covered Lines:	51561
Relevant Lines:	53687

💛 - Coveralls

mraspaud · 2023-12-13T09:04:48Z

pytroll/pyresample#450 is now merged. Do we need a pyresample release before this can work?

BENR0 · 2023-12-13T09:22:30Z

I think so because if I see correctly read the docs uses a conda installed pyresample to build the docs.
While the table should build fine in either case it would be nice to have the area names in the yaml file fixed (even if there is no agreement on a file naming pattern yet) #2310.

gerritholl · 2023-12-14T11:37:26Z

You can add yourself to AUTHORS.md while you're at it :)

mraspaud

I think this can be merged, thanks for the adding it. The filling for water bodies is not always working, but that needs to be fixed in pyresample.

Benjamin Rösner and others added 5 commits August 1, 2022 07:50

feat: add first draft of area list

e52fd96

fix: indentation of raw html block

96ae5d8

add: area_def_list.rst to gitignore

bdf89b7

add: area definition list to resample chapter

e310747

Merge branch 'main' into feat_automatic_area_def_overview

7f41639

BENR0 requested review from pnuu, djhoese and mraspaud as code owners August 5, 2022 18:18

mraspaud reviewed Aug 11, 2022

View reviewed changes

BENR0 added 4 commits November 15, 2022 09:27

refactor: move rst list generation function to pyresample

d7893d5

Merge branch 'main' into feat_automatic_area_def_overview

d18023b

fix: missing area file argument

d621089

fix: function name

4b94939

pnuu approved these changes Nov 18, 2022

View reviewed changes

BENR0 added 2 commits December 5, 2022 14:14

refactor: add class name argument to rst table header generation

fd0879d

refactor: add options for area table to datatable js init

6edf540

BENR0 mentioned this pull request Dec 5, 2022

Naming of Area definitions in yaml file #2310

Draft

mraspaud assigned BENR0 Feb 2, 2023

mraspaud added the enhancement code enhancements, features, improvements label Feb 2, 2023

BENR0 added 3 commits June 5, 2023 11:45

Merge branch 'main' into feat_automatic_area_def_overview

6a35d2f

feat: add searchable reader table

b1bacff

fix: areas yaml

f766457

BENR0 mentioned this pull request Jun 19, 2023

Area definition html representation for Jupyter notebooks pytroll/pyresample#450

Merged

4 tasks

BENR0 added 7 commits June 12, 2024 10:33

Merge branch 'main' into feat_automatic_area_def_overview

bd960ca

fix: error when projection is EPSG code

23f6ae5

fix: links for areas without html repr.

ba46aca

fix: relativ inlcude

fbb92b1

add cartopy to environment

79d95b6

fix: columns widths

f9142b0

remove testing code

269315c

mraspaud approved these changes Jun 13, 2024

View reviewed changes

mraspaud merged commit 6ed8698 into pytroll:main Jun 13, 2024
18 of 19 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automatic list with overviews of inlcuded area definitions for the documentation #2167

Automatic list with overviews of inlcuded area definitions for the documentation #2167

BENR0 commented Aug 5, 2022 •

edited

Loading

codecov bot commented Aug 5, 2022 •

edited

Loading

gerritholl commented Aug 11, 2022

mraspaud Aug 11, 2022

BENR0 Aug 11, 2022

mraspaud Aug 12, 2022

gerritholl Aug 12, 2022

djhoese Aug 14, 2022

mraspaud Aug 15, 2022

mraspaud Nov 9, 2022

pnuu Nov 9, 2022

BENR0 commented Aug 11, 2022

mraspaud commented Aug 12, 2022 •

edited

Loading

BENR0 commented Aug 12, 2022

coveralls commented Nov 15, 2022 •

edited

Loading

pnuu left a comment

BENR0 commented Nov 18, 2022 •

edited

Loading

pnuu commented Nov 23, 2022 •

edited

Loading

BENR0 commented Nov 28, 2022

BENR0 commented Dec 5, 2022

mraspaud commented Feb 2, 2023

coveralls commented Jun 5, 2023 •

edited

Loading

mraspaud commented Dec 13, 2023

BENR0 commented Dec 13, 2023

gerritholl commented Dec 14, 2023

mraspaud left a comment

Automatic list with overviews of inlcuded area definitions for the documentation #2167

Automatic list with overviews of inlcuded area definitions for the documentation #2167

Conversation

BENR0 commented Aug 5, 2022 • edited Loading

codecov bot commented Aug 5, 2022 • edited Loading

Codecov Report

gerritholl commented Aug 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BENR0 commented Aug 11, 2022

mraspaud commented Aug 12, 2022 • edited Loading

BENR0 commented Aug 12, 2022

coveralls commented Nov 15, 2022 • edited Loading

pnuu left a comment

Choose a reason for hiding this comment

BENR0 commented Nov 18, 2022 • edited Loading

Folded

Unfolded

pnuu commented Nov 23, 2022 • edited Loading

BENR0 commented Nov 28, 2022

BENR0 commented Dec 5, 2022

mraspaud commented Feb 2, 2023

coveralls commented Jun 5, 2023 • edited Loading

Pull Request Test Coverage Report for Build 9483462883

Details

💛 - Coveralls

mraspaud commented Dec 13, 2023

BENR0 commented Dec 13, 2023

gerritholl commented Dec 14, 2023

mraspaud left a comment

Choose a reason for hiding this comment

BENR0 commented Aug 5, 2022 •

edited

Loading

codecov bot commented Aug 5, 2022 •

edited

Loading

mraspaud commented Aug 12, 2022 •

edited

Loading

coveralls commented Nov 15, 2022 •

edited

Loading

BENR0 commented Nov 18, 2022 •

edited

Loading

pnuu commented Nov 23, 2022 •

edited

Loading

coveralls commented Jun 5, 2023 •

edited

Loading