Save any default CPT for remote datasets in hidden grid header #6178

PaulWessel · 2022-01-02T01:51:52Z

This addresses #6167 when we cut a subset from, say, @earth_relief_10m and then no longer know what the CPT should be. it is now added as a netCDF z-variable attribute and can thus be found by GMT as well as other environments. A new test (defect.sh) have been added that shows it is now working.
Two older tests (polycut.sh and highpass.sh) were changed to explicitly use turbo so that they would not fail due to this improvement. Closes #6167.

This addresses #6167 when we cut a subset from, sa, earth_relief_10m and then no longer know what the CPT should be.

joa-quim · 2022-01-02T13:35:29Z

src/gmt_nc.c

@@ -1029,6 +1032,8 @@ GMT_LOCAL int gmtnc_grd_info (struct GMT_CTRL *GMT, struct GMT_GRID_HEADER *head

 		/* Define z variable. Attempt to remove "scale_factor" or "add_offset" when no longer needed */
 		gmtnc_put_units (ncid, z_id, header->z_units);
+		if (GMT->parent->remote_info && GMT->parent->remote_id != GMT_NOTSET && GMT->parent->remote_info[GMT->parent->remote_id].CPT[0] != '-')	/* Subset of remote grid with default CPT, save name as an attribute */
+			gmt_M_err_trap (nc_put_att_text (ncid, z_id, "cpt", strlen (GMT->parent->remote_info[GMT->parent->remote_id].CPT), GMT->parent->remote_info[GMT->parent->remote_id].CPT));


"cpt" is a very GMT parlance. I would prefer "palette" like those grids from OceanColor

I don't mind calling it palette, but since it is just a string with a GMT CPT name it is still very GMT centric. We are not sticking a 3xn array into the grid, just a string. Still want to rename it?

Ok, sorry. I thought that it was the cpt that was added to the nc file and that would be an interesting addition. If it's just a cpt name suggestion for GMT scripts then I'm not even sure we should be doing this.

Nothing to do with GMT scripts though. This is for the module and hence for wrappers as well if interested. Since I think you pass wrapped GMT grids in Julia then this will work for you off the bat. PyGMT will need to do some other work since they read in a netcdf grid via Xarrays, so they have access to the attributes that way.

We could of course stick a CPT into the netCDF but it has little value in GMT (we would do it because the outside world wants it) and it is limiting in that 3xn matrices do not tell us continuous or discrete and now we have to add more stuff to explain that thing.

I am not exploring the hidden struct because there is no guaranties that it won't change and then each change would mean a Julia breakage. I've pulled out some hairs in the past trying to inventing a system where the structs could have some self-referencing system that could be used to find the struct members of interest regardless of structs changes. It would involve the use of offsetof C macro. But never came to get any workable prototype.

Yes, if you do not pass any default CPT but let GMT control the default CPT then it would work. Are you supplying a different default CPT than GMT would normally (turbo?). if not then just don't supply one.

I think this is a worthwhile improvement for the CLI, regardless of usability for the wrappers. While PyGMT can easily access the cpt attr, I'm not sure how it would communicate it back through the API during the GMT_Create_Data, GMT_Put_Matrix, GMT_Open_VirtualFile process.

GMT_Set_Comment is only for the 'top-level' grid header (those listed here), correct? Could anything be added to the API for setting the items that are mostly used internally (i.e., full list here)?

I don't think that is the step you will want to intervene. If you learn there is a cpt attribute then you add "-C" to the string you prepare for the module, no?

Yes, we can do that. If that is the case, we may want to eventually document that users can add a cpt attr to their netcdf grid in order to associate a default colormap with their own grids. In which case Joaquim's comment about using palette over cpt becomes relevant again.

I suppose grdedit is the module where it makes sense for us to add an attribute to the grid file. Something like

grdedit my.grd -Qelevation

This brings up the issue of "I want my specially made cpt file to be embedded in the grid", to which I suggest someone read the gmtmex_parser.c on the gymnastics on converting a very rich CPT standard into a very poor Matlab color table scheme. It would be the same issue - lots of work to accommodate all the special things in a CPT, vs just adding the name of the cpt.

joa-quim · 2022-01-02T13:41:45Z

test/grdimage/defcpt.sh

+		gmt grdimage @earth_relief_01d_g -R0/10/0/10 -B -c
+		gmt grdimage test.nc -B -c
+	gmt subplot end
+gmt end show


Here we are again creating a new PS. Preferably the test should detect the presence of the cpt but if that's too complicated, at least make the test a tinny figure.

I just updated to remove the boundary. It is a 10x10 grid - that is pretty small, no? The size of the PS is 22K, og which 20K is the PSL header.

I do not see how I can make a non-plot test to detect that GMT correctly identifies the attribute and stretches the CPT, etc.

I like my idea though: It complements the attribute idea since that solution will survive a session whereas the current CPT has a limited scope in where it is found (figure, subplot, inset), etc.

maxrjones

The current implementation is fine to me.

PaulWessel · 2022-01-03T20:48:07Z

A slight issue has arisen: We have test and doc scripts that sets a default CPT via makecpt, then cuts a subset via grdcut afterwards. Now, that is changing the current CPT. In some places I can switch the order of those to commands, but i others we set a current cpt before a subplot and then we have grdcut commands inside those panels and now the CPT takes on the one in the grid since the grdcut command forces the resetting of the current CPT.

Two options:

Undo this feature in grdcut. After all, it was added in haste yesterday to help with externals by setting the current CPT but it remains unclear if this even affects GMT.jl and it seems PyGMT can easily detect a default CPT via array attributes.
Add a modifier to set any detected attribute cpt as the current CPT

Any thoughts on this, @joa-quim and @meghanrjones? Note that GMT modules etc has no need for the grdcut feature so I am OK with option 1, but willing to do option 2 if there is any desire to keep this feature for the externals.

maxrjones · 2022-01-03T20:53:01Z

Option 1 would still leave the cpt attr and use that as the default if there isn't already a colormap set in the session, correct? If so, I think that's fine.

PaulWessel · 2022-01-03T20:55:00Z

Yes, the attribute in the written grid remains, it is just the whole section

if (GMT->current.setting.run_mode == GMT_MODERN && HH->cpt) {	/* Save the CPT as the current CPT for a modern mode session */
	struct GMT_PALETTE *P = gmt_get_palette (GMT, HH->cpt, GMT_CPT_OPTIONAL, G->header->z_min, G->header->z_max, 0.0);
	if (GMT_Destroy_Data (API, &P) != GMT_NOERROR)
		Return (API->error);
}

that will be rescinded due to unintended consequences...

maxrjones · 2022-01-03T20:55:18Z

Sounds fine.

joa-quim · 2022-01-03T20:58:58Z

Yes ls agree to let it there to be used if expressly stated and not a setting that happens by default.

PaulWessel · 2022-01-03T21:11:19Z

Yes ls agree to let it there to be used if expressly stated and not a setting that happens by default.

Could you clarify this: Are you voting to add an option to grdcut or do you agree with Meghan?

joa-quim · 2022-01-03T21:12:51Z

Agree with Meghan.

PaulWessel added 10 commits December 31, 2021 08:28

Save any default CPT in hidden grid header

980792a

This addresses #6167 when we cut a subset from, sa, earth_relief_10m and then no longer know what the CPT should be.

Merge branch 'master' into save-def-cpt-grd

44d9be0

use allocated string cpt instead

587c266

Init remote_id to -1

b40d4bc

polish

a4f4208

Fix save CPT and highpass

f4aa0e3

Update gmt_support.c

9423f09

Merge branch 'master' into save-def-cpt-grd

757c229

Add small tests

3500516

formatting

875fca4

PaulWessel added the enhancement Improving an existing feature label Jan 2, 2022

PaulWessel added this to the 6.4.0 milestone Jan 2, 2022

PaulWessel requested review from joa-quim and maxrjones January 2, 2022 01:51

PaulWessel self-assigned this Jan 2, 2022

vercel bot temporarily deployed to Preview January 2, 2022 01:52 Inactive

joa-quim reviewed Jan 2, 2022

View reviewed changes

Remove boundary annotations

d646ae9

vercel bot temporarily deployed to Preview January 2, 2022 19:55 Inactive

Save the current CPT from grdcut if modern mode

7134bad

vercel bot temporarily deployed to Preview January 3, 2022 02:40 Inactive

maxrjones approved these changes Jan 3, 2022

View reviewed changes

PaulWessel merged commit e729a9b into master Jan 3, 2022

PaulWessel deleted the save-def-cpt-grd branch January 3, 2022 20:17

PaulWessel mentioned this pull request Jan 3, 2022

Remove the unwanted setting of CPT in grdcut #6187

Merged

maxrjones added the add-changelog Add PR to the changelog label Jan 25, 2022

maxrjones mentioned this pull request Feb 4, 2023

Default CPTs for GMT remote datasets GenericMappingTools/pygmt#2325

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Save any default CPT for remote datasets in hidden grid header #6178

Save any default CPT for remote datasets in hidden grid header #6178

PaulWessel commented Jan 2, 2022 •

edited

Loading

joa-quim Jan 2, 2022

PaulWessel Jan 2, 2022

joa-quim Jan 3, 2022

PaulWessel Jan 3, 2022

joa-quim Jan 3, 2022

PaulWessel Jan 3, 2022

maxrjones Jan 3, 2022

PaulWessel Jan 3, 2022

maxrjones Jan 3, 2022

PaulWessel Jan 3, 2022

joa-quim Jan 2, 2022

PaulWessel Jan 2, 2022 •

edited

Loading

PaulWessel Jan 3, 2022

PaulWessel Jan 3, 2022

maxrjones left a comment

PaulWessel commented Jan 3, 2022

maxrjones commented Jan 3, 2022

PaulWessel commented Jan 3, 2022

maxrjones commented Jan 3, 2022

joa-quim commented Jan 3, 2022

PaulWessel commented Jan 3, 2022

joa-quim commented Jan 3, 2022

Save any default CPT for remote datasets in hidden grid header #6178

Save any default CPT for remote datasets in hidden grid header #6178

Conversation

PaulWessel commented Jan 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PaulWessel Jan 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxrjones left a comment

Choose a reason for hiding this comment

PaulWessel commented Jan 3, 2022

maxrjones commented Jan 3, 2022

PaulWessel commented Jan 3, 2022

maxrjones commented Jan 3, 2022

joa-quim commented Jan 3, 2022

PaulWessel commented Jan 3, 2022

joa-quim commented Jan 3, 2022

PaulWessel commented Jan 2, 2022 •

edited

Loading

PaulWessel Jan 2, 2022 •

edited

Loading