BRT

[gottacatchenall]

It's BRT, but real this time

[gottacatchenall]

@jmlord This almost works, but we are having issues with permissions where we can't write to the /tmp dir, which ArchGDAL.jl has to do to save geotiffs

[jmlord]

we can't write to the /tmp dir, which ArchGDAL.jl has to do to save geotiffs

Let me know if 5e13462 solves your problems.
Building here: https://github.com/GEO-BON/bon-in-a-box-pipelines/actions/runs/10962572212

[tpoisot]

Do we want to keep the precompile step since every pipeline is running in its own environment?

[gottacatchenall]

long story shart: everything is precompiled in /julia_depot for now, and the main script here activates /julia_depot as the project dir

[tpoisot]

When this works I can also add a quick bootstrap uncertainty - @jmlord / @JoryGriffith what do you think?

[gottacatchenall]

this makes a half decent map with the parameters below. (you'll have to run ./dev-server build from the pipeline engine, it takes like 10 mins but only has to run once)

input yaml

data>GBIFHeatmapFromSTAC.yml@130|taxa: plants
data>loadFromStac.yml@119|mask: null
data>loadFromStac.yml@119|stac_url: https://stac.geobon.org
data>loadFromStac.yml@139|bbox:
  - -2316297
  - -1971146
  - 1015207
  - 1511916
data>loadFromStac.yml@139|mask: null
data>loadFromStac.yml@139|stac_url: https://stac.geobon.org/
data>loadFromStac.yml@139|weight_matrix_with_ids: |
  layer, current, change
  GBSTAC|chelsa-clim|bio1, 0.7, 0.6
  GBSTAC|chelsa-clim|bio2, 0.3, 0.4
data>pyLoadObservations>pyLoadObservations.yml@96|data_source: gbif_api
data>pyLoadObservations>pyLoadObservations.yml@96|max_year: 2020
data>pyLoadObservations>pyLoadObservations.yml@96|min_year: 2010
pipeline@121:
  - Acer saccharum
pipeline@128: 1000
pipeline@137:
  - chelsa-clim|bio1
  - chelsa-clim|bio3
  - chelsa-clim|bio4
  - chelsa-clim|bio12
  - chelsa-clim|bio15
pipeline@57: EPSG:6622
pipeline@58:
  - -2316297
  - -1971146
  - 1015207
  - 1511916

[gottacatchenall]

@jmlord This almost works, but we are having issues with permissions where we can't write to the /tmp dir, which ArchGDAL.jl has to do to save geotiffs

the issue with not being able to write tiffs because of permissions issues was this nested write call.

Everything works the same (afaict) not returning ArchGDAL.write but just using write! to burn into the raster band and returning, no idea why this specifically was a permissions problem

[tpoisot]

the issue with not being able to write tiffs because of permissions issues was this nested write call.

Everything works the same (afaict) not returning ArchGDAL.write but just using write! to burn into the raster band and returning, no idea why this specifically was a permissions problem

🤷🏻 ArchGDAL, true to the Arch part of its name!

Can you open a PR with a fix?

[gottacatchenall]

yar PoisotLab/SpeciesDistributionToolkit.jl#277

[jmlord]

Is this a leftover that we can now remove, or was it intended to be plugged somewhere?

[gottacatchenall]

yeah that isn't used

[jmlord]

Great, could you please remove it and document it more thoroughly so that we can send this to an external reviewer?

ex:
tuning curve --> What does it mean, guidelines for interpretation?
range map thresholded at todo --> seems unfinished

A clear and concise description would be necessary for all IO. Remember that we now support markdown in descriptions if you feel it needs more.

Once you update the script yml, just remove and add back the output boxes and save it to update the pipeline json file with the new desc.

FYI I ran it and it went fine here.

[jmlord]

When this works I can also add a quick bootstrap uncertainty - @jmlord / @JoryGriffith what do you think?

I think that could go in another PR, since we were thinking to officially open a review process tomorrow:

• opening the review process Friday 4th (tomorrow), closing it Friday 11th.
• publishing the pull request on Discourse to make it more transparent
• sending specific invitations for reviewer (Jamie, Laura, etc.)

[gottacatchenall]

Rebased and it looks like it works now, just need to clean up the documentation a bit

[tpoisot]

We're likely going to be hitting JuliaGeo/GDAL.jl#179 at some point - so if the runs fails in the next days, that's why

[jmlord]

We're likely going to be hitting JuliaGeo/GDAL.jl#179 at some point - so if the runs fails in the next days, that's why

Can we freeze the dependency version?

[tpoisot]

yeah, we can also temporarily add LERC_jll@3and it seems to work - but maybe 1.10 isn't affected, and I have good hope that the GDAL_jll maintainers will actually solve this since it's affecting everyone

[gottacatchenall]

Added docs here and here

[glaroc]

@gottacatchenall Can you please change the output fit statistics to be a json file, as opposed to a json object inside the output.json? We haven't yet implemented viewing json objects directly in the output.json file.

[gottacatchenall]

@gottacatchenall Can you please change the output fit statistics to be a json file, as opposed to a json object inside the output.json? We haven't yet implemented viewing json objects directly in the output.json file.

@glaroc Are you fully up to date? It already does afaict.

bon-in-a-box-pipelines/scripts/SDM/BRT/io.jl

Line 77 in 715cfb0

open(output_json_path, "w") do f

[glaroc]

I tested it and all looks fine on my side.

[jmlord]

@jamiemkass This is the new SDM pipeline I was mentioning.

[tpoisot]

👋🏻 Hi @jamiemkass — this is a first working-but-rough draft of the pipeline, so we're keen to hear your thoughts!