Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for capturing data for any variable at runtime #105

Closed
2 tasks
chrispcampbell opened this issue Aug 27, 2021 · 1 comment · Fixed by #355 or #360
Closed
2 tasks

Add support for capturing data for any variable at runtime #105

chrispcampbell opened this issue Aug 27, 2021 · 1 comment · Fixed by #355 or #360
Assignees
Labels

Comments

@chrispcampbell
Copy link
Contributor

chrispcampbell commented Aug 27, 2021

When a spec.json file is used to expose a subset of output variables, only those specified variables can be accessed as outputs. For a testing tool we are working on internally, we would like to be able to inspect the data for any variable in the model, not just the ones exposed as outputs. The En-ROADS model uses thousands of variables; if we listed them all as outputs, it would use a lot of memory and would slow down the model. It would be preferable to have a way at runtime to select the variables for which to capture data into the output buffer.

I'm envisioning a few parts to this:

  1. Add a new sde generate --listjson option that outputs a JSON file containing information about all variables and subscripts used in the model. (This would also be useful for a dependency tree tracking tool that we developed internally.)

  2. Accept a new outputIndices buffer parameter:

void runModelWithBuffers(double* inputs, double* outputs, int32_t* outputIndices) {
  1. Generate a new storeOutput function that includes a big switch statement, which will select the output (and subscripts) by index based on the values from the outputIndices buffer:
void storeOutput(size_t varIndex, size_t subIndex0, size_t subIndex1, size_t subIndex2) {
#define USE_OUTPUT_INDICES 0
#if USE_OUTPUT_INDICES
  switch (varIndex) {
    case 1:
      outputVar(_a);
      break;
    case 2:
      outputVar(_b);
      break;
    case 3:
      outputVar(_c[subIndex0][subIndex1]);
      break;
    // ...
    default:
      break;
  }
#end
}

The flag would be set to 0 by default so that there would be no impact on size/speed unless you opt-in to use this new output mode.

Things to revisit before finalizing this work:

  • Allow --genc and --list in the same command so that we don't have to run sde twice
  • Exclude variables from storeOutput if they have more than 3 dimensions (we only support variables with <= 3 dimensions for now)
@chrispcampbell chrispcampbell self-assigned this Aug 27, 2021
@chrispcampbell chrispcampbell changed the title Add support for accessing data for any variable at runtime Add support for capturing data for any variable at runtime Aug 27, 2021
@ToddFincannon
Copy link
Collaborator

That looks useful. Instead of adding a new --listjson option we could add JSON output to the --list option. It already outputs all variables and subscripts in text and YAML format. It could do JSON too, or JSON could replace YAML. I'm probably the only one who uses the YAML listing, for my dependency tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants