Add part 6 for household co-analysis and Circadian Rhythm analysis

[vincentvanhees]

In this PR I add new part 6 to facilitate Household co-analysis and recording level Circadian Rhythm analysis that depends on knowledge about sleep and wake. There is more functionality I want to add, but it may be good to first merge this initial part with:

• Build framework: Create GGIR part 6: Framework + Cosinor + Household co-analysis #906
• Household co-analysis, fixes Incorporate household co-analysis #878
• Cosinor analysis as a first step of circadian rhythm analyses, fixes Add optional cosinor analysis to part 6 #708

To use this, the PR introduces a few extra arguments:

• part6HCA, Boolean to turn on Household Co-analysis
• part6_threshold_combi, Character to specify which threshold combination needs to be used in part 6 ("40_100_400") in the future we could allow for multiple threshold combinations but for now it works on one.
• part6CR, Boolean to turn on Circadian rhythm analysis
• part6Window, vector of length 2 to specify from where to where the time series should be used, e.g. c("O3", "W-2") means from 3rd onset till 2 wakeuptimes before the end.

Checklist before merging:

• Existing tests still work (check by running the test suite, e.g. from RStudio).
• Added tests (if you added functionality) or fixed existing test (if you fixed a bug).
• Updated or expanded the documentation.
• Updated release notes in inst/NEWS.Rd with a user-readable summary. Please, include references to relevant issues or PR discussions.
• Added your name to the contributors lists in the DESCRIPTION and CITATION.cff files.

[vincentvanhees]

I just made two additional commits that:

• Improve handling of non-default argument part6Window values
• Expand unit test to test those non-default part6Window values.

Outstanding work may be:

• Reconsider whether part6Window is a good argument name given that it is only used for circadian rhythm. Another approach could be to integrate it also in the household co-analysis or to change the name into part6WindowCR.
• Function g.report.part6 is not documented I see.
  However, maybe it is easier to wait for your feedback before doing these things.

[jhmigueles]

Thanks @vincentvanhees for all this work. It is great, and I got it to work as expected. Here just some input in case it helps, I think we could facilitate more using the new arguments and I also have some feedback for the documentation. More details in the specific comments.

• I think it is important to document why the household co-analysis is only available with timewindow = "WW" (I left a specific comment on this)
• New functions are missing in NAMESPACE, maybe they are not needed there, but it facilitates debugging.
• Is it possible to keep documentation on household more open for other potential applications of this functionality? e.g., matching recordings from different body attachment sites in the same participant. I see this is not what you have developed this funcionality for, but it could help to do this other application as well.
• Would it be possible to always store the part5 timeseries as part of the part 5 milestone data as GGIR does in part 1 and part 2? Then, the save_ms5raw parameters would only be only be applicable for exporting the time series in csv format. In this case we make sure that the data required to run part 6 is always part of the milestone data and make running part 6 easier for the end users.
• I would add 3 extra checks in check_params (two of them detailed in a specific comment): 1) if user sets part6HCA = TRUE, then save_ms5raw parameters are internally handled not to be problematic (exported in RData and including the invalid time); 2) if user provides only 1 threshold for lig, mod and vig, then part6_threshold_combi is internally derived; and 3) if part6 is run, then do.report should include the number 6 in the case the user forgot about it.
• Maybe for a different project in the future, if we would have a flexible enough function to run hte visualreport, this same function could be used to visualize the household co-analysis, just by adding lines to the same plots.

[jhmigueles]

Can the first and the second '-' be the first and the second character as defined by idloc? So that this is a bit more flexible and consistent with current requirements for filename

[jhmigueles]

After using this a few times, now I think this could complicate things and the current option might be easier to use.

[vincentvanhees]

yes, the challenge is that the participant ID only occurs after the seconds '-'. So, the easiest solution may be to use a different character. For example:

studyID-householdID-memberID_restOfFileName.bin

[jhmigueles]

I would add an extra check for part6_threshold_combi so that if user provides only 1 set of thresholds (most usual scenario I would say), then it takes whatever threshold combination it is. In that case this argument is only needed when user provides more than 1 set of thresholds (not very frequent). Something like:

if (length(params_phyact[["threshold.lig"]]) == 1 & length(params_phyact[["threshold.mod"]]) == 1 & length(params_phyact[["threshold.vig"]]) == 1) {

    params_phyact[["part6_threshold_combi"]] = paste(params_phyact[["threshold.lig"]], params_phyact[["threshold.mod"]], params_phyact[["threshold.vig"]], sep = "_")

}

And another check that makes sure that ms5raw parameters are correctly defined in the case that part6HCA = TRUE:

if (params_247[["part6HCA"]] == TRUE) {

    params_output[["save_ms5rawlevels"]] = TRUE
    params_output[["save_ms5raw_format"]] = "RData"
    params_output[["save_ms5raw_without_invalid"]] = FALSE

}

[vincentvanhees]

Thanks,

I would add an extra check for part6_threshold_combi...

Thanks, I am now adding this.

And another check that

This was partly in the check_params() with a minor variation:

if (params_247[["part6HCA"]] == TRUE) {
  # Add RData because part 6 will need it
  params_247[["save_ms5raw_format"]] = unique(c(params_247[["save_ms5raw_format"]], "RData"))
  params_247[["save_ms5rawlevels"]] = TRUE
  params_output[["save_ms5raw_without_invalid"]] = FALSE
}

In this way the user can still ask for .csv files if they want them.

[jhmigueles]

Should we also add save_ms5raw parameters in this section to specify that they need to be calculated and in which way?

[vincentvanhees]

They are now automatically set by which user does not need to worry about them. Correct?

[jhmigueles]

I think variable names here are lacking the unit of measurement. Is this minutes? seconds? Also for acceleration outcomes.

[vincentvanhees]

Agree, I have now updated these.

For the LUX variables I am skipping this. The unit of light intensity is LUX, but I have tried to avoid the term light as light can be misunderstood as light activity. So, everywhere in GGIR where we discuss light exposure we call it lux. Not ideal, but I am not sure how else to do it.

[jhmigueles]

It would be helpful to explain why the household co-analysis can only be derived with timewindow = "WW". I guess the reason is that many of the variables compare wakeup time of the house members and it's easier to do that in a "WW" timewindow. Although it would also be possible in a "OO" timewindow.

[vincentvanhees]

Actually, I think the WW specification is not needed. The code should work with all of them because it runs on the full timeseries and not on the part 5 windows outcomes. I will now omit those parts of the documentation.

[jhmigueles]

So, if I want to analyse circadian rhythms from first wakeup to last wakeup, this argument should be part6Window = c("W1", "W-1"), is this correct? I think this definition would benefit from a couple of quick examples on how to use it.

[jhmigueles]

Should we give the option to store this as RData to save space?

[vincentvanhees]

sorry, forgot to respond to this... now looking into it.

[vincentvanhees]

I am now changing this to store RData by default. The few users who may want to work with this data will just have to load the RData files. I think there is no major need to facilitate both csv and RData.

[vincentvanhees]

Creating a check list for myself based on your feedback:

• I think it is important to document why the household co-analysis is only available with timewindow = "WW" (I left a specific comment on this)
• New functions are missing in NAMESPACE, maybe they are not needed there, but it facilitates debugging.

• Is it possible to keep documentation on household more open for other potential applications of this functionality? e.g., matching recordings from different body attachment sites in the same participant. I see this is not what you have developed this functionality for, but it could help to do this other application as well.

Response: I think for now it is better to pitch this as a household co-analysis functionality and not broaden it, because that is what I have tested it for and for which it has a clear use-case. The disadvantage of making it more generic is that those who want to do household co-analysis may feel that the functionality is not tailored to them. If in the future there would be a clear need for multi-sensor data analysis or comparisons then I think it is best to review whether the current code is truly the best place to do it and what kind of functionality would be needed for it.

• Would it be possible to always store the part5 timeseries as part of the part 5 milestone data as GGIR does in part 1 and part 2? Then, the save_ms5raw parameters would only be only be applicable for exporting the time series in csv format. In this case we make sure that the data required to run part 6 is always part of the milestone data and make running part 6 easier for the end users.

Response: I see your point but I did not do this because:

1. This PR makes sure that the data required for part 6 is always part of the output by setting save_ms5rawlevels to TRUE and making sure there is at least RData stored. Although I agree that it will simplify the code a bit if the data was always stored in the same place.
2. Moving the time series to the normal part 5 milestone data breaks consistency for those who have used RData files from the raw.out folder. They would then have to rewrite their code/packages to use a new folder. I find it difficult to oversee how much problems this will cause.
3. The current solution allows people to copy the time series files and only work with them. If this is integrated in the part5 milestone data then there is a larger need for us to also document all the milestone data. Currently the milestone data is poorly documented. Storing them in a seperate folder may be a way to encourage users to use them, as opposed to hiding the time series with among other less user-friendly objects.
4. Store the time series in the milestone files, increases the size of GGIR output significantly for users who are not interested in the time series and who process large datasets.

• I would add 3 extra checks in check_params (two of them detailed in a specific comment): 1) if user sets part6HCA = TRUE, then save_ms5raw parameters are internally handled not to be problematic (exported in RData and including the invalid time); 2) if user provides only 1 threshold for lig, mod and vig, then part6_threshold_combi is internally derived; and 3) if part6 is run, then do.report should include the number 6 in the case the user forgot about it.

Maybe for a different project in the future, if we would have a flexible enough function to run hte visualreport, this same function could be used to visualize the household co-analysis, just by adding lines to the same plots

I agree, the plot created for HCA was mainly intended to help myself check that the code works. More generally speaking it would be good to work towards a better strategy for visualisation in GGIR, at the moment we have various isolated visualisation while it may be more efficient to have a central visualisation tool to replace all these individual visualisations. Let's discuss separately.