Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

React Integration for Itinerum #4

Open
wants to merge 42 commits into
base: explore_react_integration
Choose a base branch
from

Conversation

jruzekowicz
Copy link

Currently stuck on the variable passing/using within Tutorial.jsx.

Unable to use the variables/they are not showing as anticipated. Trying to figure out what I am missing here...

…a structure for additions.

Still lacking teating and confirmation.
Curious about changes needed in the html, and function return values.
Still trying to get variables to be present in the Tutorial.jsx file.
Stuck with the variables...
@jruzekowicz jruzekowicz changed the title Halfway there, stuck on variable integration React Integration for Itinerum Jul 15, 2021
Next Steps: Setting some JSON formatted value within the schema, and seeing if I can pull data from there.
Copy link
Owner

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put in some pseudo-like code. I hope it actually works :)
I would order the next tasks as:

  • check to see if you can pass an object instead of individual fields
  • change the schema to pass in first and last name and display them

I thought that the two-way binding worked, but actually looking again, I see that the form is actually modifying the angular scope using ng-model and the react component is not actually modifying anything.

      <form class="pure-form">
        <fieldset>
          <legend>Enter Name</legend>
          <input type="text" ng-model="person.fname" placeholder="First Name"/>
          <input type="text" ng-model="person.lname" placeholder="Last Name"/>
        </fieldset>
      </form>

Let's get this to work and then go back to ngReact to explore the two-way binding properly

www/js/intro.js Outdated
@@ -23,6 +23,7 @@ angular.module('emission.intro', ['emission.splash.startprefs',
.controller('IntroCtrl', function($scope, $state, $window, $ionicSlideBoxDelegate,
$ionicPopup, $ionicHistory, ionicToast, $timeout, CommHelper, StartPrefs, UpdateCheck, $translate, i18nUtils) {

$scope.surveyState = { test: 'test', isValid: true, schema: {}, result: {} };
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$scope.surveyState = { test: 'test', isValid: true, schema: {}, result: {} };
$scope.surveyState = { test: 'test', isValid: true, schema: {'firstname': 'Jenna', 'lastname': "Ruzekowicz"}, result: {} };

@@ -1,5 +1,5 @@
<ion-content class="has-footer">
<tutorial/>
<tutorial test="surveyState.test" isValid="surveyState.isValid" schema="surveyState.schema" result="surveyState.result"/>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why you are passing the individual fields. can you just pass in the surveyState object directly?
Probably best to try that first, since we want to pass in the schema object with sub-fields next.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am attempting to pass the survey state via:
surveyState="surveyState"
but I am hitting a hurdle with then accessing the variables within the $scope.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://stackoverflow.com/questions/49081549/passing-object-as-props-to-jsx

Thought something like this would work.. alas I can't seem to get it right.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by accessing variables within the $scope?
Also, you are already passing in an object (surveyState.schema) and it looked like that worked?
Does it mean that top level objects directly under $scope don't work?

Copy link
Author

@jruzekowicz jruzekowicz Jul 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Within the .jsx file, we do the prop type checking. I believe one of the issues here is that we originally passed in the name, ie isValid="isValid". Now, we are only passing in the $scope variable surveyState. Code from .jsx:

propTypes: {
        isValid: React.PropTypes.bool,
        schema: React.PropTypes.object,
        result: React.PropTypes.object
    }

My initial instinct was to attempt something like surveyState.result: React.PropTypes.object, but this did not work.

A few ways I've tried to pass in the surveyState:
<tutorial {surveyState} />
<tutorial {...surveyState} />
<tutorial surveyState="surevyState" />

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, you would need to change the propTypes to accept a surveyState object as well

Also, check the spelling on <tutorial surveyState="surevyState" />

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we changed the propTypes to accept a surveyState, would that be the only check?
ie. would we have something like:

propTypes : {
    surveyState: React.propTypes.object 
} 

I am not seeing how the propTypes would be changed to accept a surveyState object exactly. We would still want to do all the confirmation that isValid is a boolean etc.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah that would be the only check. Note that we already don't do any type checking within either the schema or the results, so we only lose type information for isValid with this approach.

Let's first see if it works, and then we can make the design decision.

Copy link
Author

@jruzekowicz jruzekowicz Jul 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, here is a commit that does not quite work yet, but seems to be what we want. Still trying to figure out why this would not work. d48bba1

Question: Why do we need the React.PropTypes. ... as opposed to just PropTypes. ...?

Maybe they need to be key values pairs within the scope now? As in 'isValid': false instead of isValid: false? But not sure why this would work on the individual level but not when we attempt to pass in via the entire scope variable?

Comment on lines 14 to 16
<div>
This is the test -> {this.props.test}
</div>
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully something like this works.

Suggested change
<div>
This is the test -> {this.props.test}
</div>
<div>
This is the first name -> {this.props.schema.firstname}
This is the last name -> {this.props.schema.lastname}
</div>

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! This works perfectly. (Surprised it is so simple.)

Still trying to figure out the passing in of only surveyState
@shankari
Copy link
Owner

@jruzekowicz were you able to debug in android and figure out what the error is?

@jruzekowicz
Copy link
Author

@jruzekowicz were you able to debug in android and figure out what the error is?

I was able to get the debugger up and running by the end of the day Friday. This morning I have traced it down to a type error as follows:

TypeError: Cannot read property 'schema' of undefined
    at TutorialComponent.render (Tutorial.js:31)

This error statement pertains to the following line:
{this.props.surveyState.schema.fname}
when I simply pass in surveyState as a whole. Trying to figure out if there is another way in which I need to access schema from surveyState.

@jruzekowicz
Copy link
Author

Some things that have not worked:

  • Numerical indexing does not work, example: this.props.surveyState[#]
  • String indexing does not work, example: this.props.surveyState['schema']
  • Direct attempt at access does not work: this.props.surveyState.schema
    All return the same error: TypeError: Cannot read property 'schema'/'#' of undefined

No error is returned to directly access the surveyState such as: this.props.surveyState.
I am currently trying to determine why the we are seeing "undefined". My guess here is that the surveyState is labeled as undefined because of the JSON formatting not being a strict data type.

@shankari
Copy link
Owner

shankari commented Jul 19, 2021

this seems to say that surveyState is undefined. I would stop working on this now - there are known issues with the level of data objects in angular. As long as passing in the second level structures (e.g. surveyState.schema) works reliably, let's go ahead with that instead of passing in top level structures (surveyState).

I wanted to spend some time seeing if top level structures would work, but they are not critical to the long-term design.

@jruzekowicz
Copy link
Author

this seems to say that surveyState is undefined. I would stop working on this now - there are known issues with the level of data objects in angular. As long as passing in the second level structures (e.g. surveyState.schema) works reliably, let's go ahead with that.

Sounds good. I will make those changes, re-run to confirm everything is working properly and push a commit. I scheduled a meeting for early today to discuss next steps and getting started on writing this week.

@shankari
Copy link
Owner

shankari commented Jul 19, 2021

No error is returned to directly access the surveyState such as: this.props.surveyState.

There is no error because props is defined, only surveyState is not defined. If you print out the surveyState using console.log, you will see that it is defined when you pass in a top level object.

undefined is like null. The error you are getting is an NPE.

@jruzekowicz
Copy link
Author

@shankari Let me know if there is anything majorly wrong with the basic commit here: 2d6351e
If not, I will get started on reading up on the paper requirements.

@shankari
Copy link
Owner

shankari commented Jul 19, 2021

we still need to experiment with setting values in React and ensuring that they are visible in Angular.
Memory jog: #4 (review)

@shankari
Copy link
Owner

shankari commented Jul 19, 2021

After that, we will want to retrieve the schema from @kafitz's service (e.g. e-mission/e-mission-docs#643 (comment)) and pass it in to the react component. I think @kafitz will take it from there?

@kafitz or do you also want us to save the values to the database to be pushed up?

@kafitz
Copy link

kafitz commented Jul 19, 2021

If we're able to save the values to the app's own database, that sounds ideal. I think this primarily comes down to whether we will allow users to update their own survey answers in the future. As a user, I'm always in favor of having more control, but I don't want to divert if it seems too time consuming.

For the React component, it can be just passed in as the raw JSON payload, no other parsing necessary. I've got that running on my end now with the Tutorial.jsx demo too.

Other component development thoughts:

  1. In the past year, I rewrote our Concordia dashboard in TypeScript which is much cleaner than the original JSX-only version. I have to research next how TS can be used with ngReact, but I believe it would just be an extra babel transpiling flag or step.
    I want to use this code for handling the global survey state within the ngReact component itself. This state is will be 2-way bound with Angular for storing the db and making network calls. This code already has logic for parsing the incoming JSON payload and laying out survey questions as "cards".

  2. The dashboard survey's preview does not track user answers to state, and these dashboard components are geared for adding & editing questions--not answering them. I have an existing Flutter code base with these survey cards and navigation built out, so I think it would be fastest to translate this code to Typescript. I've found Flutter & TS to have very similar syntax.

So overall my intention is to start by getting a TypeScript version of Tutorial.jsx working, merge in code from TS dashboard for state management, and then copy in the UI code from my Flutter repo. Open to comments/thoughts/suggestions

@jruzekowicz
Copy link
Author

@shankari Currently having some trouble checking the two-way binding for React to Angular.
My current understanding is that there is not a standard two-way binding implementation for React. This is an angular thing.

I have been doing some reading with some examples but am not clearly seeing how it corresponds to what we are trying to check here.

Within the Hello examples within the ngReact repo, we have input text boxes. The following code is the input for the names:

<form class="pure-form">
     <fieldset>
          <legend>Enter Name</legend>
          <input type="text" ng-model="person.fname" placeholder="First Name"/>
          <input type="text" ng-model="person.lname" placeholder="Last Name"/>
     </fieldset>
</form>

The ng-model seems to be the key for the two-way binding on the angular side. I am unfortunately extremely confused with how to/what we are showing for React to Angular?

@shankari
Copy link
Owner

@jruzekowicz we currently want to focus on a 2-way binding between react and angular. So if I make a change to fname in the react code, does the angular value change (both in the form and in the display on the top)?

How do you change variables in react? There must be a standard way...

@shankari
Copy link
Owner

Concrete goal:

  • add a "reset" button to react
  • when it is pressed, set fname and lname to blank (in the proper react way)
  • when that happens, do the angular display and the form fields become blank as well?

@jruzekowicz
Copy link
Author

var Hello = React.createClass( {
  propTypes: {
    fname: React.PropTypes.string.isRequired,
    lname: React.PropTypes.string.isRequired
  },

  changename(newname) {
    this.setState({
      fname: newname
    })
  },

  render: function() {
    return (
      <div>
        {this.props.fname}
        <input type="button" value="Log" onClick={this.changename.bind(this, 'Jenna')} />
      </div>
   );
  }
} );

This is about the implementation I have for the button, yet I am still stuck in getting it to change the actual values. I seem to have trouble working with the props ie: this.props.fname etc. I can not get the values to change. Is there anything you can see immediately that is not concurrent with your thinking @shankari ?

@shankari
Copy link
Owner

@jruzekowicz I have no familiarity with React, so I would need to look it up, explore, etc myself.
@kafitz has more experience with React, so may be able to see something obvious.
Otherwise, you just have to work with the internet and figure it out.

@kafitz
Copy link

kafitz commented Jul 20, 2021

I'm less familiar with using createClass() to build components, but nothing jumps out at first glance (if the changename function is indeed getting called).

There's 3 main ways to build a component in React: using createClass(), extending the built-in React.Component class, or most recently, their "functional components". Most current advice is to use functional components and they may be easiest to reason about. There's a comparison between them and extending the class here: https://reactjs.org/docs/hooks-state.html

I would suggest going over there because binding methods like you're doing here is one of the key areas that's been improved. I believe it may cause the component to re-render unnecessarily passing in a parameter like you're doing now.

…back on this and what improvements are needed.

Working on understanding the prop vs state differences.
@jruzekowicz
Copy link
Author

jruzekowicz commented Jul 20, 2021

@kafitz @shankari I think I am on the right track with this commit: bf9c9f0

I would love @kafitz 's feedback on this commit and how close it is to what we are targeting.

Another commit was pushed slightly after to demonstrate some of the issues with using event.target.value. May need to rethink this?

…g for Kyle's feedback on the best way to go forward.
};
this.handleChange = this.handleChange.bind(this);
}
handleChange(event) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use different handlers!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Network call:

Handled by angular side of the app. Where is this located?? net.py?? Python w/ all network calls somewhere?
How to retrieve a JSON from endpoint (first pull request) ->
Pull down the JSON and store it within the app database? (All happening outside, local database on the phone itself??)
Eventually JSON becomes the "schema"-> tells the app how to layout all the data
Return data is just the answers.
Kyle has react.
Model off of existing code.

Kyle and I discussed the use/standard of multiple handling.
@jruzekowicz
Copy link
Author

jruzekowicz commented Jul 20, 2021

@shankari Latest Commit demonstrating the react -> angular operations with a text box entry. 8da04fb

@kafitz took a look at this and suggested I move on to the next step of pulling down the survey in the JSON format. I will add to this comment with questions!

After that, we will want to retrieve the schema from @kafitz's service (e.g. e-mission/e-mission-docs#643 (comment)) and pass it in to the react component. I think @kafitz will take it from there?

Where in the code base would I be adding in the retrieving step? @kafitz mentioned you may have a common location with GET and POST requests.

@shankari
Copy link
Owner

Should we call intake first?

As I said,

I would also (for completeness), run the intake pipeline for the user you are logged in to with end-to-end testing, run the export script, and confirm that the result is in the exported .gz file.

Otherwise, your thinking looks good.

@jruzekowicz
Copy link
Author

Hmm okay. So my plan is to load the data via a call with a specific email such as:
./e-mission-py.bash bin/debug/load_timeline_for_day_and_user.py emission/tests/data/real_examples/shankari_2015-07-22 test_july_22

Then I will launch the server and log into the app with the same email of test_july_22.

I then plan to run the intake pipeline: ./e-mission-py.bash bin/debug/intake_single_user.py -e test_july_22
Question: Does this pull from the phone?

I then would want to extract that data using the following: ./e-mission-py.bash bin/debug/extract_timeline_for_day_range_and_user.py -e test_july_22 which contains the export stage of the pipeline.

What am I missing here?

Additionally, I looked at

I would force push the entries to the server and make sure that they show up there - e.g. steps for testing

e-mission#770

but got completely lost in the process. Possibly something for us to go over within the meeting today?

@shankari
Copy link
Owner

Question: Does this pull from the phone?

No. That is the part linked from e-mission#770
"End trip and force sync" etc

@shankari
Copy link
Owner

  • "End trip and force sync" is in the phone app UI (Profile -> Developer Zone)
  • for checking the usercache
(emission) e-mission-server$ ./e-mission-py.bash
>>> import ....

@jruzekowicz
Copy link
Author

@shankari
The following is the result from doing the end trip and force sync.

>>> edb.get_usercache_db().find({"metadata.key": "stats/client_nav_event"}).distinct("data.name")
['app_launched']

My worry here is that maybe I shouldn't be looking within this exact metadata.key (stats/client_nav_event). This definitely seems wrong as no edits were made to that file. Trying to figure out what this would even be under??

@shankari
Copy link
Owner

Right, you should look for the key that you just inserted manual/...

@jruzekowicz
Copy link
Author

Right, you should look for the key that you just inserted manual/...

How do I know if this will be in metadata vs data?

@jruzekowicz
Copy link
Author

>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.name")
[]

Empty result here... hmm...

@shankari
Copy link
Owner

I encourage you to read up on the e-mission data model from the Timeseries_sample ipython notebook and the thesis. I know the answer, of course, but I think it is worthwhile for you to understand this given that you have been working on the project for a semester.

@jruzekowicz
Copy link
Author

Some updates:

>>> edb.get_timeseries_db().distinct("metadata.key")
['background/filtered_location', 'background/location', 'background/motion_activity', 'statemachine/transition', 'stats/pipeline_error', 'stats/pipeline_time', 'stats/server_api_time']

I feel as if I am on track with using the get_timeseries_db() as opposed to get_usercache_db() as that is where I added it in on the server side. Additionally, I used the metadata key which returned keys in a similar fashion to how I would want "manual/survey_response" returned.
As seen above I am still not getting any of that key in the results. I am now at a dilemma of if I am not doing the accessing correctly (as I am not seeing any "manual/..." of any type), or if this is an issue with the actual saving of the results!

@jruzekowicz
Copy link
Author

jruzekowicz commented Aug 10, 2021

@kafitz Any reason I would all of the sudden be getting a 404 error for the POST request?
{status: 404, url: "http://198.245.50.61/mobile/v2/create", headers: {…}, error: "
↵<title>404 Not Found</title>

@kafitz
Copy link

kafitz commented Aug 10, 2021

@jruzekowicz sorry about that! I was dealing with a CORS issue working on our dashboard. Should be back now

@jruzekowicz
Copy link
Author

@jruzekowicz sorry about that! I was dealing with a CORS issue working on our dashboard. Should be back now

Haha no worries! I was just concerned I broke something on my end!

@shankari
Copy link
Owner

Look at the stages of the intake pipeline (e.g. what is the first stage) and read up (maybe from the thesis) on the differences between the timeseries and the usercache.

@jruzekowicz
Copy link
Author

jruzekowicz commented Aug 11, 2021

Look at the stages of the intake pipeline (e.g. what is the first stage) and read up (maybe from the thesis) on the differences between the timeseries and the usercache.
@shankari
After reading through the documentation I came up with the following:

  1. From the documentation: "All background communication to the server happens using the usercache, we pull data periodically using /usercache/get and push data periodically using /usercache/put." and "The incoming entries are initially put into the usercache and then copied to the timeseries database as part of the intake pipeline." Thus, I should not initially be checking the timeseries db as this is BEFORE any intake pipeline is being run!
  2. Thus, I re-ran the process of ending the trip and forcing the sync while keeping on eye on the console to see if I was properly getting the usercache/put and usercache/get calls. The results are as follows:
START 2021-08-11 11:00:59.199150 POST /usercache/put
END 2021-08-11 11:00:59.231932 POST /usercache/put 74434e61-b368-49de-9f54-2b3e16573d67 0.03272604942321777 
START 2021-08-11 11:00:59.318509 POST /usercache/get
END 2021-08-11 11:00:59.334018 POST /usercache/get 74434e61-b368-49de-9f54-2b3e16573d67 0.015457868576049805

which logically makes sense and confirms my understanding.

  1. I then went back and tried to confirm that the correct key was present within the usercache as follows: >>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.name") This still returned a blank list [].
  2. My next attempt was pulled from the jupyter notebook examples, where I ran: edb.get_usercache_db().distinct("metadata.key") in an effort to pull all metadata keys. Unlike the first attempt, this actually showed that the key was within the metadata! Here: ['background/battery', 'config/consent', 'diary/trips-2015-07-22', 'manual/survey_response', 'statemachine/transition', 'stats/client_nav_event', 'stats/client_time']
    Confused as to why this did not show up in the first call (bullet point 3), but alas it does seem to be present which is what we wanted to confirm.

@shankari
Copy link
Owner

Glad you figured it out! We can resolve the confusion at our 1:1 meeting.
Maybe you can now confirm that the data is exported correctly?

@jruzekowicz
Copy link
Author

Glad you figured it out! We can resolve the confusion at our 1:1 meeting.
Maybe you can now confirm that the data is exported correctly?

Yes! That is what I am now working on. I plan to run the full export pipeline and check to see if the key and the data is within the export file. Does this sound correct?

@jruzekowicz
Copy link
Author

@shankari I am seeing somewhat unique results and am trying to determine if they are expected.
After going in an doing the survey three different times, I get the following:

>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.fname")
['Jenn', 'Jenna', 'New Value']
>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.lname")
['Ruzekowicz']
>>> 

For this, I changed the first name three times, while the last name remained the same. It seems to record all new values each time. Maybe we can talk more about this? Not sure what is expected or if this is an implementation error.

@jruzekowicz
Copy link
Author

jruzekowicz commented Aug 11, 2021

Results from running the intake pipeline:
Got error unexpected indent (survey_response.py, line 2) while saving entry AttrDict({'_id': ObjectId('6112aeba3ff364694d4e4575'), 'metadata': {'key': 'manual/survey_response', 'platform': 'android', 'read_ts': 0, 'time_zone': 'America/New_York', 'type': 'message', 'write_ts': 1628614252.962}, 'user_id': UUID('7c344abc-e47a-43a6-8888-7013b2c430f9'), 'data': {'fname': 'Jenna', 'lname': 'Ruzekowicz'}}) -> None
As you can see, the data is present in the intake pipeline, but there seems to be an indent error in the survey_response.py. I am going to look at that right now.

Indentation errors fixed. However the following error still occurring:
Got error 'AttrDict' instance has no attribute 'start_ts' while saving entry AttrDict({'_id': ObjectId('611410a53ff364694d4e6fb6'), 'metadata': {'key': 'manual/survey_response', 'platform': 'android', 'read_ts': 0, 'time_zone': 'America/New_York', 'type': 'message', 'write_ts': 1628704922.192}, 'user_id': UUID('7c344abc-e47a-43a6-8888-7013b2c430f9'), 'data': {'fname': 'Fixed indentation', 'lname': 'Ruzekowicz'}}) -> None
A good sign however that the exact data I enter into the fields (in this case "Fixed indentation" is immediately shown.

@jruzekowicz
Copy link
Author

@shankari Still a little confused on how we want to incorporate the start_ts and end_ts.
I am referencing this PR: e-mission@bb02432

It still appears that the code is using a putMessage with a specific key. Would this mean I would want to incorporate the start_ts and end_ts within the message I am already sending (using the same survey_response key)?

Additionally, I am still search to find how I actually can get those time stamps. We would want the exact time stamps for when the survey is first displayed to the user, and when they finally click the submit button. I can visualize where this should be within the code, but do not yet know how to actually get an immediate time stamp.

@jruzekowicz
Copy link
Author

@shankari Still currently stuck on this. I am not sure if we already have a function that does this, but is there a way to get the exact time stamp of a specific event? i.e. a button is pressed, and we get the timestamp of that press?

@shankari
Copy link
Owner

a button is pressed, and we get the timestamp of that press?

When a button is pressed, your $scope function is called. So within the $scope, you really need the current timestamp. In javascript, that is new Date() or similar. Note that e-mission expects timestamps in seconds.

@jruzekowicz
Copy link
Author

a button is pressed, and we get the timestamp of that press?

When a button is pressed, your $scope function is called. So within the $scope, you really need the current timestamp. In javascript, that is new Date() or similar. Note that e-mission expects timestamps in seconds.

Okay, so for this implementation, would I simply look into js documentation to figure out how to get a time stamp? You noted that e-mission expects the timestamps in seconds. Is there other formatting I need to worry about?
My idea would be to call some new Date() function and set that as the start_ts, then pass the start_ts, like lname and fname within the results data.

@jruzekowicz
Copy link
Author

@shankari I was about to get the start_ts within the data, and for that ts to be directly applicable to the moment when the survey is officially loaded in displayed.

The following message now occurs:
Got error Could not match input to any of ['YYYY-MM-DD', 'YYYY/MM/DD', 'YYYY.MM.DD', 'YYYY-MM', 'YYYY/MM', 'YYYY.MM', 'YYYY', 'YYYY', 'YYYY'] on '' while saving entry AttrDict({'_id': ObjectId('611547db3ff364694d4e7dd9'), 'metadata': {'key': 'manual/survey_response', 'platform': 'android', 'read_ts': 0, 'time_zone': 'America/New_York', 'type': 'message', 'write_ts': 1628784588.937}, 'user_id': UUID('7c344abc-e47a-43a6-8888-7013b2c430f9'), 'data': {'start_ts': 1628784585.518, 'end_ts': '', 'fname': '', 'lname': ''}}) -> None

Indicating there is an error with the formatting? This seems to be different than the error that we had before. Do you have any insight on this?

@shankari
Copy link
Owner

I notice that the end_ts is blank and the error appears to be related to formatting dates. I would fill in the end_ts and retry.

@jruzekowicz
Copy link
Author

I notice that the end_ts is blank and the error appears to be related to formatting dates. I would fill in the end_ts and retry.

Success! No longer an issue. Results from filling the survey twice and checking the usercache:

>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.start_ts")
[1628788017.462, 1628788224.702]
>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.end_ts")
[1628788028.444, 1628788226.254]

Results after the intake pipeline has been run:

>>> edb.get_timeseries_db().find({"metadata.key": "manual/survey_response"}).distinct("data.end_ts")
[1628788028.444, 1628788226.254]

which confirms the results are being transferred from phone to usercache to db.

@jruzekowicz
Copy link
Author

Results from the exported file:

"metadata": {
            "key": "manual/survey_response",
            "platform": "android",
            "read_ts": 0,
            "time_zone": "America/New_York",
            "type": "message",
            "write_ts": 1628787342.232,
            "write_local_dt": {
                "year": 2021,
                "month": 8,
                "day": 12,
                "hour": 12,
                "minute": 55,
                "second": 42,
                "weekday": 3,
                "timezone": "America/New_York"
            },
            "write_fmt_time": "2021-08-12T12:55:42.232000-04:00"
        },
"data": {
            "start_ts": 1628787339.618,
            "end_ts": 1628787342.231,
            "fname": "",
            "lname": "",
            "start_local_dt": {
                "year": 2021,
                "month": 8,
                "day": 12,
                "hour": 12,
                "minute": 55,
                "second": 39,
                "weekday": 3,
                "timezone": "America/New_York"
            },

confirming that the process is completed and working!

…ses.

Testing done:
After an "end trip and sync" from the app. I checked the usercache to confirm that the start_ts, end_ts were present.
>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.start_ts")
[1628788017.462, 1628788224.702]
>>> edb.get_usercache_db().find({"metadata.key": "manual/survey_response"}).distinct("data.end_ts")
[1628788028.444, 1628788226.254]
Additionally checked to make sure the data would result in the same outcome once the intake pipeline was run.
>>> edb.get_timeseries_db().find({"metadata.key": "manual/survey_response"}).distinct("data.end_ts")
[1628788028.444, 1628788226.254]
Finally verified that the information would be present within the fully exported file:
"metadata": {
            "key": "manual/survey_response",
            "platform": "android",
            "read_ts": 0,
            "time_zone": "America/New_York",
            "type": "message",
            "write_ts": 1628787342.232,
            "write_local_dt": {
                "year": 2021,
                "month": 8,
                "day": 12,
                "hour": 12,
                "minute": 55,
                "second": 42,
                "weekday": 3,
                "timezone": "America/New_York"
            },
            "write_fmt_time": "2021-08-12T12:55:42.232000-04:00"
        },
"data": {
            "start_ts": 1628787339.618,
            "end_ts": 1628787342.231,
            "fname": "",
            "lname": "",
            "start_local_dt": {
                "year": 2021,
                "month": 8,
                "day": 12,
                "hour": 12,
                "minute": 55,
                "second": 39,
                "weekday": 3,
                "timezone": "America/New_York"
            },
shankari pushed a commit that referenced this pull request Mar 28, 2023
hack for enketo to preserve prefilled timezone
shankari pushed a commit that referenced this pull request Aug 27, 2024
Renamed clear data to something more accurate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants