Integrate e-mission and itinerum (finally!)

[shankari]

@PatGendre @asiripanich you should be happy about this :)

@kafitz and I just had the preliminary discussion around how the integration might work.
A very basic data-level (not code-level) integration would be:

• e-mission, on startup, pulls the survey JSON from the itinerum server
  • the survey JSON access code does not have authentication, so this is easy
• e-mission creates a locally generated token (similar to NREL-Lo-Hi and @asiripanich's code) which will be used to log in to the e-mission server
• itinerum will create their own instances of the e-mission server and database containers running on their own server
• we will write two integration containers:
  • one which reads the e-mission data and exposes it via a simple API
  • one which formats the data into Postgres and saves it into the Itinerum database
• the exporter container will not be externally exposed, so the API does not need authentication
• there will be a separate "EXPORT_DATA" pipeline stage which will be maintained similar to the other pipeline stages, and so will support incremental updates

That seems eminently doable in the short internship timeframe

[shankari]

@kafitz one thing that we are still missing is displaying the survey after we retrieve it. Do you have a javascript library to do that (maybe for a preview in the survey builder)? could you add the link here as well?

[kafitz]

This is something I meant to ask about this on our call today, so far that's been handled by the native mobile Itinerum apps. Currently we can see and edit the questions using the existing dashboard, but the client view for e-mission would still need to be built.

Is the best way to write some HTML/JS viewer now that could eventually be merged into the e-mission app? It might take a week or so (for a prototype), but seems doable relatively quickly.

[shankari]

@kafitz yes, a HTML+JS viewer that would be merged with the e-mission app would be the way to go. I was assuming you would have such a viewer anyway as part of the survey builder to preview the survey as it was being built. How do survey users preview the survey right now?

[kafitz]

Currently we rely on the Survey Builder wizard for laying out questions and then the admin users test how it looks by running the app. If any survey questions must be fixed, it does mean they have to clear the app data and retry.

The builder provides a decent enough overview for this, but it's written in React. Since the app is Cordova/Angular (which I'm still not super familiar with), I was thinking now I can create a very simple interface with HTML & vanilla JS and this will hopefully keep it easy to merge into the e-mission-phone code later.

Open to suggestions ofc if there's a better way to do it. I'll admit I'm slightly worried about it snowballing, but I think it should be ok if we don't worry about aesthetics until putting it into the app.

[shankari]

so I've been poking around with the edges of react for a while (https://github.com/NREL/mobility_landscapeapp is in React, for example), and it looks like you can use some form of react embedded into other applications.
https://reactjs.org/docs/getting-started.html
or
https://stackoverflow.com/questions/20947191/using-reactjs-with-angularjs

I can play around with it a bit more; if it works it seems like it would be the easiest option without introducing additional codebases that need to be maintained. It would also give us a more gradual migration path forward - as we cleanup individual components, we can rewrite them in react.

Is there a simple but meaningful react component that you can share which I can use to experiment with this approach?

[shankari]

This looks super simple, for example
https://mtalham.medium.com/build-web-component-with-react-and-use-it-in-an-angular-app-515c2fed6a53

And this:
https://github.com/ngReact/ngReact

Although it is currently archived
Famous last words 😄

[kafitz]

The Tutorial.jsx snippet on this gist page is my most basic component for the help screen users first see in the dashboard.
https://gist.github.com/kafitz/3f4995ce85e59a346f94db2bbf37bc27 This does expect Bootstrap 3 and material icons to be added during build, so I don't expect it to be pretty.

I can send a more involved example, but that may take a bit since I rewrote the codebase using Typescript/Material UI. That code resembles something closer to React Native and requires some webpack transpiling...

For anything that would go in e-mission, I would still write that in regular React for simplicity.

[shankari]

I think we can get webpack transpiling if we need to as well.
(https://mtalham.medium.com/build-web-component-with-react-and-use-it-in-an-angular-app-515c2fed6a53) seems to have a standard React app

But let's start with seeing if I can embed this first.

[shankari]

@kafitz can you add a link to the API that we can use to pull the survey JSON (I assume this is on a staging server, and there is one canonical server we can pull). we can then start with reading that. At tomorrow's meeting, let's discuss at what stage on the onboarding process we should make this call and how the user specifies the survey to pull.

[shankari]

to give some additional context: e-mission is currently written in angular1 also called angularJS on the internet. It is also a reactive framework in which you make changes to $scope variables in a controller which are automatically updated in the HTML.

So concretely, if I want to display a username, I can have <h3>{{username}}</h3> in the HTML, set $scope.username = "Jenna"; in the javascript controller and it will show up. It also has some additional tags to make the UI coding easier, so for example, (syntax may be a bit off, high level workflow only)

<ion-list ng-repeat="trip in tripList">
    <ion-item>{{trip.start_fmt_time}} -> {{trip.end_fmt_time}}</ion-item>
</ion-list>

then if the javascript has $scope.tripList = [{start_fmt_time: "...", end_fmt_time: "..."}, {...}, {...}, ...] then the list will show all the trips.

[shankari]

once we get the survey JSON from itinerum, we want to display it. @kafitz already has code to display it that is written in React, which is a very similar-looking but different javascript framework. So we are going to try to package the react code as a component and include it in the angular framework. The comments above claim it can be done. If not, we may have to rewrite parsing + display logic in angular.

So if everything works, we will use react to create a new component called SurveyDisplay or something. That would create a new HTML tag that we can use in our code. so our HTML would do something like:

<h2>Survey display</h2>
<surveydisplay data="...">...</surveydisplay>

[kafitz]

The base url for the staging server will be at: https://api.hungry.wales/mobile/v2

Attached below is a shortened version of an integration testing script which includes two functions: one to make the initial API call for the survey JSON and one to POST the answers.

I recently started adding some Swagger docs which may give some idea of the requests, but there's a lot more work to do on that to make it clear: https://api.hungry.wales/mobile/swagger/index.html

Something to note is this would be the app to use my "v2" API which was built with conditional question logic in mind. It's backwards compatible with v1 (used by existing mobile apps), but you'll find the survey value within the JSON itself is pretty meaty. I followed JSON Schema for declaring this survey model--personally I find their documentation difficult to follow, but this tool might help show it better: https://react-jsonschema-form.readthedocs.io/en/latest/usage/dependencies/

Demo code: emission-integration-demo.zip
(sorry about the zip, Github issues doesn't like .py files I guess)

[kafitz]

Also I would say that the client-side React code is about 50% there.

We have a question builder interface which parses the JSON and maps each question to components, but it will need transformed into a client version. This I was planning to help write for the UI integration, but haven't yet ruled out Angular here if that winds up making more sense.

[shankari]

two functions: one to make the initial API call for the survey JSON and one to POST the answers.

I believe that there is no auth required for the initial call for the survey, how do we authenticate to POST the answers?

[kafitz]

To clarify about the requests for retrieving the survey, there will be two POST requests:

1. There will the initial POST request to send parameters about the phone (OS, OS version, language, etc), a UUID (generated on phone, we use the UUIDv4 spec), and a survey name which will return the survey parameters and questions as JSON.

2. Once the client has responded to all survey questions, you then create a POST request with survey answers and the same UUID to authenticate.

[shankari]

Just to clarify how this would work on the e-mission side, because of our legacy of authenticating via email, we actually generate a random token first and "register" with the server, similar to step 1 in your case.
We then retrieve the UUID from the server and use it to fill out any surveys that are launched during onboarding. The surveys are currently not saved to e-mission since we don't have a survey builder, but we can fairly easily change that. We should then be able to send a POST request with the uuid and the survey result to the itinerum server.

@jruzekowicz the code to currently launch the survey with a UUID is at
https://github.com/e-mission/e-mission-phone/blob/master/www/js/survey/launch.js

It is invoked from https://github.com/e-mission/e-mission-phone/blob/ceo_ebike_project/www/js/intro.js#L112 for a google form and from https://github.com/e-mission/e-mission-phone/blob/nrel_location_history/www/js/intro.js#L144 for a kobo toolbox form

I would suggest that an initial task would be:

• use one of those branches
• in $scope.surveyLaunch, retrieve the sample JSON from @kafitz's example
• print it out on the console so we know it was retrieved
• send back a POST to @kafitz's server with the UUID and a example hardcoded filled out value
  • get the example filled out value by printing from the python script and hardcoding into the JS for now

[shankari]

e-mission allows you to launch a survey on an end of trip prompt, but the general feedback we have gotten is that end of trip prompts are really intrusive.

So if deployments do want to have an end of trip prompt, right now, the default is to go to the diary screen where you can look at the trip and answer any "labeling" questions. That label can launch a survey, but people can wait until the end of the day to do it.

Most deployments have actually switched to once a day prompts, in the evening, that just launch the diary.

[shankari]

you would replace existing code with something like (very sketchy).

$scope.surveyLaunch = function(uuid) {
     const surveyJSON = $http.post(.....);
     console.log(surveyJSON);
     const surveyResult = ... hardcoded, copied from python printout....
     $http.post(.........surveyResult, uuid.....);
}

[shankari]

Looking through the various options above, there is a clear tradeoff between immediate usability and long-term migration. Concretely:

• the medium article: works with angular 11. Can we use a similar technique with angular1? Unclear
• ngReact: archived, not updated for two years. But the examples are very clearly angular1, the code is installed using bower, the imports are included as <script tags, etc. It suggest using react2angular or ngImport.
• react2angular: not archived, also updated 2 years ago. seems pretty similar to ngReact, but the examples have imports that look like this import { react2angular } from 'react2angular'. I am not sure how this will work with our current angular1 codebase. I do see this SO post (https://stackoverflow.com/questions/43137592/import-node-module-in-angularjs) which seems to indicate that these imports work for angular2, but the README says "Use it in your Angular 1 code"
• ngimport: recommended by ngReact, this really seems to be focused around "ugly, proprietary DI". It also provides "an easy migration path to angular2, React, etc". Note that DI stands for dependency injection - this is what it looks like in angular1. This seems orthogonal to our problem of including React components although it will help with longer-term migration.

[shankari]

at this point, given the tight timeframe and importance of getting something to work, I am going to start with ngReact. Depending on how long that takes to get working, I will try out react2angular or the medium article, otherwise, we are going with ngReact.

[shankari]

Starting with ngReact....

• the examples all seem to work with bower, so using it instead of npm for the install
• after cloning directly, and using npx bower install ngReact, I can run the first static hello world!!
  • Note that the angular component has $scope.person defined, and it passes it in to the react component as props
  • the react component uses prop throughout
  • quick check: is the binding bi-directional? let's add an angular-based HTML using $scope to figure out. Works!

diff --git a/examples/hello/index.html b/examples/hello/index.html
index 1b41b94..6c39b6b 100644
--- a/examples/hello/index.html
+++ b/examples/hello/index.html
@@ -31,6 +31,12 @@
   </head>
   <body ng-app="app" ng-controller="mainCtrl">
     <div class="content">
+        <h1>
+          <div>Hello {{person.fname}} {{person.lname}}</div>
+          <small>
+            <code>&lt;From angular to check bi-directional binding/&gt;</code>
+          </small>
+        </h1>
       <div class="header">
         <h1>
           <div><react-component name="Hello" props="person"/></div>

[shankari]

So it works for some version of react. Now, let's check the jsx stuff since that is what Kyle's example uses. That looks like it basically just includes <script src="../../bower_components/react/JSXTransformer.js"></script> and then just includes the jsx file directly. Let's try https://github.com/ngReact/ngReact/blob/master/examples/jsx-transformer/index.html as a static file.

[shankari]

That does not appear to work right out of the box. The failure is

ReferenceError: Hello is not defined
    <anonymous> file:///Users/kshankar/OSS/ngReact/examples/jsx-transformer/app.js:9
    Angular 27
    <anonymous> file:///Users/kshankar/OSS/ngReact/examples/jsx-transformer/app.js:12
angular.js:15697:41

It looks like JSXTransformer is not included in react by default

$ find bower_components/ -name \*JSX\*
$

From the README, this is also "a hacky approach"

The workaround for this is hacky as the angular bootstap is postponed in with a setTimeout, so consider transforming jsx in a build step.

Going to spend another 10 mins on this, and then switch to the es6 + webpack example

[shankari]

Aha! JSXTransformer was deprecated in 0.13.x and removed in 0.14.x
https://reactjs.org/blog/2015/06/12/deprecating-jstransform-and-react-tools.html

the current version of ngReact uses react 15+, so it looks like the example was not updated. we can still use react and jsx without using webpack by including babel directly. babel can manually convert all jsx to js ahead of time.
https://reactjs.org/docs/add-react-to-a-website.html#optional-try-react-with-jsx

That approach works (see screenshot below)!

We can now include jsx for sure, even if it is not super production ready. A minor caveat was that since we split the file into multiple javascript/jsx files, it ran into CORS issues. Note that loading from files doesn't work with CORS any more,

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at .... (Reason: CORS request not http)

so we have to start up a tiny web server. I used https://github.com/danjellesma/http-server; the same as em-dashboard and everything worked great.

[shankari]

Now, let's try the "production" version of babel without going all the way to webpack and transpiling. Although perhaps we should/will get there eventually.

This turned out to be a little harder than I thought, so I'm creating a fork with my changes for the part that is already working. Then I can continue noodling on the part that is not working, but in a separate directory, and with the ability to compare working and non-working side by side.
https://github.com/shankari/ngReact

[shankari]

huh! After copying it over, the "production" version works just fine. I guess I wasn't commenting something out properly before?

I can include the converted js file directly, and remove the manual bootstrap hack in app.js and everything Just Works

[shankari]

I'm now going to try to use this itinerum example with ngReact. Then last step is to use it with e-mission-phone.
Will deal with webpack etc after discussing packaging and repo structure with @kafitz

[shankari]

It (sort of) works.

I had to remove all the parts with @import and export because otherwise, even with the autogenerated js file, I got errors like

Uncaught SyntaxError: export declarations may only appear at top level of a module

because the generated js file had

export default TutorialComponent;

at the end.

Similarly, for the import statements, we had

Uncaught SyntaxError: import declarations may only appear at top level of a module

But without that, I am not sure how to include the scss

[shankari]

I notice that none of the sample JSX for use with script tags, either in ngReact or in the basic react code
https://gist.githubusercontent.com/gaearon/c8e112dc74ac44aac4f673f2c39d19d1/raw/09b951c86c1bf1116af741fa4664511f2f179f0a/like_button.js
have any import code.

I suspect I need to remove all the imports if we want to use the <script tag approach without webpack. I will then need to compile the scss and add the relevant css as a stylesheet directly

[shankari]

Looking through the ngReact code, it looks like the error is from

 33     // a React component name must be specified
 34     if (!name) {
 35       throw new Error('ReactComponent name attribute must be specified');
 36     }
 37

which is in turn called from

256   var reactDirective = function($injector) {
257     return function(reactComponentName, staticProps, conf, injectableProps) {
258       var directive = {
259         restrict: 'E',
260         replace: true,
261         link: function(scope, elem, attrs) {
262           var reactComponent = getReactComponent(reactComponentName, $injector);

The part that is confusing me is that at that point, ngReact hasn't even started looking up the component. It is just seeing if the name exists or not. @kafitz can you add breakpoints or log statements to the ngReact code to figure out what is going on?

Maybe something like

--- /Users/kshankar/OSS/ngReact/ngReact.js	2021-07-11 10:39:22.000000000 -0700
+++ www/lib/ngReact/ngReact.js	2021-08-04 10:55:48.000000000 -0700
@@ -26,10 +26,13 @@
   // available on window
   function getReactComponent( name, $injector ) {
     // if name is a function assume it is component and return it
+    console.log("Trying to get react component with name '"+name+"'");
     if (angular.isFunction(name)) {
+      console.log("It is a function, returning the name directly '"+name+"'");
       return name;
     }

+    console.log("It is not a function, checking whether it exists '"+name+"'");
     // a React component name must be specified
     if (!name) {
       throw new Error('ReactComponent name attribute must be specified');

[shankari]

FYI, I ran with this logging enabled for only the tutorial component, and I got the following logs

[phonegap] [console.log] Trying to get react component with name 'function TutorialComponent() {
[phonegap]         _classCallCheck(this, TutorialComponent);
[phonegap]
[phonegap]         return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).apply(this, arguments));
[phonegap]     }'
[phonegap] [console.log] It is a function, returning the name directly 'function TutorialComponent() {
[phonegap]         _classCallCheck(this, TutorialComponent);
[phonegap]
[phonegap]         return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).apply(this, arguments));
[phonegap]     }'

So that's why it doesn't fail for the TutorialComponent; it is because it is already a function.

Looking at the autogenerated Tutorial.js, I see the same function defined

    function TutorialComponent() {
        _classCallCheck(this, TutorialComponent);

        return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).apply(this, arguments));
    }

Will have to see if @kafitz's new component generates something similar.

[shankari]

@kafitz I tried to compile your script using the babel command line at:
kafitz/e-mission-phone@4e2f853#diff-d5e0313ae90d6090ceb6d48af20e5cc300bc59403bcfc00c7f788fa0632c318dR1

I think I have all the correct versions installed (I had to guess, but got the most recent versions),

    "@babel/cli": "^7.14.8",
    "@babel/core": "^7.14.8",
    "@babel/plugin-proposal-class-properties": "^7.14.5",
    "@babel/preset-env": "^7.14.9",
    "@babel/preset-typescript": "^7.14.5",
    "babel-preset-react-app": "^3.1.2",
    "react-app": "^1.1.2"

but I still get the following error.

NODE_ENV=production npx babel www/components/itinerum_questionnaire/ --out-dir www/components/itinerum_questionnaire/ --extensions ".ts,.tsx" --presets @babel/preset-env,@babel/preset-typescript,react-app --plugins @babel/plugin-proposal-class-properties
Error: Plugin/Preset files are not allowed to export objects, only functions. In /Users/kshankar/e-mission/upgrade_upload_process/node_modules/babel-preset-react-app/index.js
    at createDescriptor (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/@babel/core/lib/config/config-descriptors.js:211:11)
    at createDescriptor.next (<anonymous>)
    at step (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/gensync/index.js:261:32)
    at evaluateAsync (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/gensync/index.js:291:5)
    at /Users/kshankar/e-mission/upgrade_upload_process/node_modules/gensync/index.js:44:11
    at Array.forEach (<anonymous>)
    at Function.async (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/gensync/index.js:43:15)
    at Function.all (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/gensync/index.js:216:13)
    at Generator.next (<anonymous>)
    at createDescriptors (/Users/kshankar/e-mission/upgrade_upload_process/node_modules/@babel/core/lib/config/config-descriptors.js:142:41)

[shankari]

Ah, based on
https://stackoverflow.com/a/58676558/4040267

changed the command line to

NODE_ENV=production npx babel www/components/itinerum_questionnaire/ --out-dir www/components/itinerum_questionnaire/ --extensions ".ts,.tsx" --presets @babel/preset-env,@babel/preset-typescript --plugins @babel/plugin-proposal-class-properties

(removed react-app) and it works now.

[shankari]

@kafitz are you sure you are able to run with the react_app babel plugin?

Note that for my transpiling, I used

npx babel www/components/itinerum_tutorial/ --out-dir www/components/itinerum_tutorial/ --presets react-app/prod

This used to work just fine with babel 6.x, but it fails with babel 7.x.

$ npx babel --version
7.14.8 (@babel/core 7.14.8)
$ npx babel www/components/itinerum_tutorial/ --out-dir www/components/itinerum_tutorial/ --presets react-app/prod
Error: Cannot find module 'react-app/prod'

Going back to 6.x, it works again

$ npx babel --version
6.26.0 (babel-core 6.26.3)
$ npx babel www/components/itinerum_tutorial/ --out-dir www/components/itinerum_tutorial/ --presets react-app/prod
www/components/itinerum_tutorial/Tutorial.js -> www/components/itinerum_tutorial/Tutorial.js
www/components/itinerum_tutorial/Tutorial.jsx -> www/components/itinerum_tutorial/Tutorial.js

You saw that I had to remove the react-app plugin to get your command line to work
#643 (comment)

How did you manage to get that preset to work? I want to check the output with that included.

[kafitz]

I was digging into the differences of the presets earlier, it looks like the babel-preset-react-app that you had includes things that are now part of the normal @babel/preset-react, so I had been experimenting with that in --plugins (I think we still need one of them). Both seem to yield similar outputs.

My current package,json looks like this:

  "devDependencies": {
    "@ionic/cli": "6.10.1",
    "bower": "1.8.8",
    "cordova": "9.0.0",
    "phonegap": "9.0.0+cordova.9.0.0"
  },
  "dependencies": {
    "@babel/cli": "^7.14.8",
    "@babel/core": "^7.14.8",
    "@babel/preset-env": "^7.14.9",
    "@babel/preset-react": "^7.14.5",
    "@babel/preset-typescript": "^7.14.5",
    "fs-extra": "^9.0.1",
    "klaw-sync": "^6.0.0"
  }

and my compilation command is:

NODE_ENV=production npx babel www/components/itinerum_questionnaire/ --out-dir www/components/itinerum_questionnaire/ --extensions ".ts,.tsx" --presets @babel/preset-env,@babel/preset-typescript,@babel/preset-react --plugins @babel/plugin-proposal-class-properties

NOTE: in version 7, i was only able to get the older react-app preset to "work" by removing the trailing /prod.

The significant difference I see in the output is that function you called attention to:

var TutorialComponent = function (_React$Component) {
    _inherits(TutorialComponent, _React$Component);

    function TutorialComponent() {
        _classCallCheck(this, TutorialComponent);

        return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).apply(this, arguments));
    }

    _createClass(TutorialComponent, [{
        ...transpiled JSX with "React.createElement" elements...
    }]);

    return TutorialComponent;
}(React.Component);

My version looks like this after running babel:

var ItinerumQuestionStack = function ItinerumQuestionStack(props) {
  return /*#__PURE__*/React.createElement("div", null, "HELLO WORLD!");
};

and seems to be missing most of the guts from the class version. It's odd because I did find some issues threads that indicate this should be possible:

• Stateless components do not render correctly ngReact/ngReact#142
• Props not passed to stateless components  ngReact/ngReact#172

I will have to focus on some things for a few hours and will be able to resume debugging this evening.

[shankari]

FYI: I got the react-app/prod command directly from the react website, so I don't think it is completely obsolete.
https://reactjs.org/docs/add-react-to-a-website.html#run-jsx-preprocessor

If you remove the parts of your code that require babel 7 and run the exact same command line that I use, do you get the same class structure? if so, then we know that the problem is with babel 6 v/s babel 7 and can figure out which way to go forward.

[kafitz]

The issue with v6 is I didn't think Typescript had a compatible preset so I get stuck there first.

[shankari]

Looks like this works with CRA
(facebook/create-react-app#5443)
so maybe see how they are using babel under the hood?

[kafitz]

Progress! Although I'm not entirely certain how to get Babel to fully cooperate yet...if I remove the various import/exports from my generated QuestionLayout.js, I see my "HELLO WORLD!" text.

// "use strict";

// Object.defineProperty(exports, "__esModule", {
//   value: true
// });
// exports.ItinerumQuestionStack = void 0;

/* 
 * www/itinerum_questionnaire/QuestionStack.tsx
 */
var questionMap = {
  dropdown: undefined,
  checkbox: undefined,
  number: undefined,
  address: undefined,
  textBox: undefined
};

var ItinerumQuestionStack = function ItinerumQuestionStack(props) {
  return /*#__PURE__*/React.createElement("div", null, "HELLO WORLD!");
};

// exports.ItinerumQuestionStack = ItinerumQuestionStack;

This was generated by the babel command:

NODE_ENV=production npx babel www/components/itinerum_questionnaire/ --out-dir www/components/itinerum_questionnaire/ --extensions ".ts,.tsx"

and a .babelrc file that I was temporarily using:

{
  "presets": [
    "@babel/preset-env",
    "@babel/preset-typescript",
    [ "react-app", { "absoluteRuntime": false } ]
  ],
  "plugins": [
    "@babel/plugin-proposal-class-properties" 
  ]
}

[kafitz]

Unfortunately there's still something causing it to break whenever I attempt to include an import from node_modules. I get the same Error: ReactComponent name attribute must be specified.

Such as:

/* 
 * www/itinerum_questionnaire/QuestionStack.tsx
 */
import React, { useRef } from 'react';

import { StackQuestion } from './questionnaireTypes';


const questionMap: {[key: string]: any} = {
    dropdown: undefined,
    checkbox: undefined,
    number: undefined,
    address: undefined,
    textBox: undefined,
};


export interface QuestionStackProps {
    questions: StackQuestion[];
}

const ItinerumQuestionStack = (props: QuestionStackProps) => {
    const stackRef = useRef(null);

    return (
        <div>
            abc abc
        </div>
    );
}

generating:

// "use strict";

var _typeof = require("@babel/runtime/helpers/typeof");

var _react = _interopRequireWildcard(require("react"));

function _getRequireWildcardCache(nodeInterop) { if (typeof WeakMap !== "function") return null; var cacheBabelInterop = new WeakMap(); var cacheNodeInterop = new WeakMap(); return (_getRequireWildcardCache = function _getRequireWildcardCache(nodeInterop) { return nodeInterop ? cacheNodeInterop : cacheBabelInterop; })(nodeInterop); }

function _interopRequireWildcard(obj, nodeInterop) { if (!nodeInterop && obj && obj.__esModule) { return obj; } if (obj === null || _typeof(obj) !== "object" && typeof obj !== "function") { return { "default": obj }; } var cache = _getRequireWildcardCache(nodeInterop); if (cache && cache.has(obj)) { return cache.get(obj); } var newObj = {}; var hasPropertyDescriptor = Object.defineProperty && Object.getOwnPropertyDescriptor; for (var key in obj) { if (key !== "default" && Object.prototype.hasOwnProperty.call(obj, key)) { var desc = hasPropertyDescriptor ? Object.getOwnPropertyDescriptor(obj, key) : null; if (desc && (desc.get || desc.set)) { Object.defineProperty(newObj, key, desc); } else { newObj[key] = obj[key]; } } } newObj["default"] = obj; if (cache) { cache.set(obj, newObj); } return newObj; }

/* 
 * www/itinerum_questionnaire/QuestionStack.tsx
 */
var questionMap = {
  dropdown: undefined,
  checkbox: undefined,
  number: undefined,
  address: undefined,
  textBox: undefined
};

var ItinerumQuestionStack = function ItinerumQuestionStack(props) {
  var stackRef = (0, _react.useRef)(null);
  return /*#__PURE__*/_react["default"].createElement("div", null, "abc abc");
};

Continuing to investigate.

[shankari]

@kafitz did you see the example with es6 and webpack (https://github.com/ngReact/ngReact/tree/master/examples/es6-tests-webpack). That does seem to have some imports e.g. https://github.com/ngReact/ngReact/blob/master/examples/es6-tests-webpack/lib/watch-list-component.js#L1

I didn't try to run that, since it didn't seem to be needed. LMK if that is closer to what you need, and I can see whether it works.

[shankari]

@kafitz not sure if this helps, but I have some additional information. I basically added all the tutorial stuff back as well so I could compare the tutorial and the react components side by side.

I first realized that the reactComponentName passed in was the full function for the tutorial component and undefined for the questionnaire

Creating directive with name = 'function TutorialComponent() {
        _classCallCheck(this, TutorialComponent);

        return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).apply(this, arguments));
    }'


Creating directive with name = 'undefined'

I then tried accessing the variables we pass in to the directives. TutorialComponent is a function

TutorialComponent
ƒ TutorialComponent() {
        _classCallCheck(this, TutorialComponent);

        return _possibleConstructorReturn(this, (TutorialComponent.__proto__ || Object.getPrototypeOf(TutorialComponent)).appl…

but the react component is undefined

ItinerumQuestionStack
undefined

do you have to do anything special to "export" the ItinerumQuestionStack?

[kafitz]

That example from two comments back seems to be what I'm looking for, but haven't managed it myself yet. Babel is fine when I import from a sibling file (since it generates that file .js alongside and removes the import), so I think it's generating an import/require() for the 3rd-party libraries that ngReact isn't liking.

I also noticed similar regarding the reactComponentName being the full function definition, so it made me slightly more confused how my stateless component even displayed (will still have to log that and see).

Generally speaking, I would a export default ItinerumQuestionStack; at the bottom of my component files which is what I thought could be causing these problems initially. I notice they have it still in your example above. When I manually commented out the use strict and the:

Object.defineProperty(exports, "__esModule", {
    value: true
});

at the top of my generated files as well as that bottom export, that's when it did seem to briefly work.

[kafitz]

Primarily my reservations have been having to completely re-implement my state handling and JSON logic. My original JS-only version was too messy to consider here, so this TypeScript version was a complete rewrite. So long as we can continue to use TypeScript, then it probably won't be such a big deal (just time consuming) to reformat the state components as class style ones.

[shankari]

There are a couple of other things we can try. The issue that you linked to a couple of comments above said something like

The interesting thing is that <react-component syntax works just fine.

So we could try the directive and see what it gives us.
https://github.com/ngReact/ngReact#the-react-component-directive

I would also suggest first trying to get stateless components to work with the ngReact examples, since there are code examples there and it is faster to develop directly in the browser.

We can also pass the name of the component

Alternatively you can provide the name of the component

which will trigger the alternate code path where ngReact tries to use $injector.get to find it. Maybe that works better with the stateless export?

UPDATE: Reading the injector documentation, the second path doesn't seem likely, but I don't understand angular at a deep level, so maybe worth a try?

I can try playing around with this on the ngReact repo over the weekend if you give me an MRE. Is that just www/components/itinerum_questionnaire?

[kafitz]

I've still not gotten to the root of the issue but have committed my latest half-working state here: https://github.com/kafitz/e-mission-phone/ under the same explore_react_integration branch.

I've just noticed that the package.json is not committed by default, so I'll include mine here for now:

{
  "name": "edu.berkeley.eecs.emission",
  "version": "2.5.0",
  "displayName": "emission",
  "license": "BSD-3-Clause",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/e-mission/e-mission-phone.git"
  },
  "scripts": {
    "setup-serve": "./bin/download_settings_controls.js && ./bin/setup_autodeploy.js",
    "compile-react": "babel www/components/itinerum_tutorial/ --out-dir www/components/itinerum_tutorial/ --presets react-app/prod",
    "serve": "phonegap --verbose serve"
  },
  "devDependencies": {
    "@ionic/cli": "6.10.1",
    "bower": "1.8.8",
    "cordova": "9.0.0",
    "phonegap": "9.0.0+cordova.9.0.0"
  },
  "dependencies": {
    "@babel/cli": "^7.14.8",
    "@babel/core": "^7.15.0",
    "@babel/preset-env": "^7.14.9",
    "@babel/preset-typescript": "^7.14.5",
    "@babel/runtime": "^7.14.8",
    "@material-ui/core": "^4.12.3",
    "babel-preset-react-app": "^10.0.0",
    "fs-extra": "^9.0.1",
    "klaw-sync": "^6.0.0"
  }
}

The only files I've been editing are the app.js, the survey.html and the .tsx files within that www/components/itinerum_questionnaire directory. That directory contains a very basic ItinerumQuestionStack.tsx that I've been trying to get to work first with its imports, and a ItinerumQuestionStack.orig.tsx which was originally supposed to be my reference example. The only additional file will be the tsconfig.json in the repo root.

You will see in the app.js/survey.html files commented out versions of the alternatives from the ngReact thread above. They all seem to mostly work with the TutorialComponent and my basic "Hello World" ItinerumQuestionStack.tsx one.

For compiling, my command is:

NODE_ENV=production npx babel www/components/itinerum_questionnaire/ --out-dir www/components/itinerum_questionnaire/ --extensions ".ts,.tsx" --presets @babel/preset-env,@babel/preset-typescript,react-app --plugins @babel/plugin-proposal-class-properties

which is also in a compile.txt file within my itinerum_questionnaire dir.

After that, it's the normal npm run serve.

I think part of my slowdown is not knowing how to debug off the phone so I'm often making changes, reloading em-dev and seeing if it worked. Is there a better way I should be doing it?

[shankari]

@kafitz that's why I think it might be easier to work off the ngReact repo first, since there is no phone component involved. At this point, I think that if it works in ngReact, we can get it to work on the phone.

I have a fork of ngReact where I have been including the prior version of your example. We could make a new directory and explore there first https://github.com/shankari/ngReact/

[kafitz]

I see, I didn't think of trying to use it independently so far. So that would require having a demo Angular app that works in the browser (for now) and testing my .tsx component there first? I'll start doing that next

[shankari]

the examples in ngReact already have a demo angular app. I would start by copying https://github.com/shankari/ngReact/tree/master/examples/itinerum_tutorial and going from there.

[kafitz]

Current status / what I've tried:

• All combinations of babel7 and typescript/react plugins that I can find
• use ngReact example to debug: all errors pertain to "exports" or "require()" not found
• trying as many possible options for tsconfig.json as could be located online
• online indicated that babelify/gulp/webpack needed (in addition to babel) to brings these newer versions of package imports to browser
• looking at ngReact webpack examples did not show me much on how babel is actually called, decided to include webpack
• bower seems to only supply React up to 0.16.1; need at least 0.16.8 (written with/possibly 17)
• use create-react-app to generate a working typescript/react build chain; thinking is to compile JS with that and copy as needed into e-mission-phone www folder
  • added react-app-rewired to easily customize webpack.config.json
  • this at least yielded me a working of example of my stateless React component with imports with the default index.html
  • now able to run ngReact + ItinerumQuestionStack without console errors, but cannot see ItinerumQuestionStack object in app.js (for ngReact directive)
    • no errors are probably a red herring here; assumption is module is unitialized
    • assumption is this has to do with how the React component is initially loaded by create-react-app's index.tsx. This is fine when loading the React component when mounting an index.html element like normal, but stuck on getting app.js to see my component's object variable.

Basic create-react-app example: https://github.com/kafitz/itinerum-questionnaire

Currently working on trying to combine https://github.com/ngReact/ngReact/tree/master/examples/es6-tests-webpack with create-react-app examples

[shankari]

@kafitz that's great progress. @jruzekowicz is making good progress on the last part of the integration (saving to DB and pushing up to server), so might be able to help with this soon as well in case having another person trying things out in parallel would help.

[shankari]

@kafitz have you made any progress on this? I might have some time this weekend to work on this, although my javascriot-fu is pretty weak

[shankari]

or are you giving up on the idea of stateless components and planning to use stateful componets

[kafitz]

I have the stateless components working here: https://github.com/hexmap/emission-plugin-itinerum-questionnaire I worked off the ngReact es6 example using Webpack/babel, but everything at the latest versions.

Most of my trouble this week was in passing my JSON schema from Angular->React, but I think I'd been unnecessarily caught up trying to emulate the Promises to keep it true to the ngReact example. This is finished, so now I'm working on merging my React dashboard code to parse and manage the survey with my Flutter version of the Itinerum mobile survey.

So far no big issues on that, the next big test will be when I implement navigation between survey questions.

[kafitz]

The thing that finally got the stateless version to work was having this factory function: https://github.com/hexmap/emission-plugin-itinerum-questionnaire/blob/master/lib/itinerum-survey-component.tsx#L23

Why this works, however, is still a mystery.

[shankari]

Itinerum is currently on hiatus. We have integrated an enketo toolbox-based survey for the initial survey
#727