Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/import data #291

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Feature/import data #291

wants to merge 2 commits into from

Conversation

eric-burel
Copy link
Contributor

@eric-burel eric-burel commented Sep 11, 2023

Work in progress

  • Exporting a CSV file of the survey now adds the responseId so we can later match the initial response document (the "normalizedResponse" _id is volatile and changed anytime we re-normalize data so it's not enough). I've cleaned up the UI code and the exported fields.
  • I've crafted a script that can flatten the JSON data, so they have a relatively good shape for further processing (each JSON will be spread into multiple columns depending on its keys). It runs relatively fast on the State of CSS data. It's in a separate repo, and only needs Docker to run.
  • Next step will be to craft a small import UI, that takes back a CSV containing "responseId" + whatever additional data we want, like sentiment for freeform response, entity extracted, cluster etc.
    We can store it in a separate collection first, and maybe add a separate step to merge the fields into the normalizedResponses collection so they can be used in the results app for filtering or display.

@vercel
Copy link

vercel bot commented Sep 11, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
charts ❌ Failed (Inspect) Sep 11, 2023 4:45pm

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for devographics-static failed.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/devographics-static/deploys/64ff441e84feea0008df9fb5

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for devographics-graphiql2 canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/devographics-graphiql2/deploys/64ff441e8bd5d10008c2ebff

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for stateofjs2022 canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/stateofjs2022/deploys/64ff441ee97088000874e027

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for state-of-graphql canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/state-of-graphql/deploys/64ff441ee97088000874e022

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for stateofjs canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/stateofjs/deploys/64ff441eba35ee000862ab72

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for css2023 failed.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/css2023/deploys/64ff441e5cfc1b0008b5d38d

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for tokyodev2022 canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/tokyodev2022/deploys/64ff441e8aeeda00083575d3

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for graphiql-devographics canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/graphiql-devographics/deploys/64ff441ebb5a070008787d35

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for stateofcss canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/stateofcss/deploys/64ff441efd08b40008ca94c0

@netlify
Copy link

netlify bot commented Sep 11, 2023

Deploy Preview for devographics-graphiql-internal canceled.

Name Link
🔨 Latest commit 7c70c25
🔍 Latest deploy log https://app.netlify.com/sites/devographics-graphiql-internal/deploys/64ff441e81419e0008d27630

@gatsby-cloud
Copy link

gatsby-cloud bot commented Sep 11, 2023

❌ State of CSS 2022 deploy preview failed

Your build failed. View the build logs.

1 similar comment
@gatsby-cloud
Copy link

gatsby-cloud bot commented Sep 11, 2023

❌ State of CSS 2022 deploy preview failed

Your build failed. View the build logs.

@gatsby-cloud
Copy link

gatsby-cloud bot commented Sep 11, 2023

❌ State of GraphQL 2022 deploy preview failed

Your build failed. View the build logs.

1 similar comment
@gatsby-cloud
Copy link

gatsby-cloud bot commented Sep 11, 2023

❌ State of GraphQL 2022 deploy preview failed

Your build failed. View the build logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant