Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We need to split our big object with grocery lists into separated files, each file = one grocery list object #424

Closed
atherdon opened this issue May 30, 2019 · 47 comments · Fixed by #562
Labels
Milestone

Comments

@atherdon
Copy link
Contributor

i keep it without good task explanation for a reason. want to provoke a discussion here.
Please ask your questions. @uniyalprashant9

@atherdon
Copy link
Contributor Author

@amogh-jrules as it's a big task - let's go from it. but it pretty simple

@0xamogh
Copy link
Contributor

0xamogh commented May 31, 2019

@atherdon Do we have to write scripts which take in a large object and break it down and generate smaller files?

@atherdon
Copy link
Contributor Author

if you want - yes. we actually have our generator script - that you can use for this kind of task.
my goal here - to split our old version of file/database into small files in order to easily manipulate them
and by working on this task, same to @uniyalprashant9 - you'll learn about the current structure of grocery lists(which maybe soon will be changed)

@atherdon
Copy link
Contributor Author

for generator explanation - check Readme file

@atherdon
Copy link
Contributor Author

@amogh-jrules @uniyalprashant9 please connect together in order to deal with this task. because we need to solve it together - not twice, so it's about collaboration task, not about competition

@0xamogh
Copy link
Contributor

0xamogh commented May 31, 2019 via email

@uniyalprashant9
Copy link
Contributor

uniyalprashant9 commented May 31, 2019

We have to write a script for splitting these files
https://github.com/GroceriStar/sd/tree/master/src/data/Grocery ( grocery.json and grocery0.json ) into multiple JSON files each with one element ??

@0xamogh
Copy link
Contributor

0xamogh commented May 31, 2019 via email

@atherdon
Copy link
Contributor Author

what is the difference between grocery.json and grocery0? just don't remember

@0xamogh
Copy link
Contributor

0xamogh commented May 31, 2019

I ran it through https://www.diffchecker.com, only one line is different rest all is the same

@atherdon
Copy link
Contributor Author

got it

@uniyalprashant9
Copy link
Contributor

This should be the desired output or need some changes??
Each file has one element from grocery.JSON and the file name is the same as the name attribute of each element.
Screenshot from 2019-05-31 22-34-31

@atherdon
Copy link
Contributor Author

atherdon commented May 31, 2019 via email

@atherdon
Copy link
Contributor Author

atherdon commented May 31, 2019 via email

@uniyalprashant9
Copy link
Contributor

Ok, I will change JSON to json .
I made that separately to test if it works or not, I will put in our module and open a pull request.

@atherdon
Copy link
Contributor Author

atherdon commented May 31, 2019 via email

@uniyalprashant9
Copy link
Contributor

@atherdon I made few changes in the script, now it creates a seperate directory for each object of the same name and put respective elements inside the directory.

If you need all the elements in one folder I will change it again.

I opened a pull request for the same.

@atherdon
Copy link
Contributor Author

  1. We need to create an index.js file and import all of created json files there.
    you can see as example src/files.js so our files will be accessible from the outside
  2. we should write tests for all files(connecting them to tests from index.js that we will create)
  3. we should add information to our readme - so other team members will know about our changes.
  4. how do we can call this script? do we have a package.json script for it?
  5. I think that part of your code can be replaced with an actuall code that we have at generator folder right now.

please go from first point, doing them step-by-step and pushing pull requests separately.

@uniyalprashant9
Copy link
Contributor

@atherdon I didn't understand what is to be done in point 1.

I have to make index.js file which will import all the json files that were splitted from the large object file grocery.json and it will return a list of all the elements??

@atherdon
Copy link
Contributor Author

atherdon commented Jun 1, 2019

i think yes, something similar to https://github.com/GroceriStar/sd/blob/master/src/index.js

@uniyalprashant9
Copy link
Contributor

Which ourput is required for 1

A)list of files
or
B)an object that has all the content of the splitted files.
A=>
Screenshot from 2019-06-01 22-04-33

B=>
Screenshot from 2019-06-01 22-04-59

@atherdon
Copy link
Contributor Author

atherdon commented Jun 1, 2019 via email

@uniyalprashant9
Copy link
Contributor

OK i will make the first one as index.js and the second one as indexContent.js

@uniyalprashant9
Copy link
Contributor

For 2, I have made a test.
Please check is it ok, then I will open a new pull request.
Writing test is new for me so it took time. 😅

That error in the screenshot was made intentionally to check if it is working or not. Then I reverted it back and made a final one.

Screenshot from 2019-06-02 12-26-33

@uniyalprashant9
Copy link
Contributor

@atherdon I have one doubt, I have used mocha for the test , will it get errors when running it in Travis CI?

@uniyalprashant9
Copy link
Contributor

@atherdon Please check my pull request, I have one more test module to upload. 😅

@uniyalprashant9
Copy link
Contributor

I am currently working on readme file.

@uniyalprashant9
Copy link
Contributor

uniyalprashant9 commented Jun 3, 2019

  1. We need to create an index.js file and import all of created json files there.
    you can see as example src/files.js so our files will be accessible from the outside
  2. we should write tests for all files(connecting them to tests from index.js that we will create)
  3. we should add information to our readme - so other team members will know about our changes.
  4. how do we can call this script? do we have a package.json script for it?
  5. I think that part of your code can be replaced with an actuall code that we have at generator folder right now.

please go from first point, doing them step-by-step and pushing pull requests separately.

I am merging splitObject.js in the existing writeFile.js file in generators and then will do a readme.MD update.

@atherdon
Copy link
Contributor Author

atherdon commented Jun 4, 2019

Ok, but please be careful with changes. Or create tests or ask for advice about tests as well.

@atherdon
Copy link
Contributor Author

atherdon commented Jun 4, 2019

Btw, it's still not clear for me - are we merge your changes or not? @uniyalprashant9 - you made some moves and my head is going around

@uniyalprashant9
Copy link
Contributor

Ok, but please be careful with changes. Or create tests or ask for advice about tests as well.

Yes I will

@uniyalprashant9
Copy link
Contributor

  1. We need to create an index.js file and import all of created json files there.
    you can see as example src/files.js so our files will be accessible from the outside
  2. we should write tests for all files(connecting them to tests from index.js that we will create)
  3. we should add information to our readme - so other team members will know about our changes.
  4. how do we can call this script? do we have a package.json script for it?
  5. I think that part of your code can be replaced with an actuall code that we have at generator folder right now.

please go from first point, doing them step-by-step and pushing pull requests separately.

splitObject() is in writeFile.js to split json files
getList() and getListContent() are in utils.js to retrive json files.
readme is updated.
how to call those functions is explained in readme.md.
the old splitObject.js is already merged in existing file (writeFile.js)

@uniyalprashant9
Copy link
Contributor

Only test part is remaining.

@uniyalprashant9
Copy link
Contributor

@atherdon

@atherdon
Copy link
Contributor Author

can you handle a testing part on your own or do you need help with it? we have a pretty cool test folder where you may find some similar tests that you can copy and connect to this task

@uniyalprashant9
Copy link
Contributor

uniyalprashant9 commented Jun 12, 2019

can you handle a testing part on your own or do you need help with it? we have a pretty cool test folder where you may find some similar tests that you can copy and connect to this task

@atherdon Most of the tests are to match the schema of json files, my modules split large json files into single files, retrieve list of files in the directory, they don't need tests I guess splitObject will work on parameters passed.

@atherdon
Copy link
Contributor Author

atherdon commented Jun 13, 2019 via email

@atherdon
Copy link
Contributor Author

atherdon commented Jun 13, 2019 via email

@atherdon atherdon added enhancement New feature or request in-progress labels Jun 14, 2019
@atherdon atherdon added this to the Generator milestone Jun 14, 2019
@atherdon
Copy link
Contributor Author

@uniyalprashant9 what is left for this task?

@uniyalprashant9
Copy link
Contributor

@atherdon only tests are remaining

@atherdon
Copy link
Contributor Author

atherdon commented Jun 16, 2019 via email

@uniyalprashant9
Copy link
Contributor

yes i will :)

@atherdon
Copy link
Contributor Author

@vadim9999 garb it here too. looks like we almost finished it but i'm have doubts that all of my comments was actually done/implemented

@vadim9999
Copy link
Collaborator

ok

@vadim9999
Copy link
Collaborator

I think it's done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants