Flb/update excel parser accroding to thermodynamics example #336

francescalb · 2021-12-22T12:22:56Z

Description:

Fixes are needed to make the excelparser more robust:

remove trailing and leading white spaces in prefLabels and parents
convert all concept data to str columns (and disregard values that are "nan")
force making an ontology by skipping all errors if desired
imported ontologies are now imported from a separate speradsheet
improved warnings when there are errors in prefLabels or parents
added option of defining 'exptected' when adding data. This means that if a concept is missing a field that is expected a warning will be printed.
if skos and dcterms are missing, elucidation and examples will be written as comment, and altLabels as labels.
Updated test with lines in the input xls with errors/mistakes that the parser should tackle.

Type of change:

Bug fix.
New feature.
Documentation update.

Checklist:

This checklist can be used as a help for the reviewer.

Is the code easy to read and understand?
Are comments for humans to read, not computers to disregard?
Does a new feature has an accompanying new test (in the CI or unit testing schemes)?
Has the documentation been updated as necessary?
Does this close the issue?
Is the change limited to the issue?
Are errors handled for all outcomes?
Does the new feature provide new restrictions on dependencies, and if so is this documented?

Comments:

Much of what is in this file should go into the excel2onto tool.

Added example folder

…ose-302-excelparser

removed rows with missing value in prefLabel Convert prefLabel to string

…ose-302-excelparser

Metadata should be factored out later

Co-authored-by: Jesper Friis <[email protected]>

data argument can either be metadata (pd.DataFrame) or a row for a concept (pd.Series)

…ose-302-excelparser

cannot be added due to errors in prefLabel: prefLabels with spaces, concepts that already exist (These are diregarded - it might be good to add an option for these in the future if it should be allowed to change or add to concepts), concepts where prefLabel is completely missing. Concepts missing elucidation OK. Leading and trailing spaces are removed.

owlready2.Thing

If force: add elucidation as rdfs.comment, add altLAbel as rdfs.label, add example as rdfs.comment

…odynamics_example

…mple' of github.com:emmo-repo/EMMO-python into flb/update-excel-parser-accroding-to-thermodynamics_example

ontopy/excelparser.py

…odynamics_example

CasperWA

Nothing big. I'll return this as a comment review and approve one you've gone over the comments.

ontopy/excelparser.py

tests/test_excelparser.py

tools/excel2onto

francescalb and others added 30 commits November 26, 2021 09:54

Added first draft of excel2onto

8b248fd

Dummy example for make_microstructure_onto.py added.

eab0472

Much of what is in this file should go into the excel2onto tool.

Added module excelparser which is used by excel2onto

e7e2df3

Added example folder

Merge branch 'master' into flb/close-302-excelparser

cf035af

Added choice for sheetname for Concepts

8db19da

Merge branch 'master' of github.com:emmo-repo/EMMO-python into flb/cl…

9c621c5

…ose-302-excelparser

Cleaned up dataframe to be analysed

dd5794a

removed rows with missing value in prefLabel Convert prefLabel to string

Added option to read base_iri from metadata

7db9a8e

Added import ontologies from excel

991278f

Try to fix documentation

ed11bcf

Merge branch 'master' of github.com:emmo-repo/EMMO-python into flb/cl…

b0d9362

…ose-302-excelparser

Fixed documentation error

8d69516

Added pandas to requirements

912ad2b

Corrected google docstring for returning Tuple with typing

ce0476b

Fixed typing error

9bd9e83

Added openpyxl to requirements

d746812

Added first test for adding metadata authors and contributors

83f48b8

Metadata should be factored out later

Added own function for metadata

b4f436c

Added more metadata and altLabels

38199f1

Added altLabel

b81fd0a

Corrected so that empty fields are not included

72dd4b0

owlready2.Ontology->ontopy.Ontology

9f89d10

Co-authored-by: Jesper Friis <[email protected]>

parent=owl:Thing if parents missing in parser

8dca4b2

Co-authored-by: Jesper Friis <[email protected]>

typo

a51dcb1

Co-authored-by: Jesper Friis <[email protected]>

Changed owlready.Ontology to ontopy.ontology.Ontology

091d2cb

Added helper function for adding metadata

b3b7c22

Use function for adding literals also on concepts

0e62f86

_add_literal/_parse_literal accespt pd.Series and pd.DataFrame for data

b527856

data argument can either be metadata (pd.DataFrame) or a row for a concept (pd.Series)

Merge branch 'master' of github.com:emmo-repo/EMMO-python into flb/cl…

9209a4a

…ose-302-excelparser

Added a test for the excel_parser

8a59622

francescalb and others added 13 commits January 4, 2022 16:48

If missing or wrong parents, concepts are defined directly under

b64c394

owlready2.Thing

Added test of concept whos parent is defined later

c59b53f

Work in progress: fixing not-yet-defined-parent

14358c4

Fixed not yet defined parents

e037e9f

Changed all data in conceptata to strings

71984d2

If force: add elucidation as rdfs.comment, add altLAbel as rdfs.label, add example as rdfs.comment

Values converted to sring 'nan' are not considered when adding literal

8336fa8

Added som test for invalid parent

addb7fe

Merge branch 'master' into flb/update-excel-parser-accroding-to-therm…

13fd309

…odynamics_example

Merge branch 'master' into flb/update-excel-parser-accroding-to-therm…

f4f217b

…odynamics_example

updated test for excelparser

8f58a2e

Merge branch 'flb/update-excel-parser-accroding-to-thermodynamics_exa…

ffff9cd

…mple' of github.com:emmo-repo/EMMO-python into flb/update-excel-parser-accroding-to-thermodynamics_example

Removed print

702dc9d

francescalb commented Jan 5, 2022

View reviewed changes

ontopy/excelparser.py Outdated Show resolved Hide resolved

parent_name written out correctly in error msg

cfeffec

francescalb requested a review from CasperWA January 5, 2022 09:12

francescalb marked this pull request as ready for review January 5, 2022 09:13

francescalb removed a link to an issue Jan 5, 2022

Excelparser: Automatize emmo-based? #335

Closed

francescalb and others added 2 commits January 5, 2022 16:08

Merge branch 'master' into flb/update-excel-parser-accroding-to-therm…

2848cfe

…odynamics_example

Fixed some pylit issues that resurfaced pon mergin in master

abe9105

CasperWA reviewed Jan 7, 2022

View reviewed changes

Fixes, mostly documentation after review.

7298d73

francescalb requested a review from CasperWA January 7, 2022 14:59

CasperWA and others added 3 commits January 7, 2022 16:17

Fix doc-string indent

a99bee5

Very minor string fix.

83b9d85

Minor doc-string prettification

d43bb72

CasperWA approved these changes Jan 7, 2022

View reviewed changes

f-String corrections

637c35d

francescalb merged commit 2964d0f into master Jan 7, 2022

CasperWA deleted the flb/update-excel-parser-accroding-to-thermodynamics_example branch March 2, 2022 16:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flb/update excel parser accroding to thermodynamics example #336

Flb/update excel parser accroding to thermodynamics example #336

francescalb commented Dec 22, 2021 •

edited

Loading

CasperWA left a comment

Flb/update excel parser accroding to thermodynamics example #336

Flb/update excel parser accroding to thermodynamics example #336

Conversation

francescalb commented Dec 22, 2021 • edited Loading

Description:

Type of change:

Checklist:

Comments:

CasperWA left a comment

Choose a reason for hiding this comment

francescalb commented Dec 22, 2021 •

edited

Loading