Use of xsd:decimal instead of xsd:float for conversion factors

[jmkeil]

Using a comparison of unit ontologies with ABECTO I became aware that OM uses xsd:float to represent conversion factors and offsets. I think, instead xsd:decimal should be used, as binary floating point datatypes are not able to exactly represent many of the conversion values. Especially 0.1, 0.01, 0.001, … can not be represented. Only the lexical representation seems to be exact. However, the lexical to value mapping will map them to a slightly different value. This runs the risk of numerical problems in applications. I outlined this in detail in arXiv:2011.08077.

[HajoRijgersberg]

Thanks again so much, Jan Martin, for all your effort, evaluating OM and the other unit ontologies! Also this issue, that you raise here, is very important. I'll read the paper you wrote about it! I'll keep you - again - updated!

[HajoRijgersberg]

I agree that we should use xsd:decimal. I'll process that. But before I can do that I'll have to represent the many scientific notations in OM (with an e notation like in 3e4) to plain decimal numbers, which is required for xsd:decimal. This will take a long time. I'll keep you updated about this process!

[jmkeil]

Maybe some RegEx might ease that?
Examples:

Pattern	replace by	before	after
rdf:datatype="&xsd;float">(\d+)(e|E)3<	rdf:datatype="&xsd;decimal">$1000<	… rdf:datatype="&xsd;float">3141592653e3<…	… rdf:datatype="&xsd;decimal">3141592653000<…
rdf:datatype="&xsd;float">(\d+).(\d)(e|E)3<	rdf:datatype="&xsd;decimal">$1$200<	… rdf:datatype="&xsd;float">314159265.3e3<…	… rdf:datatype="&xsd;decimal">314159265300<…
rdf:datatype="&xsd;float">(\d+).(\d\d)(e|E)3<	rdf:datatype="&xsd;decimal">$1$20<	… rdf:datatype="&xsd;float">31415926.53e3<…	… rdf:datatype="&xsd;decimal">31415926530<…
rdf:datatype="&xsd;float">(\d+).(\d\d\d)(e|E)3<	rdf:datatype="&xsd;decimal">$1<	… rdf:datatype="&xsd;float">3141592.653e3<…	… rdf:datatype="&xsd;decimal">3141592653<…
rdf:datatype="&xsd;float">(\d+).(\d\d\d)(\d*)(e|E)3<	rdf:datatype="&xsd;decimal">$1.$2<	… rdf:datatype="&xsd;float">314159.2653e3<… … rdf:datatype="&xsd;float">31415.92653e3<… …	… rdf:datatype="&xsd;decimal">314159265.3<… … rdf:datatype="&xsd;decimal">31415926.53<… …

[dr-shorthair]

Easier (and safer) to use SPARQL?

I just did similar for QUDT - see qudt/qudt-public-repo#311 (comment)
You should be able to adapt these queries.

[HajoRijgersberg]

Thanks, Jan Martin and Simon, Problem is that I work with an organized ASCII file as source. So I should perform these actions with string manipulation in a text file.
(I am working with an ASCII file to keep the overview in the ontology and to be able to perform complex copy and paste actions for defining "batch" quantities and units, which I still need at the moment due to lack of time for automating the various processes that come with maintaining and extending OM.)
So, I would need the RegEx commands you suggest, Jan Martin, to be converted to replace actions in text.

[jmkeil]

Doing #80 first might ease solving this issue.

[HajoRijgersberg]

Once again, thanx for your suggestions Jan Martin and Simon. Apart from your suggestions, I have started to perform the conversions manually because of the urgency. Jan Martin and I had email contact about it (for me to fully understand the issue and what should be done, i.e., converting to decimals and integers), and I find the issue so important that I have immediately started. I think it will work out in a few weeks (doing in-between other activities). I have done more elaborate things in the past. ;) Wish me luck however. ;)

[jmkeil]

Easier (and safer) to use SPARQL?

I just did similar for QUDT - see qudt/qudt-public-repo#311 (comment) You should be able to adapt these queries.

@dr-shorthair How did you manage to not suffer from the rounding issues? A compliant engine should have create lexicals with tiny offset to the correct value (e.g. 0.00000000999999993922529 for 1.0E-8).

[jmkeil]

Solved by #89.