CSVStorage and Lineendings #298

tom-wa · 2015-09-17T23:03:30Z

No description provided.

markus2330 · 2015-09-18T05:37:32Z

src/plugins/ni/nickel-1.1.0/src/tests/ni_parse_oddities.ini

-\[not a section\]=yeah
-loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooongggggggg=truncated
-something = ; nothing
+a = \777


This should not be commited.

markus2330 · 2015-09-18T06:52:58Z

src/plugins/lineendings/README.md

+## Introduction ##
+
+The Lineendings Plugin verifies the Lineendings of a file.
+If inconsistent lineendings or lineendings that don't match `config/valid` are detected the plugin yields an error.


again, no config/

Can you explain valid a bit more? Does it contain a sequence of lineendings? If \r\r\f is given, how does it know if \r\f should be accepted?

Can you give an example how to use?

valid could be set to CR, LF or CRLF (string value) to tell the plugin to reject all other lineendings. without the valid key, only inconsistent lineendings are rejected
gonna update the readmes

markus2330 · 2015-09-18T07:00:40Z

Overall very nice and clean code, impressive work. Thank you!

markus2330 · 2015-09-18T15:35:05Z

Two more remarks:

you can remove what you have done from the TODO lists
please add the plugins in cmake/ElektraCache

markus2330 · 2015-09-18T15:35:10Z

add to whitelist

markus2330 · 2015-09-18T15:35:16Z

jenkins build please

markus2330 · 2015-09-19T08:56:13Z

src/plugins/enum/README.md

+
+## Usage ##
+
+The plugin checks every Key in the Keyset for the Metakey `check/enum` containing a list with the syntax `['string1', 'string2', 'string3', ..., 'stringN']` and compares each value with the string value of the Key. If no match is found an error is returned.


About the syntax: Why is it surrounded with []? It actually does make sense if you also can specify an enum somewhere else (e.g. in the plugin configuration) and reuse it check/enum (then without square brackets).

I am wondering how fast the check will perform with an enum of 700 values, copied to 700 places? A reference to the plugins configuration, and not copying the enum values, is certainly faster, but maybe its a premature optimization.

And more about the syntax: how is it possible to use ' within the string?

One way would be escaping with \, another one would be to provide different quotes (e.g. ' and") and automatically concatinate them, e.g.:
['"' "str'ing" '"', "abc"] would be "str'ing" and abc. Then the usage of , to separate would pay off. What do you think?

Thinking about it, the [] are not even needed when refering to outside defined enums is allowed. You simply know if you refer to something else, when its not quoted in '.

So given a defined abc = 'a', 'b', 'c' one can use check/type with abc, 'd' without ambiguity. What do you think?

The [] isn't needed,i was trying to adapt the syntax of the specification file. Basically every alphanumeric string divided by an non-alphanumeric character is considered as a valid element right now, but i can change it to just use a comma as a valid seperator, values in <> to keys and not quoted values to plugin configured values ?

The [] isn't needed,i was trying to adapt the syntax of the specification file.

The specification could be in any syntax as long as it is parsable by one of Elektras plugin capable of storing meta data. We are now defining the syntax within the value, not the syntax that will be stored on the disc. You could easily write a plugin that wraps every value with `[]`.

Basically every alphanumeric string divided by an non-alphanumeric character is considered as a valid element right now,

So currently only alphanumeric strings can be used within the enum? Its quite restricting, but basically it would suffice for a first version.

but i can change it to just use a comma as a valid seperator

Yes, its better to be strict in the beginning. We can relax it later on if its somehow needed without breaking anyones configuration.

values in <> to keys

I am not sure what you mean here.

and not quoted values to plugin configured values ?

This is would be an optional feature. If you do not have worries that it might be slow in the 700x700 situation, we do not need it for a first version.

By the <> part i meant something like:
system/configs/enums/valid1='a','b','c'
system/configs/enums/valid2='g','h','i'

And the check/enum meta key could look like <enums/valid1>, 'd','e','f',<enums/valid2>

So the plugin would check for a,b,c,d,e,f,g,h

I would prefer to have as little syntactic clutter as possible (aka not having
<>). Syntax should be introduced in the storage plugins, not within values.

One could even argue to introduce an array:

check/enum check/enum/#0=a check/enum/#1=b check/enum/#1=c

Where a, b and c are literals, e.g. serialized in json:

{ "check" : { "enum" : ["a", "b", "c"] } }

Obviously, the data structure would be huge, we would not have nice references
(without introducing syntax) and you would need to reimplement most things.
So the 'a', 'b' for now and var, 'a', 'b', "'" for later seems like a good
middle ground.

Btw. I used JSON as example, because it looks nearly identical to your initial proposal.

markus2330 · 2015-09-21T09:36:04Z

Please ping me when you are ready.

markus2330 · 2015-09-21T16:57:59Z

jenkins build please

markus2330 · 2015-09-21T17:40:08Z

404 buildjob http://build.libelektra.org:8080/job/elektra-mergerequests/404/console reports
http://build.libelektra.org:8080/job/elektra-mergerequests/ws/build/Testing/20150921-1702/DynamicAnalysis.xml

==10564== 3 bytes in 1 blocks are indirectly lost in loss record 1 of 58
==10564==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==10564==    by 0x4E904CB: elektraStrNDup (internal.c:306)
==10564==    by 0x4E89525: keySetMeta (keymeta.c:489)
==10564==    by 0x4F17688: elektraAddWarning118 (kdberrors.h:7671)
==10564==    by 0x4F17688: csvRead (csvstorage.c:207)
==10564==    by 0x4F17688: elektraCsvstorageGet (csvstorage.c:267)
==10564==    by 0x4025E3: testreadwriteinvalid (testmod_csvstorage.c:48)
==10564==    by 0x4025E3: main (testmod_csvstorage.c:65)
==10564== 
==10564== 3 bytes in 1 blocks are indirectly lost in loss record 2 of 58

markus2330 · 2015-09-22T12:52:11Z

src/plugins/csvstorage/csvstorage.c

+	if(readHeaderKey)
+	{
+		const char *printHeaderString = keyString(readHeaderKey);
+		printf("printHeaderString: %s\n", printHeaderString);


should be removed

markus2330 · 2015-09-22T16:46:36Z

Merged, please do not work on this branch.

Leftovers are documented in #306, but are not urgent. (Feel free to add more)

Thank you! Impressive work!

tom-wa added 3 commits September 18, 2015 00:35

Added CSVStorage and Lineendings Plugin

7a26ac7

fixed errors/warnings

5739c4a

Added Enum Plugin

84daebd

markus2330 reviewed Sep 18, 2015
View reviewed changes

Added calculate Plugin

01ceda7

markus2330 reviewed Sep 19, 2015
View reviewed changes

tom-wa force-pushed the master branch from 0f88b55 to 01ceda7 Compare September 19, 2015 16:47

tom-wa added 6 commits September 20, 2015 14:13

Updated CSVStorage and Lieneendings

a4f6d7f

fixed csvstorage

a3b5582

updated enum plugin

982c524

removed calculate plugin from master branch

15112a5

fixed broken testfile and ugly cast in csvstorage

3658360

removed useless presetstorage from lineendings

eab59de

tom-wa added 2 commits September 21, 2015 11:57

updated include_common, removed csvstorage from read-write storage tests

bd0b444

Fixed memory leaks

8609a25

Updated Metadata.ini

357502a

markus2330 reviewed Sep 22, 2015
View reviewed changes

tom-wa added 2 commits September 22, 2015 17:45

fixed memleak and some issues

54bd037

fixed return value for get

03800c2

markus2330 closed this Sep 22, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CSVStorage and Lineendings #298

CSVStorage and Lineendings #298

tom-wa commented Sep 17, 2015

markus2330 Sep 18, 2015

markus2330 Sep 22, 2015

markus2330 Sep 18, 2015

tom-wa Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 Sep 19, 2015

markus2330 Sep 19, 2015

markus2330 Sep 19, 2015

tom-wa Sep 19, 2015

markus2330 Sep 19, 2015 via email

tom-wa Sep 19, 2015

markus2330 Sep 19, 2015

markus2330 commented Sep 21, 2015

markus2330 commented Sep 21, 2015

markus2330 commented Sep 21, 2015

markus2330 Sep 22, 2015

markus2330 commented Sep 22, 2015


		## Usage ##

		The plugin checks every Key in the Keyset for the Metakey `check/enum` containing a list with the syntax `['string1', 'string2', 'string3', ..., 'stringN']` and compares each value with the string value of the Key. If no match is found an error is returned.

CSVStorage and Lineendings #298

CSVStorage and Lineendings #298

Conversation

tom-wa commented Sep 17, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

markus2330 commented Sep 18, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markus2330 Sep 19, 2015 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markus2330 commented Sep 21, 2015

markus2330 commented Sep 21, 2015

markus2330 commented Sep 21, 2015

Choose a reason for hiding this comment

markus2330 commented Sep 22, 2015