Skip to content

Commit

Permalink
Merge pull request #7 from ridencww/develop
Browse files Browse the repository at this point in the history 
Release 2.2.4
  • Loading branch information
@ridencww
ridencww authored Mar 23, 2018
2 parents 2dac9a3 + 349a54c commit 16e0ecb
Show file tree
Hide file tree
Showing 8 changed files with 163 additions and 97 deletions.
49 changes: 27 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,44 +128,49 @@ Examples:

## Version History

2.2.4
* Find now returns 1 instead of 0 for empty search strings (e.g., find("abc", """) == 1).
* Refactor MAKEBOOLEAN to use a set of valid boolean expressions that can be parsed.
* Improved detection of syntax errors, especially for tenary statements.

2.2.3
* Improved cacheing and use of pre-parsed expressions, especially with strings containing multiple expressions
* Added option in DATEADD to add/subtract a number of milliseconds
^ Added optional milliseconds parameter to MAKEDATE, default to zero if not provided
* Fixed issue in MAKEDATE where milliseconds were set to current time and not zero
* Fixed issue with ISBOOLEAN that resulted in false negative detections of BOOLEAN values
* Improved cacheing and use of pre-parsed expressions, especially with strings containing multiple expressions.
* Added option in DATEADD to add/subtract a number of milliseconds.
^ Added optional milliseconds parameter to MAKEDATE, default to zero if not provided.
* Fixed issue in MAKEDATE where milliseconds were set to current time and not zero.
* Fixed issue with ISBOOLEAN that resulted in false negative detections of BOOLEAN values.

2.2.2
* Support backslash as escape lead-in character in STRING tokens (e.g., 6\" 3\')
* Fix NullPointerException caused by dereferencing an empty array
* Flag type mismatches and throw ParseException
* Support backslash as escape lead-in character in STRING tokens (e.g., 6\" 3\').
* Fix NullPointerException caused by dereferencing an empty array.
* Flag type mismatches and throw ParseException.

2.2.1
* Added ability to set the timezone to use by the parser
* Allow additional @ at start of field name (e.g., @id, @@id, @@@id)
* Make ParserException.formatMessage public so it can be used by function writers
* Build test.jar to make UnitTestBase and other halpers available to developers writing functions
* Added ability to set the timezone to use by the parser.
* Allow additional @ at start of field name (e.g., @id, @@id, @@@id).
* Make ParserException.formatMessage public so it can be used by function writers.
* Build test.jar to make UnitTestBase and other halpers available to developers writing functions.

2.2.0
* Updated README.md
* Support for two dimensional arrays up to 10000 x 256
* Moved date functions (DATEADD, DATEFORMAT. etc.) into FunctionToolbox
* Support for two dimensional arrays up to 10000 x 256.
* Moved date functions (DATEADD, DATEFORMAT. etc.) into FunctionToolbox.

2.1.0
* Updated README.md
* Moved GUID, LEFTOF, and RIGHTOF into FunctionToolbox
* Access to system properties and environment variables disabled by default
* Fixed processing of NOT
* Fixed precedence of exponentiation operator
* Improved reporting of dangling operators
* Updated README.md.
* Moved GUID, LEFTOF, and RIGHTOF into FunctionToolbox.
* Access to system properties and environment variables disabled by default.
* Fixed processing of NOT.
* Fixed precedence of exponentiation operator.
* Improved reporting of dangling operators.

2.0.0
* Structural refactor
* Structural refactor.
* Provided a set of TokenType operators that most can use as-is.
* Supplied a toolbox of functions that can be optionally used by the basic Parser. Users can continue to define their own custom functions.
* Fixed shallow copy issue with token values for expressions that were cached.

1.0.0 Initial release to GitHub
1.0.0 Initial release to GitHub.

## License

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

<groupId>com.creativewidgetworks</groupId>
<artifactId>expression-evaluator</artifactId>
<version>2.2.3</version>
<version>2.2.4</version>

<packaging>jar</packaging>

Expand Down
115 changes: 73 additions & 42 deletions src/main/java/com/creativewidgetworks/expressionparser/FunctionToolbox.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@
import java.text.DecimalFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.Stack;
import java.util.UUID;
import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;

public class FunctionToolbox {
private Parser parser;

private static final Set<String> SET_TRUE = new HashSet<String>(Arrays.asList(new String[] {"1","on","t","true","y","yes"}));
private static final Set<String> SET_FALSE = new HashSet<String>(Arrays.asList(new String[] {"0","off","f","false","n","no"}));

// Used for isNumber
private final Pattern pattern_NUMBER = Pattern.compile(TokenType.NUMBER.getRegex(new Parser()), Pattern.CASE_INSENSITIVE | Pattern.MULTILINE);

Expand Down Expand Up @@ -700,8 +700,8 @@ public Value _FACTORIAL(Token function, Stack<Token> stack) throws ParserExcepti

/*
* Returns a 1 based index of search within target string
* find("Ralph", "") -> 0
* find("", "lp") -> 0
* find("Ralph", "") -> 1
* find("Ralph", "lp") -> 3
* find("RalphRalph", "lp", 5) -> 8
*/
Expand All @@ -713,12 +713,17 @@ public Value _FIND(Token function, Stack<Token> stack) {
String str = args[0].asString();
String searchFor = args[1].asString();
BigDecimal start = function.getArgc() == 3 ? args[2].asNumber() : BigDecimal.ONE;
if (str != null && searchFor != null &&
str.length() > 0 && searchFor.length() > 0) {
int startAt = start.intValue();
if (startAt < str.length()) {
BigDecimal bd = new BigDecimal(str.indexOf(searchFor, --startAt));
value.setValue(bd.add(BigDecimal.ONE));
if (str != null) {
if (searchFor == null || searchFor.length() == 0) {
value.setValue(BigDecimal.ONE);
} else {
if (str.length() > 0) {
int startAt = start.intValue();
if (startAt < str.length()) {
BigDecimal bd = new BigDecimal(str.indexOf(searchFor, --startAt));
value.setValue(bd.add(BigDecimal.ONE));
}
}
}
}

Expand Down Expand Up @@ -989,17 +994,30 @@ public Value _ISBLANK(Token function, Stack<Token> stack) {

/*
* Returns whether or not the value of the expression is of the type BOOLEAN. Note that this
* method calls MakeBoolean and then tests the result
* method calls MakeBoolean and then tests the result.
* isBoolean(1) -> true
* isBoolean(1==1) -> true
* isBoolean("1") -> true
* makeBoolean("1.0") -> true
* makeBoolean("0") -> false
* makeBoolean("0.0") -> false
* makeBoolean("true") -> true
* makeBoolean("yes") -> true
* makeBoolean("on") -> true
* makeBoolean("X") -> false
* makeBoolean("<null>") -> false
* makeBoolean("2.0") -> false
* isBoolean("1.0") -> true
* isBoolean("on") -> true
* isBoolean("t") -> true
* isBoolean("true") -> true
* isBoolean("y") -> true
* isBoolean("yes") -> true
*
* isBoolean(0) -> true
* isBoolean(1==0) -> true
* isBoolean("0") -> true
* isBoolean("0.0") -> true
* isBoolean("off") -> true
* isBoolean("f") -> true
* isBoolean("false") -> true
* isBoolean("n") -> true
* isBoolean("no") -> true
*
* isBoolean("<null>") -> false
* isBoolean("") -> false
* isBoolean("noway") -> false *
*/
public Value _ISBOOLEAN(Token function, Stack<Token> stack) {
Value value = _MAKEBOOLEAN(function, stack);
Expand Down Expand Up @@ -1168,38 +1186,51 @@ public Value _LOWER(Token function, Stack<Token> stack) {
}

/*
* Create a BOOLEAN value from an input string
* Create a BOOLEAN value from an input string - DATE types always return NULL/FALSE
* makeBoolean(1) -> true
* makeBoolean(1==1) -> true
* makeBoolean("1") -> true
* makeBoolean("1.0") -> true
* makeBoolean("0") -> false
* makeBoolean("0.0") -> false
* makeBoolean("on") -> true
* makeBoolean("t") -> true
* makeBoolean("true") -> true
* makeBoolean("y") -> true
* makeBoolean("yes") -> true
* makeBoolean("on") -> true
* makeBoolean("X") -> null
*
* makeBoolean(0) -> false
* makeBoolean(1==0) -> false
* makeBoolean("0") -> false
* makeBoolean("0.0") -> false
* makeBoolean("off") -> false
* makeBoolean("f") -> false
* makeBoolean("false") -> false
* makeBoolean("n") -> false
* makeBoolean("no") -> false
*
* makeBoolean("<null>") -> null
* makeBoolean("2.0") -> null
* makeBoolean("") -> null
* makeBoolean("noway") -> null
*/
public Value _MAKEBOOLEAN(Token function, Stack<Token> stack) {
Value value = new Value(function.getText()).setValue((Boolean) null);

Token token = stack.pop();
if (token.asString() != null && token.getValue().getType() != ValueType.DATE) {
// Try conversion of boolean-like strings
String str = token.asString() + "~";
if (str.length() > 1 && ("1~true~yes~on~").contains(str.toLowerCase())) {
value.setValue(Boolean.TRUE);
} else if (str.length() > 1 && ("0~false~no~off~").contains(str.toLowerCase())) {
value.setValue(Boolean.FALSE);
} else {
// probe for variations of 0 and 1
str = token.asString();
if (pattern_NUMBER.matcher(str).find()) {
BigDecimal bd = new BigDecimal(str);
if (bd.compareTo(BigDecimal.ZERO) == 0) {
value.setValue(Boolean.FALSE);
} else if (bd.compareTo(BigDecimal.ONE) == 0) {
value.setValue(Boolean.TRUE);
String str = token.asString();
if (str != null) {
str = str.toLowerCase();
if (SET_TRUE.contains(str)) {
value.setValue(Boolean.TRUE);
} else if (SET_FALSE.contains(str)) {
value.setValue(Boolean.FALSE);
} else {
if (pattern_NUMBER.matcher(str).find()) {
BigDecimal bd = new BigDecimal(str);
if (bd.compareTo(BigDecimal.ZERO) == 0) {
value.setValue(Boolean.FALSE);
} else if (bd.compareTo(BigDecimal.ONE) == 0) {
value.setValue(Boolean.TRUE);
}
}
}
}
Expand Down
13 changes: 10 additions & 3 deletions src/main/java/com/creativewidgetworks/expressionparser/Parser.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@ public Value eval(String source) {

/*----------------------------------------------------------------------------*/

public List<Token> tokenize(String input, boolean wantWhitespace) {
public List<Token> tokenize(String input, boolean wantWhitespace) throws ParserException {
int offset = 0;
int row = 1;

Expand Down Expand Up @@ -466,10 +466,17 @@ public List<Token> tokenize(String input, boolean wantWhitespace) {
// Remove the NOMATCH signifying end-of-expression
if (tokens.size() > 1) {
int last = tokens.size() - 1;
Token token = tokens.get(last);
if (TokenType.NOMATCH.equals(token.getType())) {
Token lastToken = tokens.get(last);
if (TokenType.NOMATCH.equals(lastToken.getType())) {
tokens.remove(last);
}

// Check for invalid tokens in the expression
for (Token token : tokens) {
if (TokenType.NOMATCH.equals(token.getType())) {
setStatusAndFail(token, "error.invalid_token");
}
}
}

return tokens;
Expand Down
1 change: 1 addition & 0 deletions src/main/resources/com/creativewidgetworks/expressionparser/parser.properties
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ error.index_out_of_range=Index value of {0} is out of the range of 0..{1}
error.invalid_operator=Invalid operator: {0}
error.invalid_operator_boolean=Invalid operator for boolean operations: {0}
error.invalid_regex_pattern=Invalid regex pattern: {0}
error.invalid_token=Syntax error, bad token
error.missing_bracket=Syntax error, missing bracket. Expected {0}
error.missing_parens=Syntax error, missing parenthesis. Expected {0}
error.missing_telse=Syntax error, {0} without a matching {1}
Expand Down
47 changes: 30 additions & 17 deletions src/test/java/com/creativewidgetworks/expressionparser/FunctionToolboxTest.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -460,12 +460,12 @@ public void testFIND() throws Exception {
validateExceptionThrown(parser, "FIND('A','B','C')", "FIND parameter 3 expected type NUMBER, but was STRING", 1, 5);

validateNumericResult(parser, "FIND(null,'abc')", "0");
validateNumericResult(parser, "FIND('AbCdEfG', null)", "0");
validateNumericResult(parser, "FIND('', '')", "0");
validateNumericResult(parser, "FIND('AbCdEfG', '')", "0");
validateNumericResult(parser, "FIND('AbCdEfG', 'cd')", "0");
validateNumericResult(parser, "FIND('AbCdEfG', 'CD')", "0");
validateNumericResult(parser, "FIND('AbCdEfG', 'Cd')", "3");
validateNumericResult(parser, "FIND('AbCdEfG', null)", "1");
validateNumericResult(parser, "FIND('', '')", "1");
validateNumericResult(parser, "FIND('AbCdEfG', '')", "1");
validateNumericResult(parser, "FIND('AbCdEfGCd', 'Cd', 5)", "8");
validateNumericResult(parser, "FIND('AbCdEfGCd', 'Cd', -15)", "3");
validateNumericResult(parser, "FIND('AbCdEfGCd', 'Cd', 15)", "0");
Expand Down Expand Up @@ -825,28 +825,41 @@ public void testMAKEBOOLEAN() throws Exception {
validateExceptionThrown(parser, "MAKEBOOLEAN()", "MAKEBOOLEAN expected 1 parameter(s), but got 0", 1, 12);
validateExceptionThrown(parser, "MAKEBOOLEAN('test', 'test')", "MAKEBOOLEAN expected 1 parameter(s), but got 2", 1, 12);

// Not really boolean evaluations
validateBooleanResult(parser, "MAKEBOOLEAN(NOW())", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN(null)", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('Noway')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN(1.000000000000001)", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN(0)", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('0')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('0.0')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('off')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('OFF')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('false')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('False')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN(1==0)", Boolean.FALSE);

// Boolean.TRUE values
validateBooleanResult(parser, "MAKEBOOLEAN(1)", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN(2-1)", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN(1==1)", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('1')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('1.000')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('on')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('ON')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('t')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('true')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('y')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('yes')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('TRUE')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN('trUe')", Boolean.TRUE);
validateBooleanResult(parser, "MAKEBOOLEAN(1==1)", Boolean.TRUE);

// Boolean.FALSE values
validateBooleanResult(parser, "MAKEBOOLEAN(0)", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN(1==0)", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('0')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('0.000')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('off')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('f')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('false')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('n')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('no')", Boolean.FALSE);

// Verify case insensitive
validateBooleanResult(parser, "MAKEBOOLEAN('tRuE')", Boolean.TRUE);

// Verify exact, case-insensitive match required
validateBooleanResult(parser, "MAKEBOOLEAN('yep')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('atrue')", Boolean.FALSE);
validateBooleanResult(parser, "MAKEBOOLEAN('truefact')", Boolean.FALSE);
}

@Test
Expand Down
Loading

0 comments on commit 16e0ecb

Please sign in to comment.