Additional Functions implementation for Eval Engine

[yliuuuu]

Relevant Issues

• [Closes/Related To] Issue #XXX

Description

• exists
• cardinality
• bit_length
• octet_length
• Extract.
• overlay (rewrite)

Other Information

• Updated Unreleased Section in CHANGELOG: [YES/NO]

  • < If NO, why? >

• Any backward-incompatible changes? [YES/NO]

  • < If YES, why? >
  • < For this purpose, we define backward-incompatible changes as changes that—when consumed—can potentially result in
    errors for users that are using our public APIs or the entities that have public visibility in our code-base. >

• Any new external dependencies? [YES/NO]

  • < If YES, which ones and why? >
  • < In addition, please also mention any other alternatives you've considered and the reason they've been discarded >

• Do your changes comply with the Contributing Guidelines
  and Code Style Guidelines? [YES/NO]

License Information

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (partiql-eval@d749103). Click here to learn what that means.

Additional details and impacted files

@@               Coverage Diff               @@
##             partiql-eval    #1375   +/-   ##
===============================================
  Coverage                ?   50.32%           
  Complexity              ?     1045           
===============================================
  Files                   ?      165           
  Lines                   ?    13129           
  Branches                ?     2452           
===============================================
  Hits                    ?     6607           
  Misses                  ?     5862           
  Partials                ?      660           

Flag	Coverage Δ
CLI	13.77% <ø> (?)
EXAMPLES	80.28% <ø> (?)
LANG	54.71% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Conformance comparison report-Cross Engine

	Base (eval)	legacy	+/-
% Passing	80.72%	92.51%	11.79%
✅ Passing	4696	5382	686
❌ Failing	1122	436	-686
🔶 Ignored	0	0	0
Total Tests	5818	5818	0
Number passing in both: 4524

Number failing in both: 264

Number passing in eval engine but fail in legacy engine: 172

Number failing in eval engine but pass in legacy engine: 858
⁉️ CONFORMANCE REPORT REGRESSION DETECTED ⁉️
The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.
858 test(s) were failing in eval but now pass in legacy. Before merging, confirm they are intended to pass.
The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Conformance comparison report-Cross Commit-EVAL

	Base (d749103)	e7e1dca	+/-
% Passing	77.09%	80.72%	3.63%
✅ Passing	4485	4696	211
❌ Failing	1333	1122	-211
🔶 Ignored	0	0	0
Total Tests	5818	5818	0
Number passing in both: 4485

Number failing in both: 1122

Number passing in Base (d749103) but now fail: 0

Number failing in Base (d749103) but now pass: 211
211 test(s) were previously failing but now pass. Before merging, confirm they are intended to pass
The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Conformance comparison report-Cross Commit-LEGACY

	Base (d749103)	e7e1dca	+/-
% Passing	92.51%	92.51%	0.00%
✅ Passing	5382	5382	0
❌ Failing	436	436	0
🔶 Ignored	0	0	0
Total Tests	5818	5818	0
Number passing in both: 5382

Number failing in both: 436

Number passing in Base (d749103) but now fail: 0

Number failing in Base (d749103) but now pass: 0

[johnedquinn]

Overall looks great. Some minor comments.

[johnedquinn]

QQ: You're the one most familiar with date time semantics, but should this be 0? While it's not a boolean value, in boolean logic, an unknown value gets coerced to a NULL value.

The Ion spec says:

2007-02-23T20:14:33.079Z // The same instant in UTC ("zero" or "zulu")
2007-02-23T20:14:33.079+00:00 // The same instant, with explicit local offset
2007-02-23T20:14:33.079-00:00 // The same instant, with unknown local offset

Which shows that the timestamps themselves are equivalent. But, if you were to want to derive information from your data, I feel as if NULL would be returned upon extracting an unknown offset.

[yliuuuu]

This is a good question.

While I don't know what should the decision be, for the time being I decided to follow the behavior from the existing pipelines, which returns zero (in form of decimal for some reasons)...

EXTRACT(TIMEZONE_HOUR FROM `2024-01-01T23:12:59-00:00`)
   | 
===' 
0.

I think for the time being we can stay with that behavior, but I will add a todo note for this line.

Great call out. Thanks.

[johnedquinn]

LGTM!