Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement function timezone() #1386

Merged
merged 46 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
99da434
Added conversion str to int
realHannes Apr 25, 2024
424d023
templated function for toNumeric, add declaration to NaryExpression.h
realHannes Apr 25, 2024
0117e82
str to num for SparqlExpression implemented + added test
realHannes Apr 25, 2024
35fd0b1
Merge branch 'ad-freiburg:master' into master
realHannes Apr 26, 2024
94356c2
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
decc8ba
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
850152c
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
d650d67
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
46cc697
using now absl::from_chars() and stripping whitespaces for string to …
realHannes Apr 26, 2024
7fc5c28
added new functions to processIriFuntionCall() (for string to number)
realHannes Apr 26, 2024
efb0e24
renaming to: toIntExpression and toDoubleExpression for later more ge…
realHannes Apr 26, 2024
a88537c
made format (clang-format-16)
realHannes Apr 26, 2024
ca1e2e0
Update src/parser/sparqlParser/SparqlQleverVisitor.cpp
realHannes Apr 29, 2024
4adc831
Update src/parser/sparqlParser/SparqlQleverVisitor.cpp
realHannes Apr 29, 2024
d0f0d63
renaming in NaryExpression.h for accordance with other function, addi…
realHannes Apr 29, 2024
a118609
added test coverage for function calls makeIntExpression and make Dou…
realHannes Apr 29, 2024
062052e
toNumeric has now correct behavior and uses absl::from_chars() and st…
realHannes Apr 29, 2024
6d0f42a
made clang-format for NaryExpressionImpl.h
realHannes Apr 29, 2024
f90b8e2
Merge branch 'ad-freiburg:master' into master
realHannes May 6, 2024
fb88493
Merge branch 'ad-freiburg:master' into master
realHannes May 15, 2024
b2eb514
Merge remote-tracking branch 'upstream/master'
realHannes May 16, 2024
b165ac1
Merge branch 'ad-freiburg:master' into master
realHannes Jun 1, 2024
7a3dfb2
Merge branch 'master' of https://github.com/realHannes/qlever
realHannes Jun 1, 2024
fc0ad3a
Merge branch 'ad-freiburg:master' into master
realHannes Jun 6, 2024
f3e6086
Merge branch 'ad-freiburg:master' into master
realHannes Jun 7, 2024
fd4c351
Merge branch 'ad-freiburg:master' into master
realHannes Jun 10, 2024
220c9bf
Merge branch 'ad-freiburg:master' into master
realHannes Jun 12, 2024
a81cb8a
Merge branch 'ad-freiburg:master' into master
realHannes Jun 13, 2024
acc0109
Merge branch 'ad-freiburg:master' into master
realHannes Jun 14, 2024
cb8e560
Merge branch 'ad-freiburg:master' into master
realHannes Jun 20, 2024
2b39970
Merge branch 'ad-freiburg:master' into master
realHannes Jun 28, 2024
a0101e4
Merge branch 'ad-freiburg:master' into master
realHannes Jun 30, 2024
2d3861b
implemented expression timezone() + anility to compare w.r.t. timezone
realHannes Jun 30, 2024
8432c20
added class for DayTimeDuration (xsd:dayTimeDuration)
realHannes Jul 8, 2024
ed7fad5
Merge branch 'master' into add_timezone_expression
realHannes Jul 8, 2024
f951979
fix merge conflict
realHannes Jul 8, 2024
f876cde
remove std::move
realHannes Jul 8, 2024
a5e5909
one more build fix
realHannes Jul 8, 2024
b0b054e
improve / fix tests
realHannes Jul 8, 2024
1b5ec54
small change to test
realHannes Jul 9, 2024
b52a517
use absl::StrFormat + changes for sonarcloud
realHannes Jul 9, 2024
9302a8d
throw DurationOverFlowException in correct namespace
realHannes Jul 9, 2024
bf3a24f
applied proposed changes
realHannes Jul 12, 2024
caa9b4d
use initialization list
realHannes Jul 12, 2024
0fdd658
update src/util/Duration.h
realHannes Jul 15, 2024
52fecaa
update src/util/Duration.h
realHannes Jul 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions src/engine/sparqlExpressions/DateExpressions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,14 @@

namespace sparqlExpression {
namespace detail {

using LiteralOrIri = ad_utility::triple_component::LiteralOrIri;
using Literal = ad_utility::triple_component::Literal;

// Date functions.
// The input is `std::nullopt` if the argument to the expression is not a date.

//______________________________________________________________________________
inline auto extractYear = [](std::optional<DateOrLargeYear> d) {
if (!d.has_value()) {
return Id::makeUndefined();
Expand All @@ -16,6 +22,7 @@ inline auto extractYear = [](std::optional<DateOrLargeYear> d) {
}
};

//______________________________________________________________________________
inline auto extractMonth = [](std::optional<DateOrLargeYear> d) {
// TODO<C++23> Use the monadic operations for std::optional
if (!d.has_value()) {
Expand All @@ -28,6 +35,7 @@ inline auto extractMonth = [](std::optional<DateOrLargeYear> d) {
return Id::makeFromInt(optionalMonth.value());
};

//______________________________________________________________________________
inline auto extractDay = [](std::optional<DateOrLargeYear> d) {
// TODO<C++23> Use the monadic operations for `std::optional`.
if (!d.has_value()) {
Expand All @@ -40,9 +48,7 @@ inline auto extractDay = [](std::optional<DateOrLargeYear> d) {
return Id::makeFromInt(optionalDay.value());
};

using LiteralOrIri = ad_utility::triple_component::LiteralOrIri;
using Literal = ad_utility::triple_component::Literal;

//______________________________________________________________________________
inline auto extractStrTimezone =
[](std::optional<DateOrLargeYear> d) -> IdOrLiteralOrIri {
if (d.has_value()) {
Expand All @@ -53,6 +59,23 @@ inline auto extractStrTimezone =
return Id::makeUndefined();
};

//______________________________________________________________________________
inline auto extractTimezoneDurationFormat =
realHannes marked this conversation as resolved.
Show resolved Hide resolved
[](std::optional<DateOrLargeYear> d) -> IdOrLiteralOrIri {
if (d.has_value()) {
const auto& optTimezoneDuration = d.value().getTimezoneAsDurationFromDate();
if (!optTimezoneDuration.has_value()) {
return Id::makeUndefined();
} else {
return LiteralOrIri{Literal::fromStringRepresentation(
absl::StrCat("\""sv, optTimezoneDuration.value(), "\"^^<"sv,
XSD_DAYTIME_DURATION_TYPE, ">"))};
}
}
return Id::makeUndefined();
};

//______________________________________________________________________________
template <auto dateMember, auto makeId>
inline auto extractTimeComponentImpl = [](std::optional<DateOrLargeYear> d) {
if (!d.has_value() || !d->isDate()) {
Expand All @@ -65,19 +88,23 @@ inline auto extractTimeComponentImpl = [](std::optional<DateOrLargeYear> d) {
return std::invoke(makeId, std::invoke(dateMember, date));
};

//______________________________________________________________________________
constexpr auto extractHours =
extractTimeComponentImpl<&Date::getHour, &Id::makeFromInt>;
constexpr auto extractMinutes =
extractTimeComponentImpl<&Date::getMinute, &Id::makeFromInt>;
constexpr auto extractSeconds =
extractTimeComponentImpl<&Date::getSecond, &Id::makeFromDouble>;

//______________________________________________________________________________
NARY_EXPRESSION(YearExpression, 1, FV<decltype(extractYear), DateValueGetter>);
NARY_EXPRESSION(MonthExpression, 1,
FV<decltype(extractMonth), DateValueGetter>);
NARY_EXPRESSION(DayExpression, 1, FV<decltype(extractDay), DateValueGetter>);
NARY_EXPRESSION(TimezoneStrExpression, 1,
FV<decltype(extractStrTimezone), DateValueGetter>);
NARY_EXPRESSION(TimezoneDurationExpression, 1,
FV<decltype(extractTimezoneDurationFormat), DateValueGetter>);
NARY_EXPRESSION(HoursExpression, 1,
FV<decltype(extractHours), DateValueGetter>);
NARY_EXPRESSION(MinutesExpression, 1,
Expand All @@ -99,6 +126,10 @@ SparqlExpression::Ptr makeTimezoneStrExpression(SparqlExpression::Ptr child) {
return std::make_unique<TimezoneStrExpression>(std::move(child));
}

SparqlExpression::Ptr makeTimezoneExpression(SparqlExpression::Ptr child) {
return std::make_unique<TimezoneDurationExpression>(std::move(child));
}

SparqlExpression::Ptr makeMonthExpression(SparqlExpression::Ptr child) {
return std::make_unique<MonthExpression>(std::move(child));
}
Expand Down
1 change: 1 addition & 0 deletions src/engine/sparqlExpressions/NaryExpression.h
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ SparqlExpression::Ptr makeMinutesExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeHoursExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeDayExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeTimezoneStrExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeTimezoneExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeMonthExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeYearExpression(SparqlExpression::Ptr child);

Expand Down
2 changes: 2 additions & 0 deletions src/global/Constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ static const char XSD_DATE_TYPE[] = "http://www.w3.org/2001/XMLSchema#date";
static const char XSD_GYEAR_TYPE[] = "http://www.w3.org/2001/XMLSchema#gYear";
static const char XSD_GYEARMONTH_TYPE[] =
"http://www.w3.org/2001/XMLSchema#gYearMonth";
static const char XSD_DAYTIME_DURATION_TYPE[] =
"http://www.w3.org/2001/XMLSchema#dayTimeDuration";

constexpr inline char XSD_INT_TYPE[] = "http://www.w3.org/2001/XMLSchema#int";
static const char XSD_INTEGER_TYPE[] =
Expand Down
2 changes: 2 additions & 0 deletions src/parser/sparqlParser/SparqlQleverVisitor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2020,6 +2020,8 @@ ExpressionPtr Visitor::visit([[maybe_unused]] Parser::BuiltInCallContext* ctx) {
return createUnary(&makeDayExpression);
} else if (functionName == "tz") {
return createUnary(&makeTimezoneStrExpression);
} else if (functionName == "timezone") {
return createUnary(&makeTimezoneExpression);
} else if (functionName == "now") {
AD_CONTRACT_CHECK(argList.empty());
return std::make_unique<NowDatetimeExpression>(startTime_);
Expand Down
55 changes: 48 additions & 7 deletions src/util/Date.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

#include "util/Log.h"

// ____________________________________________________________________________________________________
// _____________________________________________________________________________
std::string Date::formatTimeZone() const {
auto impl = []<typename T>(const T& value) -> std::string {
if constexpr (std::is_same_v<T, NoTimeZone>) {
Expand All @@ -27,7 +27,7 @@ std::string Date::formatTimeZone() const {
return std::visit(impl, getTimeZone());
}

// ____________________________________________________________________________________________________
// _____________________________________________________________________________
std::pair<std::string, const char*> Date::toStringAndType() const {
std::string dateString;
const char* type = nullptr;
Expand Down Expand Up @@ -64,12 +64,26 @@ std::pair<std::string, const char*> Date::toStringAndType() const {
return {absl::StrCat(dateString, formatTimeZone()), type};
}

// _________________________________________________________________
// _____________________________________________________________________________
std::pair<std::string, const char*> DateOrLargeYear::toStringAndType() const {
if (isDate()) {
return getDateUnchecked().toStringAndType();
}

// If a Date::TimeZone is contained it will be exported as a
// `xsd:dayTimeDuration` type.
if (isTimezone()) {
const auto& optDurationStr = formatTimezoneAsDaytimeDuration(getTimezone());
if (optDurationStr.has_value()) {
return std::pair{optDurationStr.value(), XSD_DAYTIME_DURATION_TYPE};
} else {
// When no timezone was set (Date::NoTimeZone), instead of raising an
// error, just return an empty string as a kind of default for undefined
// xsd:dayTimeDuration.
return std::pair{"", XSD_DAYTIME_DURATION_TYPE};
}
}

using F = absl::ParsedFormat<'d'>;

// Format the year using `format` and return the pair of `(formattedYear,
Expand Down Expand Up @@ -174,7 +188,7 @@ static DateOrLargeYear makeDateOrLargeYear(std::string_view fullInput,
Date{static_cast<int>(year), month, day, hour, minute, second, timeZone}};
}

// __________________________________________________________________________________
// _____________________________________________________________________________
DateOrLargeYear DateOrLargeYear::parseXsdDatetime(std::string_view dateString) {
constexpr static ctll::fixed_string dateTime =
dateRegex + "T" + timeRegex + grp(timeZoneRegex) + "?";
Expand All @@ -193,7 +207,7 @@ DateOrLargeYear DateOrLargeYear::parseXsdDatetime(std::string_view dateString) {
parseTimeZone(match));
}

// __________________________________________________________________________________
// _____________________________________________________________________________
DateOrLargeYear DateOrLargeYear::parseXsdDate(std::string_view dateString) {
constexpr static ctll::fixed_string dateTime =
dateRegex + grp(timeZoneRegex) + "?";
Expand All @@ -209,7 +223,7 @@ DateOrLargeYear DateOrLargeYear::parseXsdDate(std::string_view dateString) {
parseTimeZone(match));
}

// __________________________________________________________________________________
// _____________________________________________________________________________
DateOrLargeYear DateOrLargeYear::parseGYear(std::string_view dateString) {
constexpr static ctll::fixed_string yearRegex = "(?<year>-?\\d{4,})";
constexpr static ctll::fixed_string dateTime =
Expand All @@ -224,7 +238,7 @@ DateOrLargeYear DateOrLargeYear::parseGYear(std::string_view dateString) {
parseTimeZone(match));
}

// __________________________________________________________________________________
// _____________________________________________________________________________
DateOrLargeYear DateOrLargeYear::parseGYearMonth(std::string_view dateString) {
constexpr static ctll::fixed_string yearRegex =
"(?<year>-?\\d{4,})-(?<month>\\d{2})";
Expand All @@ -241,6 +255,23 @@ DateOrLargeYear DateOrLargeYear::parseGYearMonth(std::string_view dateString) {
parseTimeZone(match));
}

// _____________________________________________________________________________
std::optional<std::string> DateOrLargeYear::formatTimezoneAsDaytimeDuration(
Date::TimeZone timezone) const {
auto impl = []<typename T>(const T& value) -> std::optional<std::string> {
if constexpr (std::is_same_v<T, Date::NoTimeZone>) {
return std::nullopt;
} else if constexpr (std::is_same_v<T, Date::TimeZoneZ>) {
return "PT0S";
} else {
static_assert(std::is_same_v<T, int>);
std::string_view sign = value < 0 ? "-" : "+";
return absl::StrCat(sign, "PT", std::abs(value), "H");
}
};
return std::visit(impl, timezone);
}

// _____________________________________________________________________-
int64_t DateOrLargeYear::getYear() const {
if (isDate()) {
Expand Down Expand Up @@ -290,3 +321,13 @@ std::string DateOrLargeYear::getStrTimezone() const {
return "";
}
}

// _____________________________________________________________________-
std::optional<std::string> DateOrLargeYear::getTimezoneAsDurationFromDate()
const {
if (isDate()) {
return formatTimezoneAsDaytimeDuration(getDateUnchecked().getTimeZone());
} else {
return std::nullopt;
}
}
56 changes: 50 additions & 6 deletions src/util/Date.h
Original file line number Diff line number Diff line change
Expand Up @@ -320,17 +320,18 @@ class Date {
// can be currently represented by the year class [-9999, 9999]. The underlying
// format is as follows (starting from the most significant bit):
// - 5 bits that are always zero and ignored by the representation.
// - 2 bits that store 0 (negative year), 1 (datetime with a year in [-9999,
// 9999]), or 2 (positive year).
// - 2 bits that store 1 (negative year), 2 (datetime with a year in [-9999,
// 9999]), 3 (positive year), or 0 (timezone).
// These values are chosen s.t. that the order of the bits is correct.
// - 57 bits that either encode the `Date`, or a year as a signed integer via
// the `ad_utility::NBitInteger` class.
class DateOrLargeYear {
private:
// The tags to discriminate between the stored formats.
static constexpr uint64_t negativeYear = 0;
static constexpr uint64_t datetime = 1;
static constexpr uint64_t positiveYear = 2;
static constexpr uint64_t timezone = 0;
static constexpr uint64_t negativeYear = 1;
static constexpr uint64_t datetime = 2;
static constexpr uint64_t positiveYear = 3;
realHannes marked this conversation as resolved.
Show resolved Hide resolved
// The number of bits for the actual value.
static constexpr uint8_t numPayloadBits = 64 - Date::numUnusedBits;

Expand All @@ -354,6 +355,22 @@ class DateOrLargeYear {
bits_ = std::bit_cast<uint64_t>(d) | (datetime << numPayloadBits);
}

// Construct a `DateOrLargeYear` given a `Timezone` object from the `Date`
// class. This Constructor sets the value of `timezone` (0) as the
// most significant bit. Thus compared with other underlying values a
// `DateOrLargeYear` can hold (`(large) positive year`, `(large) negative
// year` and `dateTime`), the here constructed value will always be smaller.
// (`DateOrLargeYear` compares w.r.t. year, we assume a duration constructed
// from a `TimeZone` doesn't hold a year)
// When comparing two `DateOrLargeYear` objects with two underlying
// (Date)TimeZone as duration values, we implicitly compare only w.r.t. the
// `Date::TimeZone` values, because all values except the timezone get some
// default values.
explicit DateOrLargeYear(Date::TimeZone tz) {
bits_ = std::bit_cast<uint64_t>(Date{0, 0, 0, -1, 0, 0.0, tz}) |
(timezone << numPayloadBits);
}
realHannes marked this conversation as resolved.
Show resolved Hide resolved

// Construct from a `year`. Only valid if the year is outside the legal range
// for a year in the `Date` class.
explicit DateOrLargeYear(int64_t year, Type type) {
Expand All @@ -372,6 +389,9 @@ class DateOrLargeYear {
// True iff a complete `Date` is stored and not only a large year.
bool isDate() const { return bits_ >> numPayloadBits == datetime; }

// True iff constructed with `Date::TimeZone`.
bool isTimezone() const { return bits_ >> numPayloadBits == timezone; };
realHannes marked this conversation as resolved.
Show resolved Hide resolved

// Return the underlying `Date` object. The behavior is undefined if
// `isDate()` is `false`.
Date getDateUnchecked() const { return std::bit_cast<Date>(bits_); }
Expand All @@ -383,6 +403,19 @@ class DateOrLargeYear {
return getDateUnchecked();
}

// Return the underlying `Date::TimeZone` object. The behavior is undefined if
// `isTimeZone()` is `false`.
Date::TimeZone getTimezoneUnchecked() const {
return std::bit_cast<Date>(bits_).getTimeZone();
}

// Return the underlying `Date::TimeZone` object. An asserstion fails if
// `isTimezone()` is `false`
Date::TimeZone getTimezone() const {
AD_CONTRACT_CHECK(bits_ >> numPayloadBits == timezone);
return getTimezoneUnchecked();
}

// Get the stored year, no matter if it's stored inside a `Date` object or
// directly.
int64_t getYear() const;
Expand All @@ -401,6 +434,16 @@ class DateOrLargeYear {
// `tz()`-function.
std::string getStrTimezone() const;

// Correctly format a `xsd:dayTimeDuration` for the underlying `TimeZone`.
// (if a value w.r.t. `TimeZone` was given).
std::optional<std::string> formatTimezoneAsDaytimeDuration(
Date::TimeZone tz) const;

// If a Date is contained w.r.t. this `DateOrLargeYear`, retrieve the
// contained timezone as a suitable string for `xsd:dayTimeDuration`. In case
// no `Date` is contained, return `std::nullopt`.
std::optional<std::string> getTimezoneAsDurationFromDate() const;

Type getType() const {
return static_cast<Type>(ad_utility::bitMaskForLowerBits(numTypeBits) &
bits_);
Expand All @@ -410,7 +453,8 @@ class DateOrLargeYear {
// pointer to the IRI of the corresponding datatype (currently always
// `xsd:dateTime`). Large years are currently always exported as
// `xsd:dateTime` with January 1, 00:00 hours (This is the
// format used by Wikidata).
// format used by Wikidata). If a timezone (significant bit set to 0) is
// contained, the exported value will be a `xsd:dayTimeDuration` value.
std::pair<std::string, const char*> toStringAndType() const;

// The bitwise comparison also corresponds to the semantic ordering of years
Expand Down
33 changes: 33 additions & 0 deletions test/DateTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -522,11 +522,44 @@ TEST(DateOrLargeYear, Order) {
DateOrLargeYear d1{-12345, DateOrLargeYear::Type::Year};
DateOrLargeYear d2{Date{2022, 7, 16}};
DateOrLargeYear d3{12345, DateOrLargeYear::Type::Year};
DateOrLargeYear d4{Date::NoTimeZone{}};
DateOrLargeYear d5{Date::TimeZoneZ{}};
DateOrLargeYear d6{Date::TimeZone{5}};
DateOrLargeYear d7{Date::TimeZone{-12}};

ASSERT_EQ(d1, d1);
ASSERT_EQ(d2, d2);
ASSERT_EQ(d3, d3);
ASSERT_EQ(d4, d4);
ASSERT_EQ(d5, d5);
ASSERT_EQ(d6, d6);
ASSERT_EQ(d7, d7);
ASSERT_LT(d1, d2);
ASSERT_LT(d2, d3);
ASSERT_LT(d1, d3);
ASSERT_LT(d6, d1);
ASSERT_LT(d4, d1);
ASSERT_LT(d6, d2);
ASSERT_LT(d5, d6);
ASSERT_LT(d4, d6);
ASSERT_LT(d4, d5);
ASSERT_LT(d5, d6);
ASSERT_LT(d7, d6);
ASSERT_LT(d7, d5);
ASSERT_LT(d7, d4);
}

TEST(DateOrLargeYear, testStringAndType) {
const auto& p1 = DateOrLargeYear{Date::NoTimeZone{}}.toStringAndType();
const auto& p2 = DateOrLargeYear{Date::TimeZoneZ{}}.toStringAndType();
const auto& p3 = DateOrLargeYear{Date::TimeZone{5}}.toStringAndType();
const auto& p4 = DateOrLargeYear{Date::TimeZone{-12}}.toStringAndType();
ASSERT_EQ(p1.first, "");
EXPECT_STREQ(p1.second, XSD_DAYTIME_DURATION_TYPE);
ASSERT_EQ(p2.first, "PT0S");
EXPECT_STREQ(p2.second, XSD_DAYTIME_DURATION_TYPE);
ASSERT_EQ(p3.first, "+PT5H");
EXPECT_STREQ(p3.second, XSD_DAYTIME_DURATION_TYPE);
ASSERT_EQ(p4.first, "-PT12H");
EXPECT_STREQ(p4.second, XSD_DAYTIME_DURATION_TYPE);
}
Loading
Loading