Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add datetime support #569

Merged
merged 5 commits into from
May 19, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
* Support user state. ([#525](https://github.com/realm/realm-dart/pull/525))
* Support getting user id and identities. ([#525](https://github.com/realm/realm-dart/pull/525))
* Support user logout. ([#525](https://github.com/realm/realm-dart/pull/525))
* Added support for DateTime properties. ([#569](https://github.com/realm/realm-dart/pull/569))

### Fixed
* Fixed an issue that would result in the wrong transaction being rolled back if you start a write transaction inside a write transaction. ([#442](https://github.com/realm/realm-dart/issues/442))
Expand Down
18 changes: 17 additions & 1 deletion lib/src/native/realm_core.dart
Original file line number Diff line number Diff line change
Expand Up @@ -1351,6 +1351,9 @@ Pointer<realm_value_t> _toRealmValue(Object? value, Allocator allocator) {
return realm_value;
}

const int _microsecondsPerSecond = 1000 * 1000;
const int _nanosecondsPerMicrosecond = 1000;

void _intoRealmValue(Object? value, Pointer<realm_value_t> realm_value, Allocator allocator) {
if (value == null) {
realm_value.ref.type = realm_value_type.RLM_TYPE_NULL;
Expand Down Expand Up @@ -1398,6 +1401,17 @@ void _intoRealmValue(Object? value, Pointer<realm_value_t> realm_value, Allocato
}
realm_value.ref.type = realm_value_type.RLM_TYPE_UUID;
break;
case DateTime:
final microseconds = (value as DateTime).toUtc().microsecondsSinceEpoch;
final seconds = microseconds ~/ _microsecondsPerSecond;
int nanoseconds = _nanosecondsPerMicrosecond * (microseconds % _microsecondsPerSecond);
if (microseconds < 0 && nanoseconds != 0) {
nanoseconds = nanoseconds - _nanosecondsPerMicrosecond * _microsecondsPerSecond;
}
realm_value.ref.values.timestamp.seconds = seconds;
realm_value.ref.values.timestamp.nanoseconds = nanoseconds;
realm_value.ref.type = realm_value_type.RLM_TYPE_TIMESTAMP;
break;
default:
throw RealmException("Property type ${value.runtimeType} not supported");
}
Expand Down Expand Up @@ -1431,7 +1445,9 @@ extension on Pointer<realm_value_t> {
case realm_value_type.RLM_TYPE_BINARY:
throw Exception("Not implemented");
case realm_value_type.RLM_TYPE_TIMESTAMP:
throw Exception("Not implemented");
final seconds = ref.values.timestamp.seconds;
final nanoseconds = ref.values.timestamp.nanoseconds;
return DateTime.fromMicrosecondsSinceEpoch(seconds * _microsecondsPerSecond + nanoseconds ~/ _nanosecondsPerMicrosecond, isUtc: true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there should be .toLocal() at the end. When we pass the value to core we convert it toUtc. Here we are receiving utc from core and we have to convert it back to local before to return.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be converting dates to local because Core doesn't store timezone info. It'll be misleading for developers if we returned local date here. Also, there's no guarantee that the original date the developer passed was in local time - it could be UTC or some completely different timezone.

Copy link
Contributor

@desistefanova desistefanova May 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not needed the core to keep time zone. if you pass a local date from one device, then when you convert toUtc the method will use the current time zone of the device. And when they receive it on another device it will be converted to the device local time.
In case we want the users to manage their dates then we shouldn't convert their dates to UTC in _intoRealmValue method without they to know. Probably have to throw if (value as DateTime).isUtc==false before to change their date.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a little annoying if we force people to always convert dates to utc although I agree it's more correct as we explicitly tell them that Realm will lose the timezone information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When they create a DateTime in Dart it is local by default unless they have used DateTime.utc constructor. It will be good to have at least some warning or api doc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely document it in the main docs site. We could also generate an API doc on the user model property, though not sure how intrusive that will be.

case realm_value_type.RLM_TYPE_DECIMAL128:
throw Exception("Not implemented");
case realm_value_type.RLM_TYPE_OBJECT_ID:
Expand Down
147 changes: 147 additions & 0 deletions test/realm_object_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import 'dart:io';
import 'package:test/test.dart' hide test, throws;
import '../lib/realm.dart';
import '../lib/realm.dart';

import 'test.dart';

Expand Down Expand Up @@ -68,6 +69,20 @@ class _BoolValue {
late bool value;
}

extension on DateTime {
String toNormalizedDateString() {
final utc = toUtc();
desistefanova marked this conversation as resolved.
Show resolved Hide resolved
// This is kind of silly, but Core serializes negative dates as -003-01-01 12:34:56
final utcYear = utc.year < 0 ? '-${utc.year.abs().toString().padLeft(3, '0')}' : utc.year.toString().padLeft(4, '0');

// For some reason Core always rounds up to the next second for negative dates, so we need to do the same
final seconds = utc.microsecondsSinceEpoch < 0 && utc.microsecondsSinceEpoch % 1000000 != 0 ? utc.second + 1 : utc.second;
return '$utcYear-${_format(utc.month)}-${_format(utc.day)} ${_format(utc.hour)}:${_format(utc.minute)}:${_format(seconds)}';
}

static String _format(int value) => value.toString().padLeft(2, '0');
}

Future<void> main([List<String>? args]) async {
print("Current PID $pid");

Expand Down Expand Up @@ -378,4 +393,136 @@ Future<void> main([List<String>? args]) async {
expect(realm.find<BoolValue>(1)!.toJson().replaceAll('"', '').contains("value:true"), isTrue);
expect(realm.find<BoolValue>(2)!.toJson().replaceAll('"', '').contains("value:false"), isTrue);
});

final epochZero = DateTime.fromMillisecondsSinceEpoch(0, isUtc: true);

bool _canCoreRepresentDateInJson(DateTime date) {
// Core has a bug where negative and zero dates are not serialized correctly to json.
// https://jira.mongodb.org/browse/RCORE-1083
if (date.compareTo(epochZero) <= 0) {
return Platform.isMacOS || Platform.isIOS;
}

// Very large dates are also buggy on Android and Windows
if (date.compareTo(DateTime.utc(10000)) > 0) {
return Platform.isMacOS || Platform.isIOS || Platform.isLinux;
}

return true;
}

void expectDateInJson(DateTime? date, String json, String propertyName) {
if (date == null) {
expect(json, contains('"$propertyName":null'));
} else if (_canCoreRepresentDateInJson(date)) {
expect(json, contains('"$propertyName":"${date.toNormalizedDateString()}"'));
}
}

final dates = [
DateTime.utc(1970).add(Duration(days: 100000000)),
DateTime.utc(1970).subtract(Duration(days: 99999999)),
DateTime.utc(2020, 1, 1, 12, 34, 56, 789, 999),
DateTime.utc(2022),
DateTime.utc(1930, 1, 1, 12, 34, 56, 123, 456),
];
for (final date in dates) {
test('Date roundtrips correctly: $date', () {
final config = Configuration([AllTypes.schema]);
final realm = getRealm(config);
final obj = realm.write(() {
return realm.add(AllTypes('', false, date, 0, ObjectId(), Uuid.v4(), 0));
});

final json = obj.toJson();
expectDateInJson(date, json, 'dateProp');

expect(obj.dateProp, equals(date));
});
}

for (final list in [
dates,
<DateTime>{},
[DateTime(0)]
]) {
test('List of ${list.length} dates roundtrips correctly', () {
final config = Configuration([AllCollections.schema]);
final realm = getRealm(config);
final obj = realm.write(() {
return realm.add(AllCollections(dates: list));
});

final json = obj.toJson();
for (var i = 0; i < list.length; i++) {
final expectedDate = list.elementAt(i).toUtc();
if (_canCoreRepresentDateInJson(expectedDate)) {
expect(json, contains('"${expectedDate.toNormalizedDateString()}"'));
}

expect(obj.dates[i], equals(expectedDate));
}
});
}

test('Date converts to utc', () {
final config = Configuration([AllTypes.schema]);
final realm = getRealm(config);

final date = DateTime.now();
expect(date.isUtc, isFalse);

final obj = realm.write(() {
return realm.add(AllTypes('', false, date, 0, ObjectId(), Uuid.v4(), 0));
});

final json = obj.toJson();
expectDateInJson(date, json, 'dateProp');

expect(obj.dateProp.isUtc, isTrue);
expect(obj.dateProp, equals(date.toUtc()));
});

test('Date can be used in queries', () {
final config = Configuration([AllTypes.schema]);
final realm = getRealm(config);

final date = DateTime.now();

realm.write(() {
realm.add(AllTypes('abc', false, date, 0, ObjectId(), Uuid.v4(), 0));
realm.add(AllTypes('cde', false, DateTime.now().add(Duration(seconds: 1)), 0, ObjectId(), Uuid.v4(), 0));
});

var results = realm.all<AllTypes>().query('dateProp = \$0', [date]);
expect(results.length, equals(1));
expect(results.first.stringProp, equals('abc'));
});

test('Date preserves precision', () {
final config = Configuration([AllTypes.schema]);
final realm = getRealm(config);

final date1 = DateTime.now().toUtc();
final date2 = date1.add(Duration(microseconds: 1));
final date3 = date1.subtract(Duration(microseconds: 1));

realm.write(() {
realm.add(AllTypes('1', false, date1, 0, ObjectId(), Uuid.v4(), 0));
realm.add(AllTypes('2', false, date2, 0, ObjectId(), Uuid.v4(), 0));
realm.add(AllTypes('3', false, date3, 0, ObjectId(), Uuid.v4(), 0));
});

final lessThan1 = realm.all<AllTypes>().query('dateProp < \$0', [date1]);
expect(lessThan1.single.stringProp, equals('3'));
expect(lessThan1.single.dateProp, equals(date3));

final moreThan1 = realm.all<AllTypes>().query('dateProp > \$0', [date1]);
expect(moreThan1.single.stringProp, equals('2'));
expect(moreThan1.single.dateProp, equals(date2));

final equals1 = realm.all<AllTypes>().query('dateProp = \$0', [date1]);
expect(equals1.single.stringProp, equals('1'));
expect(equals1.single.dateProp, equals(date1));
});
}
22 changes: 22 additions & 0 deletions test/test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,28 @@ class $RemappedClass {
late List<$RemappedClass> listProperty;
}

@RealmModel()
class _AllTypes {
late String stringProp;
late bool boolProp;
late DateTime dateProp;
late double doubleProp;
late ObjectId objectIdProp;
late Uuid uuidProp;
late int intProp;
}

@RealmModel()
class _AllCollections {
late List<String> strings;
late List<bool> bools;
late List<DateTime> dates;
late List<double> doubles;
late List<ObjectId> objectIds;
late List<Uuid> uuids;
late List<int> ints;
}

String? testName;
final baasApps = <String, BaasApp>{};
final _openRealms = Queue<Realm>();
Expand Down
Loading