Skip to content

Commit

Permalink
feat(dynamite)!: add experimental standalone JsonSchema generation
Browse files Browse the repository at this point in the history
Signed-off-by: Nikolas Rimikis <[email protected]>
  • Loading branch information
Leptopoda committed Apr 26, 2024
1 parent 4d7302f commit a1705e0
Show file tree
Hide file tree
Showing 15 changed files with 789 additions and 36 deletions.
14 changes: 8 additions & 6 deletions packages/dynamite/dynamite/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
Provides a [Dart Build System](https://github.com/dart-lang/build) builder for generating clients from [OpenAPI specifications](https://swagger.io/specification/).
Version `0.3.0` adds experimental support for [JsonSchema](https://json-schema.org) models.

The builder generates code if it find files with an `.openapi.json` or `.openapi.yaml` extension in the lib directory.
The builder generates code if it find files with an `.openapi.json`, `.openapi.yaml` or `.json_schema.json` extension in the lib directory.

# Setup

Expand All @@ -11,9 +12,9 @@ dependencies:
built_value: ^8.9.0
collection: ^1.0.0
dynamite_runtime: ^0.1.0
http: ^1.2.0
http: ^1.2.0 # only needed for openapi libraries
meta: ^1.0.0
uri: ^1.0.0
uri: ^1.0.0 # only needed for openapi libraries
dev_dependencies:
build_runner: ^2.4.8
built_value_generator: ^8.9.0
Expand All @@ -25,7 +26,7 @@ To generate code you need to invoke the `build_runner` with the following comman
```sh
dart run build_runner build
```
The builder will look for any files ending with either `.openapi.json` or `.openapi.yaml` and place the generated code next to the specifications in a file ending with `.openapi.dart`.
The builder will look for any files ending with either `.openapi.json`, `.openapi.yaml` or `.json_schema.json` and place the generated code next to the specifications in a file ending with `.openapi.dart` or `.json_schema.dart`.
For a full example checkout the [example package](https://github.com/nextcloud/neon/tree/main/packages/dynamite/dynamite/example) using the OpenAPI petstore specification.


Expand All @@ -37,7 +38,7 @@ You can configure code generation by setting values in the `build.yaml`.
targets:
$default:
builders:
dynamite:
dynamite|dynamite_openapi:
options:
# Options configure how source code is generated.
#
Expand All @@ -48,6 +49,7 @@ targets:
- discarded_futures
- public_member_api_docs
- unreachable_switch_case
# Add coverage ignore comments to fromJson and toJson methods.
coverage_ignores:
- 'const .*\._\(\);'
- 'factory .*\.fromJson\(Map<String, dynamic> json\) => jsonSerializers\.deserializeWith\(serializer, json\)!;'
Expand All @@ -65,7 +67,7 @@ targets:

# Versioning

Dynamite does not yet support the full OpenAPI specification. It currently supports the most common subset of the functionality of the versions 3.0 and 3.1.
Dynamite does not yet support the full OpenAPI or JsonSchema specification. It currently supports the most common subset of the functionality of the OpenAPI versions 3.0 and 3.1 and JsonSchema 2020-12.
Feel free to open an issue if you rely on any functionality not yet supported.
The version number of this package will be updated in regards to the generated code. For example if there is a breaking change in the generated code the major version of this package will be updated.

Expand Down
11 changes: 9 additions & 2 deletions packages/dynamite/dynamite/build.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
builders:
dynamite:
import: 'package:dynamite/builder.dart'
openapi:
import: 'package:dynamite/dynamite.dart'
builder_factories: ['openAPIBuilder']
build_extensions: { '.openapi.json': ['openapi.dart'] }
auto_apply: dependents
build_to: source
runs_before: ['built_value_generator|built_value']
json_schema:
import: 'package:dynamite/dynamite.dart'
builder_factories: ['jsonSchemaBuilder']
build_extensions: { '.json_schema.json': ['json_schema.dart'] }
auto_apply: dependents
build_to: source
runs_before: ['built_value_generator|built_value']
2 changes: 1 addition & 1 deletion packages/dynamite/dynamite/example/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ targets:
builders:
built_value_generator|built_value:
enabled: true
dynamite:
dynamite|openapi:
options:
pageWidth: 120
analyzer_ignores:
Expand Down
4 changes: 0 additions & 4 deletions packages/dynamite/dynamite/lib/builder.dart

This file was deleted.

8 changes: 7 additions & 1 deletion packages/dynamite/dynamite/lib/dynamite.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,10 @@
/// details, and `build.yaml` for how these builders are configured by default.
library dynamite;

export 'src/openapi_builder.dart';
import 'package:build/build.dart' show Builder, BuilderOptions;
import 'package:dynamite/src/json_schema_builder.dart' show JsonSchemaBuilder;
import 'package:dynamite/src/openapi_builder.dart' show OpenAPIBuilder;

Builder openAPIBuilder(BuilderOptions options) => OpenAPIBuilder(options);

Builder jsonSchemaBuilder(BuilderOptions options) => JsonSchemaBuilder(options);
50 changes: 35 additions & 15 deletions packages/dynamite/dynamite/lib/src/builder/generate_schemas.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import 'package:dynamite/src/builder/resolve_type.dart';
import 'package:dynamite/src/builder/state.dart';
import 'package:dynamite/src/helpers/dart_helpers.dart';
import 'package:dynamite/src/helpers/docs.dart';
import 'package:dynamite/src/models/json_schema.dart' as json_schema;
import 'package:dynamite/src/models/openapi.dart' as openapi;
import 'package:dynamite/src/models/type_result.dart';

Expand All @@ -13,27 +14,46 @@ Iterable<Spec> generateSchemas(
if (spec.components?.schemas != null) {
for (final schema in spec.components!.schemas!.entries) {
final identifier = toDartName(schema.key, className: true);
final result = resolveType(
state,
identifier,

final result = generateSchema(
schema.value,
identifier,
state,
);

// TypeDefs should only be generated for top level schemas.
if (result is TypeResultBase || result.isTypeDef) {
yield TypeDef((b) {
if (schema.value.deprecated) {
b.annotations.add(refer('Deprecated').call([refer("''")]));
}

b
..docs.addAll(escapeDescription(schema.value.formattedDescription()))
..name = identifier
..definition = refer(result.dartType.name);
});
if (result != null) {
yield result;
}
}
}

yield* state.output;
}

Spec? generateSchema(
json_schema.JsonSchema schema,
String identifier,
State state,
) {
final result = resolveType(
state,
identifier,
schema,
);

// TypeDefs should only be generated for top level schemas.
if (result is TypeResultBase || result.isTypeDef) {
return TypeDef((b) {
if (schema.deprecated) {
b.annotations.add(refer('Deprecated').call([refer("''")]));
}

b
..docs.addAll(escapeDescription(schema.formattedDescription()))
..name = identifier
..definition = refer(result.dartType.name);
});
}

return null;
}
18 changes: 14 additions & 4 deletions packages/dynamite/dynamite/lib/src/helpers/version_checker.dart
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
import 'package:build/build.dart' hide log;
import 'package:dynamite/src/helpers/logger.dart';
import 'package:dynamite/src/models/dynamite_config/builder_type.dart';
import 'package:pub_semver/pub_semver.dart';
import 'package:pubspec_parse/pubspec_parse.dart';

// Also update the README.md if you change this.
final dependencies = {
final openapiDependencies = {
...jsonSchemaDependencies,
'http': Version.parse('1.2.0'),
'uri': Version.parse('1.0.0'),
};

final jsonSchemaDependencies = {
'built_collection': Version.parse('5.0.0'),
'built_value': Version.parse('8.9.0'),
'collection': Version.parse('1.0.0'),
'dynamite_runtime': Version.parse('0.3.0'),
'http': Version.parse('1.2.0'),
'meta': Version.parse('1.0.0'),
'uri': Version.parse('1.0.0'),
};

// Also update the README.md if you change this.
Expand All @@ -20,9 +25,14 @@ final devDependencies = {
};

/// Checks whether the correct version of the dependencies are present in the pubspec.yaml file.
Future<bool> helperVersionCheck(BuildStep buildStep) async {
Future<bool> helperVersionCheck(BuildStep buildStep, DynamiteBuilder builderType) async {
final pubspecAsset = AssetId(buildStep.inputId.package, 'pubspec.yaml');

final dependencies = switch (builderType) {
DynamiteBuilder.openapi => openapiDependencies,
DynamiteBuilder.jsonSchema => jsonSchemaDependencies,
};

if (!await buildStep.canRead(pubspecAsset)) {
log.severe('Failed to read the pubspec.yaml file. Version constraints of required packages can not be validated.');
return true;
Expand Down
143 changes: 143 additions & 0 deletions packages/dynamite/dynamite/lib/src/json_schema_builder.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
import 'dart:async';
import 'dart:convert';

import 'package:build/build.dart' hide log;
import 'package:code_builder/code_builder.dart';
import 'package:dart_style/dart_style.dart';
import 'package:dynamite/src/builder/generate_ofs.dart';
import 'package:dynamite/src/builder/generate_schemas.dart';
import 'package:dynamite/src/builder/serializer.dart';
import 'package:dynamite/src/builder/state.dart';
import 'package:dynamite/src/helpers/dart_helpers.dart';
import 'package:dynamite/src/helpers/logger.dart';
import 'package:dynamite/src/helpers/version_checker.dart';
import 'package:dynamite/src/models/config.dart';
import 'package:dynamite/src/models/dynamite_config/builder_type.dart';
import 'package:dynamite/src/models/json_schema.dart' as json_schema;
import 'package:path/path.dart' as p;

class JsonSchemaBuilder implements Builder {
JsonSchemaBuilder(
BuilderOptions options,
) : buildConfig = DynamiteConfig.fromJson(options.config);

@override
final buildExtensions = const {
'.json_schema.json': ['.json_schema.dart'],
};

/// The configuration for this builder.
final DynamiteConfig buildConfig;

/// The dynamite builder type.
static const DynamiteBuilder builderType = DynamiteBuilder.jsonSchema;

@override
Future<void> build(BuildStep buildStep) async {
log.severe('''
JsonSchema generation for dynamite is still experimental.
Features might be broken or limited.
Please provide feedback at: https://github.com/nextcloud/neon/issues
''');

final result = await helperVersionCheck(buildStep, builderType);
if (!result) {
return;
}

final inputId = buildStep.inputId;
final outputId = inputId.changeExtension('.dart');

try {
final json = jsonDecode(await buildStep.readAsString(inputId)) as Map<String, dynamic>;

final schema = json_schema.serializers.deserializeWith(
json_schema.JsonSchema.serializer,
json,
)!;

final config = buildConfig.configFor(inputId.path);
final state = State(config, json);

final output = Library((b) {
final analyzerIgnores = state.buildConfig.analyzerIgnores;
if (analyzerIgnores != null) {
b.ignoreForFile.addAll(analyzerIgnores);
}

b
..generatedByComment = 'JsonSchema models generated by Dynamite. Do not manually edit this file.'
..directives.addAll([
Directive.import('dart:convert'),
Directive.import('dart:typed_data'),
Directive.import('package:built_collection/built_collection.dart'),
Directive.import('package:built_value/built_value.dart'),
Directive.import('package:built_value/json_object.dart'),
Directive.import('package:built_value/serializer.dart'),
Directive.import('package:collection/collection.dart'),
Directive.import('package:dynamite_runtime/models.dart'),
]);

final identifier = schema.id?.pathSegments.last ?? inputId.pathSegments.last;

final result = generateSchema(
schema,
toDartName(identifier, className: true),
state,
);

if (result != null) {
b.body.add(result);
}

b
..body.addAll(state.output)
..body.addAll(buildOfsExtensions(state))
..body.addAll(buildSerializer(state));

// Part directive need to be generated after everything else so we know if we need it.
if (state.hasResolvedBuiltTypes) {
b.directives.add(
Directive.part(p.basename(outputId.changeExtension('.g.dart').path)),
);
}

if (state.buildConfig.experimental) {
b.annotations.add(
refer('experimental', 'package:meta/meta.dart'),
);
}
});

var outputString = output.accept(state.emitter).toString();

final coverageIgnores = state.buildConfig.coverageIgnores;
if (coverageIgnores != null) {
for (final ignore in coverageIgnores) {
final pattern = RegExp(ignore);

outputString = outputString.replaceAllMapped(
pattern,
(match) => ' // coverage:ignore-start\n${match.group(0)}\n // coverage:ignore-end',
);
}
}

final formatter = DartFormatter(pageWidth: buildConfig.pageWidth);
unawaited(
buildStep.writeAsString(
outputId,
formatter.format(outputString),
),
);
} catch (error, stackTrace) {
log.severe(
'Issue generating the library for $inputId',
error,
stackTrace,
);

rethrow;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/// The dynamite builder type.
enum DynamiteBuilder {
/// When the executing builder is a `OpenAPIBuilder`.
openapi,

/// When the executing builder is a `JsonSchemaBuilder`.
jsonSchema,
}
6 changes: 5 additions & 1 deletion packages/dynamite/dynamite/lib/src/openapi_builder.dart
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import 'package:dynamite/src/helpers/docs.dart';
import 'package:dynamite/src/helpers/logger.dart';
import 'package:dynamite/src/helpers/version_checker.dart';
import 'package:dynamite/src/models/config.dart';
import 'package:dynamite/src/models/dynamite_config/builder_type.dart';
import 'package:dynamite/src/models/openapi.dart' as openapi;
import 'package:path/path.dart' as p;
import 'package:version/version.dart';
Expand All @@ -38,9 +39,12 @@ class OpenAPIBuilder implements Builder {
/// The configuration for this builder.
final DynamiteConfig buildConfig;

/// The dynamite builder type.
static const DynamiteBuilder builderType = DynamiteBuilder.openapi;

@override
Future<void> build(BuildStep buildStep) async {
final result = await helperVersionCheck(buildStep);
final result = await helperVersionCheck(buildStep, builderType);
if (!result) {
return;
}
Expand Down
Loading

0 comments on commit a1705e0

Please sign in to comment.