Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifies DataModel and Language implementations for code generator #81

Merged
merged 5 commits into from
Feb 19, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions src/bin/ion/commands/beta/generate/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ impl CodeGenContext {
}
}

/// Represents an abstract data type type that can be used to determine which templates can be used for code generation.
/// A target-language-agnostic data type that determines which template(s) to use for code generation.
#[derive(Debug, Clone, PartialEq, Serialize)]
pub enum AbstractDataType {
Copy link
Contributor Author

@desaikd desaikd Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(🗺️ PR tour) Renames DataModel to AbstractDataType. Also modifies Sequence enum variant to store information about sequence data type. (#69)

// a struct with a scalar value (used for `type` constraint)
// A scalar value (e.g. a string or integer or user defined type)
Value,
// a struct with a sequence/collection value (used for `element` constraint)
// the parameter string represents the data type of the sequence
// A series of zero or more values whose type is described by the nested `String` (e.g. a list)
Sequence(String),
// A collection of field name/value pairs (e.g. a map)
Struct,
}

Expand All @@ -37,8 +37,8 @@ impl Display for AbstractDataType {
f,
"{}",
match self {
AbstractDataType::Value => "single value struct",
AbstractDataType::Sequence(_) => "sequence value struct",
AbstractDataType::Value => "scalar value struct",
AbstractDataType::Sequence(_) => "sequence",
AbstractDataType::Struct => "struct",
}
)
Expand Down
80 changes: 39 additions & 41 deletions src/bin/ion/commands/beta/generate/generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ impl<'a> CodeGenerator<'a, RustLanguage> {
Self {
output,
anonymous_type_counter: 0,
tera: Tera::new("src/bin/ion/commands/beta/generate/templates/**/*.templ").unwrap(),
tera: Tera::new("src/bin/ion/commands/beta/generate/templates/rust/*.templ").unwrap(),
phantom: PhantomData,
}
}
Expand All @@ -40,8 +40,9 @@ impl<'a> CodeGenerator<'a, RustLanguage> {
let mut modules = vec![];
let mut module_context = tera::Context::new();

// Register a tera filter that can be used to convert a string to upper camel case
// Register a tera filter that can be used to convert a string based on case
self.tera.register_filter("upper_camel", Self::upper_camel);
self.tera.register_filter("snake", Self::snake);
// Register a tera filter that can be used to see if a type is built in data type or not
self.tera
.register_filter("is_built_in_type", Self::is_built_in_type);
Expand All @@ -61,7 +62,7 @@ impl<'a> CodeGenerator<'a, RustLanguage> {
module_context: &mut Context,
) -> CodeGenResult<()> {
module_context.insert("modules", &modules);
let rendered = self.tera.render("rust/mod.templ", module_context)?;
let rendered = self.tera.render("mod.templ", module_context)?;
let mut file = File::create(self.output.join("mod.rs"))?;
file.write_all(rendered.as_bytes())?;
Ok(())
Expand All @@ -73,7 +74,7 @@ impl<'a> CodeGenerator<'a, JavaLanguage> {
Self {
output,
anonymous_type_counter: 0,
tera: Tera::new("src/bin/ion/commands/beta/generate/templates/**/*.templ").unwrap(),
tera: Tera::new("src/bin/ion/commands/beta/generate/templates/java/*.templ").unwrap(),
phantom: PhantomData,
}
}
Expand All @@ -83,7 +84,7 @@ impl<'a> CodeGenerator<'a, JavaLanguage> {
// this will be used for Rust to create mod.rs which lists all the generated modules
let mut modules = vec![];

// Register a tera filter that can be used to convert a string to upper camel case
// Register a tera filter that can be used to convert a string based on case
self.tera.register_filter("upper_camel", Self::upper_camel);

for isl_type in schema.types() {
Expand Down Expand Up @@ -114,6 +115,25 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
))
}

/// Represents a [tera] filter that converts given tera string value to [snake case].
/// Returns error if the given value is not a string.
///
/// For more information: <https://docs.rs/tera/1.19.0/tera/struct.Tera.html#method.register_filter>
///
/// [tera]: <https://docs.rs/tera/latest/tera/>
/// [upper camel case]: <https://docs.rs/convert_case/latest/convert_case/enum.Case.html#variant.Snake>
pub fn snake(
value: &tera::Value,
_map: &HashMap<String, tera::Value>,
) -> Result<tera::Value, tera::Error> {
Ok(tera::Value::String(
value
.as_str()
.ok_or(tera::Error::msg("Required string for this filter"))?
.to_case(Case::Snake),
))
}

/// Represents a [tera] filter that return true if the value is a built in type, otherwise returns false.
///
/// For more information: <https://docs.rs/tera/1.19.0/tera/struct.Tera.html#method.register_filter>
Expand All @@ -124,9 +144,9 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
_map: &HashMap<String, tera::Value>,
) -> Result<tera::Value, tera::Error> {
Ok(tera::Value::Bool(L::is_built_in_type(
value
.as_str()
.ok_or(tera::Error::msg("Required string for this filter"))?,
value.as_str().ok_or(tera::Error::msg(
"`is_built_in_type` called with non-String Value",
))?,
)))
}

Expand All @@ -135,7 +155,7 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
modules: &mut Vec<String>,
isl_type: &IslType,
) -> CodeGenResult<()> {
let abstract_data_type_name = isl_type
let isl_type_name = isl_type
.name()
.clone()
.unwrap_or_else(|| format!("AnonymousType{}", self.anonymous_type_counter));
Expand All @@ -145,11 +165,8 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
let mut imports: Vec<Import> = vec![];
let mut code_gen_context = CodeGenContext::new();

// Set the target kind name of the abstract data type (i.e. enum/class)
context.insert(
"target_kind_name",
&abstract_data_type_name.to_case(Case::UpperCamel),
);
// Set the ISL type name for the generated abstract data type
context.insert("target_kind_name", &isl_type_name.to_case(Case::UpperCamel));

let constraints = isl_type.constraints();
for constraint in constraints {
Expand All @@ -166,29 +183,16 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
context.insert("imports", &imports);

// add fields for template
if code_gen_context.abstract_data_type == Some(AbstractDataType::Struct)
|| code_gen_context.abstract_data_type == Some(AbstractDataType::Value)
|| matches!(
code_gen_context.abstract_data_type,
Some(AbstractDataType::Sequence(_))
)
{
if let Some(abstract_data_type) = &code_gen_context.abstract_data_type {
context.insert("fields", &tera_fields);
if let Some(abstract_data_type) = &code_gen_context.abstract_data_type {
context.insert("abstract_data_type", abstract_data_type);
} else {
return invalid_abstract_data_type_error(
context.insert("abstract_data_type", abstract_data_type);
} else {
return invalid_abstract_data_type_error(
"Can not determine abstract data type, constraints are mapping not mapping to an abstract data type.",
);
}
}

self.render_generated_code(
modules,
&abstract_data_type_name,
&mut context,
&mut code_gen_context,
)
self.render_generated_code(modules, &isl_type_name, &mut context, &mut code_gen_context)
}

fn render_generated_code(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(🗺️ PR tour) This method is just extracted from previous generate method for better modularization of the code.

Expand All @@ -205,11 +209,7 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
let rendered = self
.tera
.render(
&format!(
"{}/{}.templ",
L::string_value(),
L::template_as_string(template)
),
&format!("{}.templ", L::template_as_string(template)),
context,
)
.unwrap();
Expand All @@ -233,8 +233,7 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
IslTypeRef::Named(name, _) => {
if !L::is_built_in_type(name) {
imports.push(Import {
module_name: name.to_case(Case::Snake),
type_name: name.to_case(Case::UpperCamel),
name: name.to_string(),
});
}
let schema_type: IonSchemaType = name.into();
Expand All @@ -248,8 +247,7 @@ impl<'a, L: Language> CodeGenerator<'a, L> {
self.generate_abstract_data_type(modules, type_def)?;
let name = format!("AnonymousType{}", self.anonymous_type_counter);
imports.push(Import {
module_name: name.to_case(Case::Snake),
type_name: name.to_case(Case::UpperCamel),
name: name.to_string(),
});
name
}
Expand Down
2 changes: 1 addition & 1 deletion src/bin/ion/commands/beta/generate/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ impl IonCliCommand for GenerateCommand {
"java" => CodeGenerator::<JavaLanguage>::new(output).generate(schema)?,
"rust" => CodeGenerator::<RustLanguage>::new(output).generate(schema)?,
_ => bail!(
"Unsupported programming language: {}, this tool only supports Java and Rust code generation.",
"Programming language '{}' is not yet supported. Currently supported targets: 'java', 'rust'",
language
)
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% if import %}
import ion_data_model.{{ import_type }};
import ion_data_model.{{ import.name | upper_camel }};
{% endif %}
public final class {{ target_kind_name }} {
{% for field in fields -%}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use ion_rs::{IonResult, IonReader, Reader, StreamItem};
{% for import in imports %}
use crate::ion_data_model::{{ import.module_name }}::{{ import.type_name }};
use crate::ion_data_model::{{ import.name | snake }}::{{ import.name | upper_camel }};
{% endfor %}

#[derive(Debug, Clone, Default)]
Expand Down
30 changes: 15 additions & 15 deletions src/bin/ion/commands/beta/generate/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,27 @@ pub struct Field {
pub(crate) value: String,
}

/// Represents an import statement in a module file.
/// Represents an import in a generated code file.
/// This will be used by template engine to fill import statements of a type definition.
#[derive(Serialize)]
pub struct Import {
pub(crate) module_name: String,
pub(crate) type_name: String,
pub(crate) name: String,
}

pub trait Language {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(🗺️ PR tour) Adds a trait implementation of Language instead of an enum (#71). This trait will be implemented for different programming languages and can be used as a generic on CodeGenerator to trigger specific behavior based on programming language.

/// Provides a file extension based on programming language
fn file_extension() -> String;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like most (all?) of these -> String methods could be -> &str.


/// Returns string representation of programming language
fn string_value() -> String;
/// Returns string representation of programming language name
fn name() -> String;

/// Provides file name based on programming language standards
/// Provides generated code's file name for given `name` based on programming language standards
/// e.g.
/// In Rust, this will return a string casing `name` to [Case::Snake].
/// In Java, this will return a string casing `name` to [Case::UpperCamel]
fn file_name(name: &str) -> String;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the input name? An ISL type name? We should spell it out in the doc comment.

Copy link
Contributor Author

@desaikd desaikd Feb 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method basically converts the given name into a file name for generated code based on programming language.

Below is a possible comment modification:

/// Provides generated code's file name for given `name` based on programming language standards
/// e.g.
///     In Rust, this will return a string casing `name` to [Case::Snake].
///     In Java, this will return a string casing `name` to  [Case::UpperCamel]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the input string is always, say, a target language type name, it would be good to communicate that expectation in the method name and/or the doc comment:

fn file_name_for_type(name: &str) -> String;


/// Maps the given ISL type name to a target type
/// Maps the given ISL type to a target type name
fn target_type(ion_schema_type: &IonSchemaType) -> String;

/// Provides given target type as sequence
Expand All @@ -45,13 +47,13 @@ pub trait Language {
/// Java field name case -> [Case::Camel]
fn field_name_case() -> Case;

/// Returns true if its a built in type otherwise returns false
/// Returns true if the type name specified is provided by the target language implementation
fn is_built_in_type(name: &str) -> bool;

/// Returns the template as string based on programming language
/// e.g.
/// Template<RustLanguage>::Struct -> "struct"
/// Template<JavaLanguage>::Class -> "class"
/// In Rust, Template::Struct -> "struct"
/// In Java, Template::Struct -> "class"
fn template_as_string(template: &Template) -> String;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a method on Template instead? (Template::name(&self) -> &str)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Note: The doc comment here is outdated as Template isn't really generic over Language)
Template is not actually generic over Language. I tried that approach but that leads to having two implementations of template_as_string for Template<RustLanguage> and Template<JavaLanguage> which means there is no Template<Language>.
Hence I have kept Language::template_as_string with all other language related config methods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about naming it template_name(...) to communicate what the resulting String represents?

}

Expand All @@ -62,7 +64,7 @@ impl Language for JavaLanguage {
"java".to_string()
}

fn string_value() -> String {
fn name() -> String {
"java".to_string()
}

Expand Down Expand Up @@ -115,7 +117,7 @@ impl Language for RustLanguage {
"rs".to_string()
}

fn string_value() -> String {
fn name() -> String {
"rust".to_string()
}

Expand Down Expand Up @@ -176,9 +178,7 @@ impl TryFrom<Option<&AbstractDataType>> for Template {

fn try_from(value: Option<&AbstractDataType>) -> Result<Self, Self::Error> {
match value {
Some(AbstractDataType::Value)
| Some(AbstractDataType::Sequence(_))
| Some(AbstractDataType::Struct) => Ok(Template::Struct),
Some(_) => Ok(Template::Struct),
None => invalid_abstract_data_type_error(
"Can not get a template without determining data model first.",
),
Expand Down