diff --git a/CHANGELOG.md b/CHANGELOG.md index b73982a9f..14c703554 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,11 @@ and this project does not currently adhere to a particular versioning scheme. ### Fixed - Fix variable binding in pattern matching when the irrefutable pattern optimization occurs. ([#618][gh-618]) +- Don't warn on unused generated definitions. ([#514][gh-514]) + +### Added + +- Add import system ([#544][gh-544]) ## [0.2.36] - 2024-07-04 @@ -379,3 +384,6 @@ and this project does not currently adhere to a particular versioning scheme. [gh-586]: https://github.com/HigherOrderCO/Bend/issues/586 [gh-596]: https://github.com/HigherOrderCO/Bend/issues/596 [gh-618]: https://github.com/HigherOrderCO/Bend/issues/618 +[gh-514]: https://github.com/HigherOrderCO/Bend/issues/514 +[gh-544]: https://github.com/HigherOrderCO/Bend/pull/544 +[Unreleased]: https://github.com/HigherOrderCO/Bend/compare/0.2.36...HEAD diff --git a/docs/imports.md b/docs/imports.md new file mode 100644 index 000000000..ada25cd6f --- /dev/null +++ b/docs/imports.md @@ -0,0 +1,181 @@ +# Import System + +## Case Sensitivity +All import paths are case-sensitive. Ensure that the case used in import statements matches exactly with the file and directory names. + +## Syntax +Imports can be declared two ways: + +```py +from path import name +# or +import path/name +``` + +## Project Structure +Let's assume we have a bend project with the following structure: + +``` +my_project/ +├── main.bend +├── utils/ +│ ├── helper.bend +│ │ └── def calc +│ └── math.bend +│ ├── def add +│ └── def subtract +``` + +## Importing Relative Paths +Paths starting with `./` or `../` are imported relative to the file. + +### Example: +```py +# if used inside `my_project/*.bend` +from ./utils import helper +# if used inside `my_project/*/*.bend` +import ../utils/helper +``` + +This will bind `calc` from `helper.bend` as `helper/calc`. + +## Importing Absolute Paths +Otherwise, paths imported are relative to the folder of the main file. + +### Example: +```py +from utils import math +# or +import utils/math +``` + +This will bind `add` and `subtract` from `math.bend` as `math/add` and `math/subtract`. + +## Importing Specific Top-Level Names +You can import specific top-level names from a file. + +### Example: +```py +from utils/helper import calc +from utils/math import (add, subtract) +# or +import (utils/helper/calc, utils/math/add, utils/math/subtract) +# or +import utils/helper/calc +import utils/math/add +import utils/math/subtract +``` + +This will bind the names `calc`, `add` and `subtract` from their respective files. + +## Importing All Names +You can import all top-level names from a file using the wildcard `*`. + +### Example: +```py +from utils/math import * +``` + +This will bind the names `add` and `subtract` from `math.bend`. + +## Importing All `.bend` Files from a Folder +You can import all `.bend` files from a folder using the wildcard `*`. + +### Example: +```py +from utils import * +``` + +This will bind the names from `helper.bend` and `math.bend`, as `helper/calc`, `math/add` and `math/subtract`. + +## Aliasing Imports +You can alias imports to a different name for convenience. + +### Importing a File with Alias +Import the `file` top-level name from `file.bend` aliased to `alias`, and all other names as `alias/name`. + +### Example: +```py +from utils import helper as utilsHelper +import utils/math as mathLib +``` + +This will bind the names from `helper.bend` and `math.bend`, as `utilsHelper/calc`, `mathLib/add` and `mathLib/subtract`. + +### Importing Specific Names with Aliases +You can import specific top-level names and alias them to different names. + +### Example: +```py +from utils/helper import calc as calcFunc +from utils/math import (add as addFunc, subtract as subFunc) +# or +import (utils/math/add as addFunc, utils/math/subtract as subFunc) +``` + +This will bind `calc`, `add` and `subtract` as `calcFunc`, `addFunc` and `subFunc` from their respective files. + +## Project Structure +Let's assume we have a bend project with the following structure: + +``` +my_project/ +├── main.bend +├── types/ +│ ├── List.bend +│ │ └── type List: Nil | (Cons ..) +│ └── List/ +│ ├── concat.bend +│ │ └── def concat +│ └── append.bend +│ └── def append +│ └── def helper +``` + +## Importing data types + +You can import a data type and its constructors by only importing its name. + +### Example: +```py +from types/List import List +# behaves the same as +from types/List import (List, List/Nil, List/Cons) +``` + +Importing only `List` from `List.bend` will import the type `List` and bind its constructors name `List/Nil` and `List/Cons`. + +## Importing files with a top level name equal to its name + +When a file and a top-level name in it share a name, for example, `List.bend`, `concat.bend` and `append.bend`, the bind of that import is simplified to the file name. + +### Example: +```py +from types/List import append +``` + +This will bind `append` and `append/helper` from `append.bend`. + +## Files and directories with the same name + +When files and directories share a name, both share the import namespace: + +```py +from types/List import (List, concat) +``` + +This will attempt to import from both the `List.bend` file and the `List` folder, resulting in the binds `List/Nil`, `List/Cons` and `concat`. + +```py +from types/List import * +``` + +This will import all the names from `List.bend`, then all the files inside the `List` folder, resulting in the binds `List/Nil`, `List/Cons`, `concat`, `append` and `append/helper`. + +In both cases, if a name is present as a top-level name on the file, and as a `.bend` file name inside the folder, it will result in an error. + +If you only want to import `List.bend` and not search the files in its folder, you can use the `import path` syntax: + +```py +import types/List +``` diff --git a/docs/syntax.md b/docs/syntax.md index e2c7dfa57..f95497d4a 100644 --- a/docs/syntax.md +++ b/docs/syntax.md @@ -1234,3 +1234,33 @@ impossible to write in normal Bend syntax. It will also ignore all term-level compiler passes and so can be useful for writing programs with exact behaviour that won't ever be changed or optimized by the compiler. + +# Import Syntax + +### Import Relative to the File +Paths starting with `./` or `../` are imported relative to the file. + +### Import Relative to the Main Folder +Paths that do not start with `./` or `../` are relative to the folder of the main file. + +## Syntax + +### Import Specific Names from a File, or Files from a Folder +```py +from path import name +from path import (name1, name2) +import (path/name1, path/name2) +``` + +### Import All Names from a File, or All Files from a Folder +```py +from path import * +``` + +### Aliasing Imports +```py +from path import name as alias +from path import (name1 as Alias1, name2 as Alias2) +import path as alias +import (path/name1 as Alias1, path/name2 as Alias2) +``` diff --git a/src/diagnostics.rs b/src/diagnostics.rs index c38294357..2b436e9c3 100644 --- a/src/diagnostics.rs +++ b/src/diagnostics.rs @@ -22,6 +22,8 @@ pub struct DiagnosticsConfig { pub unused_definition: Severity, pub repeated_bind: Severity, pub recursion_cycle: Severity, + pub missing_main: Severity, + pub import_shadow: Severity, } #[derive(Debug, Clone)] @@ -57,6 +59,8 @@ pub enum WarningType { UnusedDefinition, RepeatedBind, RecursionCycle, + MissingMain, + ImportShadow, } impl Diagnostics { @@ -235,6 +239,9 @@ impl DiagnosticsConfig { unused_definition: severity, repeated_bind: severity, recursion_cycle: severity, + import_shadow: severity, + // Should only be changed manually, as a missing main is always a error to hvm + missing_main: Severity::Error, verbose, } } @@ -247,6 +254,8 @@ impl DiagnosticsConfig { WarningType::IrrefutableMatch => self.irrefutable_match, WarningType::RedundantMatch => self.redundant_match, WarningType::UnreachableMatch => self.unreachable_match, + WarningType::MissingMain => self.missing_main, + WarningType::ImportShadow => self.import_shadow, } } } diff --git a/src/fun/builtins.rs b/src/fun/builtins.rs index 1ef955ec8..23aa87405 100644 --- a/src/fun/builtins.rs +++ b/src/fun/builtins.rs @@ -1,4 +1,7 @@ -use super::{parser::TermParser, Book, Name, Num, Pattern, Term}; +use super::{ + parser::{ParseBook, TermParser}, + Book, Name, Num, Pattern, Term, +}; use crate::maybe_grow; const BUILTINS: &str = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/src/fun/builtins.bend")); @@ -42,13 +45,15 @@ pub const BUILTIN_CTRS: &[&str] = pub const BUILTIN_TYPES: &[&str] = &[LIST, STRING, NAT, TREE, MAP, IO]; -impl Book { - pub fn builtins() -> Book { +impl ParseBook { + pub fn builtins() -> Self { TermParser::new(BUILTINS) - .parse_book(Book::default(), true) + .parse_book(Self::default(), true) .expect("Error parsing builtin file, this should not happen") } +} +impl Book { pub fn encode_builtins(&mut self) { for def in self.defs.values_mut() { for rule in def.rules.iter_mut() { diff --git a/src/fun/check/set_entrypoint.rs b/src/fun/check/set_entrypoint.rs index 017b2d9c4..219a32669 100644 --- a/src/fun/check/set_entrypoint.rs +++ b/src/fun/check/set_entrypoint.rs @@ -1,4 +1,5 @@ use crate::{ + diagnostics::WarningType, fun::{Book, Ctx, Definition, Name}, ENTRY_POINT, HVM1_ENTRY_POINT, }; @@ -43,7 +44,7 @@ impl Ctx<'_> { (None, None, None) => { let entrypoint = self.book.entrypoint.clone().unwrap_or(Name::new(ENTRY_POINT)); - self.info.add_book_error(EntryErr::NotFound(entrypoint)) + self.info.add_book_warning(EntryErr::NotFound(entrypoint), WarningType::MissingMain) } } diff --git a/src/fun/load_book.rs b/src/fun/load_book.rs index 9151b9fb1..4e94a1fb8 100644 --- a/src/fun/load_book.rs +++ b/src/fun/load_book.rs @@ -1,15 +1,41 @@ -use crate::fun::{self, parser::TermParser}; +use super::{ + parser::{ParseBook, TermParser}, + Book, Name, +}; +use crate::{ + diagnostics::{Diagnostics, DiagnosticsConfig}, + imports::PackageLoader, +}; use std::path::Path; // TODO: Refactor so that we don't mix the two syntaxes here. /// Reads a file and parses to a definition book. -pub fn load_file_to_book(path: &Path) -> Result { - let builtins = fun::Book::builtins(); +pub fn load_file_to_book( + path: &Path, + package_loader: impl PackageLoader, + diag: DiagnosticsConfig, +) -> Result { let code = std::fs::read_to_string(path).map_err(|e| e.to_string())?; - do_parse_book(&code, path, builtins) + load_to_book(path, &code, package_loader, diag) } -pub fn do_parse_book(code: &str, path: &Path, builtins: fun::Book) -> Result { - TermParser::new(code).parse_book(builtins, false).map_err(|e| format!("In {} :\n{}", path.display(), e)) +pub fn load_to_book( + origin: &Path, + code: &str, + package_loader: impl PackageLoader, + diag: DiagnosticsConfig, +) -> Result { + let builtins = ParseBook::builtins(); + let book = do_parse_book(code, origin, builtins)?; + book.load_imports(package_loader, diag) +} + +pub fn do_parse_book(code: &str, origin: &Path, mut book: ParseBook) -> Result { + book.source = Name::new(origin.to_string_lossy()); + TermParser::new(code).parse_book(book, false).map_err(|e| format!("In {} :\n{}", origin.display(), e)) +} + +pub fn do_parse_book_default(code: &str, origin: &Path) -> Result { + do_parse_book(code, origin, ParseBook::builtins())?.to_fun() } diff --git a/src/fun/mod.rs b/src/fun/mod.rs index 17b0becfa..04a14d1b8 100644 --- a/src/fun/mod.rs +++ b/src/fun/mod.rs @@ -1,11 +1,16 @@ use crate::{ diagnostics::{Diagnostics, DiagnosticsConfig}, + imports::Import, maybe_grow, multi_iterator, ENTRY_POINT, }; use indexmap::{IndexMap, IndexSet}; use interner::global::{GlobalPool, GlobalString}; use itertools::Itertools; -use std::{borrow::Cow, collections::HashMap, hash::Hash, ops::Deref}; +use std::{ + borrow::Cow, + hash::Hash, + ops::{Deref, Range}, +}; pub mod builtins; pub mod check; @@ -49,6 +54,9 @@ pub struct Book { /// A custom or default "main" entrypoint. pub entrypoint: Option, + + /// Imports declared in the program. + pub imports: Vec, } pub type Definitions = IndexMap; @@ -61,7 +69,18 @@ pub type Constructors = IndexMap; pub struct Definition { pub name: Name, pub rules: Vec, - pub builtin: bool, + pub source: Source, +} + +#[derive(Debug, Clone)] +pub enum Source { + Builtin, + /// Was generated by the compiler. + Generated, + /// Source code location from the current book. + Local(Range), + /// Imported from another package. + Imported, } /// An HVM native definition. @@ -69,7 +88,7 @@ pub struct Definition { pub struct HvmDefinition { pub name: Name, pub body: hvm::ast::Net, - pub builtin: bool, + pub source: Source, } /// A pattern matching rule of a definition. @@ -254,10 +273,10 @@ pub enum Tag { } /// A user defined datatype -#[derive(Debug, Clone, Default)] +#[derive(Debug, Clone)] pub struct Adt { pub ctrs: IndexMap>, - pub builtin: bool, + pub source: Source, } #[derive(Debug, Clone, Default)] @@ -753,6 +772,33 @@ impl Term { } } + /// Substitute the occurrences of a constructor name with the given name. + pub fn subst_ctrs(&mut self, from: &Name, to: &Name) { + maybe_grow(|| { + for child in self.children_mut() { + child.subst_ctrs(from, to); + } + }); + + match self { + Term::Fold { arms, .. } | Term::Mat { arms, .. } => { + for (arm, _, _) in arms { + if let Some(nam) = arm { + if nam == from { + *nam = to.clone(); + } + } + } + } + Term::Open { typ, .. } => { + if typ == from { + *typ = to.clone(); + } + } + _ => (), + } + } + /// Substitute the occurrence of an unscoped variable with the given term. pub fn subst_unscoped(&mut self, from: &Name, to: &Term) { maybe_grow(|| { @@ -772,8 +818,8 @@ impl Term { /// Collects all the free variables that a term has /// and the number of times each var is used - pub fn free_vars(&self) -> HashMap { - fn go_term(term: &Term, free_vars: &mut HashMap) { + pub fn free_vars(&self) -> IndexMap { + fn go_term(term: &Term, free_vars: &mut IndexMap) { maybe_grow(|| { if let Term::Var { nam } = term { *free_vars.entry(nam.clone()).or_default() += 1; @@ -784,7 +830,7 @@ impl Term { go_term(child, &mut new_scope); for nam in binds.flatten() { - new_scope.remove(nam); + new_scope.shift_remove(nam); } free_vars.extend(new_scope); @@ -976,6 +1022,19 @@ impl Rule { } impl Definition { + pub fn new(name: Name, rules: Vec, source: Source) -> Self { + Self { name, rules, source } + } + + pub fn new_gen(name: Name, rules: Vec, builtin: bool) -> Self { + let source = if builtin { Source::Builtin } else { Source::Generated }; + Self { name, rules, source } + } + + pub fn is_builtin(&self) -> bool { + self.source.is_builtin() + } + pub fn arity(&self) -> usize { self.rules[0].arity() } @@ -1010,7 +1069,9 @@ impl Name { } pub fn def_name_from_generated(&self) -> Name { - if let Some((nam, _)) = self.split_once("__") { + if let Some(nam) = self.strip_prefix("__") { + Name::new(nam) + } else if let Some((nam, _)) = self.split_once("__") { Name::new(nam) } else { self.clone() @@ -1059,6 +1120,16 @@ impl Book { } } +impl Source { + pub fn is_builtin(&self) -> bool { + matches!(self, Source::Builtin) + } + + pub fn is_local(&self) -> bool { + matches!(self, Source::Local(..)) + } +} + #[test] fn num_to_from_bits() { let a = [ diff --git a/src/fun/parser.rs b/src/fun/parser.rs index f9cd0d656..0792704e1 100644 --- a/src/fun/parser.rs +++ b/src/fun/parser.rs @@ -2,15 +2,61 @@ use std::ops::Range; use crate::{ fun::{ - display::DisplayFn, Adt, Book, CtrField, Definition, FanKind, HvmDefinition, MatchRule, Name, Num, Op, - Pattern, Rule, Tag, Term, STRINGS, + display::DisplayFn, Adt, Adts, Constructors, CtrField, FanKind, HvmDefinition, HvmDefinitions, MatchRule, + Name, Num, Op, Pattern, Rule, Source, Tag, Term, STRINGS, }, imp::{parser::PyParser, Enum, RepeatedNames, Variant}, + imports::{Import, ImportCtx, ImportType}, maybe_grow, }; use highlight_error::highlight_error; +use indexmap::IndexMap; +use itertools::Itertools; use TSPL::Parser; +type FunDefinition = super::Definition; +type ImpDefinition = crate::imp::Definition; + +/// Intermediate representation of a program. +#[derive(Debug, Clone, Default)] +pub struct ParseBook { + /// The `functional` function definitions. + pub fun_defs: IndexMap, + + /// The `imperative` function definitions. + pub imp_defs: IndexMap, + + /// HVM native function definitions. + pub hvm_defs: HvmDefinitions, + + /// The algebraic datatypes defined by the program + pub adts: Adts, + + /// To which type does each constructor belong to. + pub ctrs: Constructors, + + /// Imported packages to be loaded in the program + pub import_ctx: ImportCtx, + + /// Source of the book + pub source: Name, +} + +impl ParseBook { + pub fn contains_def(&self, name: &Name) -> bool { + self.fun_defs.contains_key(name) || self.imp_defs.contains_key(name) || self.hvm_defs.contains_key(name) + } + + pub fn contains_builtin_def(&self, name: &Name) -> Option { + self + .fun_defs + .get(name) + .map(|d| d.is_builtin()) + .or_else(|| self.imp_defs.get(name).map(|d| d.source.is_builtin())) + .or_else(|| self.hvm_defs.get(name).map(|d| d.source.is_builtin())) + } +} + // Bend grammar description: // ::= ( | )* // ::= "type" "=" ( | "(" ()* ")" )+ @@ -65,7 +111,7 @@ impl<'a> TermParser<'a> { /* AST parsing functions */ - pub fn parse_book(&mut self, default_book: Book, builtin: bool) -> ParseResult { + pub fn parse_book(&mut self, default_book: ParseBook, builtin: bool) -> ParseResult { let mut book = default_book; let mut indent = self.advance_newlines()?; let mut last_rule = None; @@ -116,8 +162,10 @@ impl<'a> TermParser<'a> { // Fun type definition } else { self.index = rewind_index; - let (nam, adt) = self.parse_datatype(builtin)?; + let (nam, ctrs) = self.parse_datatype()?; let end_idx = *self.index(); + let source = if builtin { Source::Builtin } else { Source::Local(ini_idx..end_idx) }; + let adt = Adt { ctrs, source }; self.add_fun_type(&mut book, nam, adt, ini_idx..end_idx)?; indent = self.advance_newlines()?; last_rule = None; @@ -135,10 +183,37 @@ impl<'a> TermParser<'a> { continue; } + // Import declaration + if self.try_parse_keyword("from") { + self.skip_trivia(); + let import = self.parse_from_import()?; + book.import_ctx.add_import(import); + indent = self.advance_newlines()?; + last_rule = None; + continue; + } + + if self.try_parse_keyword("import") { + self.skip_trivia(); + let imports = self.parse_import()?; + for imp in imports { + book.import_ctx.add_import(imp); + } + indent = self.advance_newlines()?; + last_rule = None; + continue; + } + // Fun function definition let ini_idx = *self.index(); let (name, rule) = self.parse_rule()?; let end_idx = *self.index(); + + if let Some(def) = book.imp_defs.get(&name) { + let msg = Self::redefinition_of_function_msg(def.source.is_builtin(), &name); + return self.with_ctx(Err(msg), ini_idx..end_idx); + } + self.add_fun_def(&name, rule, builtin, &last_rule, &mut book, ini_idx..end_idx)?; indent = self.advance_newlines()?; last_rule = Some(name); @@ -147,7 +222,7 @@ impl<'a> TermParser<'a> { Ok(book) } - fn parse_datatype(&mut self, builtin: bool) -> ParseResult<(Name, Adt)> { + fn parse_datatype(&mut self) -> ParseResult<(Name, IndexMap>)> { // type name = ctr (| ctr)* self.skip_trivia(); let name = self.labelled(|p| p.parse_top_level_name(), "datatype name")?; @@ -157,8 +232,7 @@ impl<'a> TermParser<'a> { ctrs.push(self.parse_datatype_ctr(&name)?); } let ctrs = ctrs.into_iter().collect(); - let adt = Adt { ctrs, builtin }; - Ok((name, adt)) + Ok((name, ctrs)) } fn parse_datatype_ctr(&mut self, typ_name: &Name) -> ParseResult<(Name, Vec)> { @@ -200,10 +274,58 @@ impl<'a> TermParser<'a> { let mut p = hvm::ast::CoreParser::new(&self.input[*self.index()..]); let body = p.parse_net()?; *self.index() = ini_idx + *p.index(); - let def = HvmDefinition { name: name.clone(), body, builtin }; + let end_idx = *self.index(); + let source = if builtin { Source::Builtin } else { Source::Local(ini_idx..end_idx) }; + let def = HvmDefinition { name: name.clone(), body, source }; Ok(def) } + fn parse_from_import(&mut self) -> Result { + // from path import package + // from path import (a, b) + // from path import * + let path = self.parse_restricted_name("Path")?; + self.consume("import")?; + + let relative = path.starts_with("./") | path.starts_with("../"); + + if self.try_consume("*") { + return Ok(Import::new(path, ImportType::Glob, relative)); + } + + if self.try_consume("(") { + let sub = self.list_like(|p| p.parse_name_maybe_alias("Name"), "", ")", ",", false, 1)?; + return Ok(Import::new(path, ImportType::List(sub), relative)); + } + + let (import, alias) = self.parse_name_maybe_alias("Import")?; + Ok(Import::new(path, ImportType::Single(import, alias), relative)) + } + + fn parse_import(&mut self) -> Result, String> { + // import path + // import (path/a, path/b) + + let new_import = |import: Name, alias: Option, relative: bool| -> Import { + let (path, import) = match import.rsplit_once('/') { + Some((start, end)) => (Name::new(start), Name::new(end)), + None => (Name::default(), import), + }; + + Import::new(path, ImportType::Single(import, alias), relative) + }; + + if self.try_consume("(") { + let list = self.list_like(|p| p.parse_import_name("Name"), "", ")", ",", false, 1)?; + let imports = list.into_iter().map(|(a, b, c)| new_import(a, b, c)).collect_vec(); + return Ok(imports); + } + + let (import, alias, relative) = self.parse_import_name("Import")?; + let import = new_import(import, alias, relative); + Ok(vec![import]) + } + fn parse_rule(&mut self) -> ParseResult<(Name, Rule)> { // (name pat*) = term // name pat* = term @@ -783,28 +905,32 @@ impl<'a> TermParser<'a> { rule: Rule, builtin: bool, last_rule: &Option, - book: &mut Book, + book: &mut ParseBook, span: Range, ) -> ParseResult<()> { - match (book.defs.get_mut(name), last_rule) { + match (book.fun_defs.get_mut(name), last_rule) { // Continuing with a new rule to the current definition (Some(def), Some(last_rule)) if last_rule == name => { def.rules.push(rule); + if let Source::Local(s) = &mut def.source { + s.end = span.end; + } } // Trying to add a new rule to a previous definition, coming from a different rule. (Some(def), Some(_)) => { - let msg = Self::redefinition_of_function_msg(def.builtin, name); + let msg = Self::redefinition_of_function_msg(def.is_builtin(), name); return self.with_ctx(Err(msg), span); } // Trying to add a new rule to a previous definition, coming from another kind of top-level. (Some(def), None) => { - let msg = Self::redefinition_of_function_msg(def.builtin, name); + let msg = Self::redefinition_of_function_msg(def.is_builtin(), name); return self.with_ctx(Err(msg), span); } // Adding the first rule of a new definition (None, _) => { - self.check_top_level_redefinition(name, book, span)?; - book.defs.insert(name.clone(), Definition { name: name.clone(), rules: vec![rule], builtin }); + self.check_top_level_redefinition(name, book, span.clone())?; + let source = if builtin { Source::Builtin } else { Source::Local(span) }; + book.fun_defs.insert(name.clone(), FunDefinition::new(name.clone(), vec![rule], source)); } } Ok(()) @@ -813,19 +939,18 @@ impl<'a> TermParser<'a> { fn add_imp_def( &mut self, mut def: crate::imp::Definition, - book: &mut Book, + book: &mut ParseBook, span: Range, builtin: bool, ) -> ParseResult<()> { - self.check_top_level_redefinition(&def.name, book, span)?; - def.order_kwargs(book)?; - def.gen_map_get(); - let def = def.to_fun(builtin)?; - book.defs.insert(def.name.clone(), def); + self.check_top_level_redefinition(&def.name, book, span.clone())?; + let source = if builtin { Source::Builtin } else { Source::Local(span) }; + def.source = source; + book.imp_defs.insert(def.name.clone(), def); Ok(()) } - fn add_hvm(&mut self, def: HvmDefinition, book: &mut Book, span: Range) -> ParseResult<()> { + fn add_hvm(&mut self, def: HvmDefinition, book: &mut ParseBook, span: Range) -> ParseResult<()> { self.check_top_level_redefinition(&def.name, book, span)?; book.hvm_defs.insert(def.name.clone(), def); Ok(()) @@ -834,12 +959,13 @@ impl<'a> TermParser<'a> { fn add_imp_type( &mut self, enum_: Enum, - book: &mut Book, + book: &mut ParseBook, span: Range, builtin: bool, ) -> ParseResult<()> { self.check_type_redefinition(&enum_.name, book, span.clone())?; - let mut adt = Adt { ctrs: Default::default(), builtin }; + let source = if builtin { Source::Builtin } else { Source::Local(span.clone()) }; + let mut adt = Adt { ctrs: Default::default(), source }; for variant in enum_.variants { self.check_top_level_redefinition(&enum_.name, book, span.clone())?; book.ctrs.insert(variant.name.clone(), enum_.name.clone()); @@ -849,12 +975,21 @@ impl<'a> TermParser<'a> { Ok(()) } - fn add_fun_type(&mut self, book: &mut Book, nam: Name, adt: Adt, span: Range) -> ParseResult<()> { + fn add_fun_type( + &mut self, + book: &mut ParseBook, + nam: Name, + adt: Adt, + span: Range, + ) -> ParseResult<()> { if book.adts.contains_key(&nam) { let msg = TermParser::redefinition_of_type_msg(&nam); return self.with_ctx(Err(msg), span); } else { for ctr in adt.ctrs.keys() { + if let Some(builtin) = book.contains_builtin_def(ctr) { + return Err(TermParser::redefinition_of_function_msg(builtin, ctr)); + } match book.ctrs.entry(ctr.clone()) { indexmap::map::Entry::Vacant(e) => _ = e.insert(nam.clone()), indexmap::map::Entry::Occupied(e) => { @@ -871,13 +1006,14 @@ impl<'a> TermParser<'a> { fn add_object( &mut self, obj: Variant, - book: &mut Book, + book: &mut ParseBook, span: Range, builtin: bool, ) -> ParseResult<()> { self.check_type_redefinition(&obj.name, book, span.clone())?; - self.check_top_level_redefinition(&obj.name, book, span)?; - let mut adt = Adt { ctrs: Default::default(), builtin }; + self.check_top_level_redefinition(&obj.name, book, span.clone())?; + let source = if builtin { Source::Builtin } else { Source::Local(span) }; + let mut adt = Adt { ctrs: Default::default(), source }; book.ctrs.insert(obj.name.clone(), obj.name.clone()); adt.ctrs.insert(obj.name.clone(), obj.fields); book.adts.insert(obj.name, adt); @@ -887,11 +1023,11 @@ impl<'a> TermParser<'a> { fn check_top_level_redefinition( &mut self, name: &Name, - book: &mut Book, + book: &mut ParseBook, span: Range, ) -> ParseResult<()> { - if let Some(def) = book.defs.get(name) { - let msg = Self::redefinition_of_function_msg(def.builtin, name); + if let Some(builtin) = book.contains_builtin_def(name) { + let msg = Self::redefinition_of_function_msg(builtin, name); return self.with_ctx(Err(msg), span); } if book.ctrs.contains_key(name) { @@ -905,7 +1041,12 @@ impl<'a> TermParser<'a> { Ok(()) } - fn check_type_redefinition(&mut self, name: &Name, book: &mut Book, span: Range) -> ParseResult<()> { + fn check_type_redefinition( + &mut self, + name: &Name, + book: &mut ParseBook, + span: Range, + ) -> ParseResult<()> { if book.adts.contains_key(name) { let msg = Self::redefinition_of_type_msg(name); return self.with_ctx(Err(msg), span); @@ -999,8 +1140,6 @@ impl Indent { } } -impl Book {} - impl<'a> ParserCommons<'a> for TermParser<'a> {} pub trait ParserCommons<'a>: Parser<'a> { @@ -1038,6 +1177,24 @@ pub trait ParserCommons<'a>: Parser<'a> { self.parse_restricted_name("Variable") } + fn parse_name_maybe_alias(&mut self, label: &str) -> ParseResult<(Name, Option)> { + let name = self.parse_restricted_name(label)?; + + if self.try_consume("as") { + self.skip_trivia(); + let alias = self.parse_restricted_name("Alias")?; + Ok((name, Some(alias))) + } else { + Ok((name, None)) + } + } + + fn parse_import_name(&mut self, label: &str) -> Result<(Name, Option, bool), String> { + let (import, alias) = self.parse_name_maybe_alias(label)?; + let relative = import.starts_with("./") | import.starts_with("../"); + Ok((import, alias, relative)) + } + /// Consumes exactly the text without skipping. fn consume_exactly(&mut self, text: &str) -> ParseResult<()> { if self.input().get(*self.index()..).unwrap_or_default().starts_with(text) { diff --git a/src/fun/term_to_net.rs b/src/fun/term_to_net.rs index 808b63317..4a07bc816 100644 --- a/src/fun/term_to_net.rs +++ b/src/fun/term_to_net.rs @@ -21,13 +21,17 @@ pub fn book_to_hvm(book: &Book, diags: &mut Diagnostics) -> Result<(hvm::ast::Bo let mut hvm_book = hvm::ast::Book { defs: Default::default() }; let mut labels = Labels::default(); - let main = book.entrypoint.as_ref().unwrap(); + let main = book.entrypoint.as_ref(); for def in book.defs.values() { for rule in def.rules.iter() { let net = term_to_hvm(&rule.body, &mut labels); - let name = if def.name == *main { book.hvm_entrypoint().to_string() } else { def.name.0.to_string() }; + let name = if main.is_some_and(|m| &def.name == m) { + book.hvm_entrypoint().to_string() + } else { + def.name.0.to_string() + }; match net { Ok(net) => { diff --git a/src/fun/transform/definition_merge.rs b/src/fun/transform/definition_merge.rs index 58f371e5e..a58befbea 100644 --- a/src/fun/transform/definition_merge.rs +++ b/src/fun/transform/definition_merge.rs @@ -35,13 +35,12 @@ impl Book { let new_name = Name::new(equal_defs.iter().join(MERGE_SEPARATOR)); // Builtin origin takes precedence - let builtin = equal_defs.iter().any(|nam| self.defs[nam].builtin); + let builtin = equal_defs.iter().any(|nam| self.defs[nam].is_builtin()); if equal_defs.len() > 1 { // Merging some defs // Add the merged def - let new_def = - Definition { name: new_name.clone(), rules: vec![Rule { pats: vec![], body: term }], builtin }; + let new_def = Definition::new_gen(new_name.clone(), vec![Rule { pats: vec![], body: term }], builtin); self.defs.insert(new_name.clone(), new_def); // Remove the old ones and write the map of old names to new ones. for name in equal_defs { diff --git a/src/fun/transform/definition_pruning.rs b/src/fun/transform/definition_pruning.rs index 9dcebc83c..0e9ba616b 100644 --- a/src/fun/transform/definition_pruning.rs +++ b/src/fun/transform/definition_pruning.rs @@ -1,6 +1,6 @@ use crate::{ diagnostics::WarningType, - fun::{Book, Ctx, Name, Term}, + fun::{Book, Ctx, Name, Source, Term}, maybe_grow, }; use hvm::ast::{Net, Tree}; @@ -33,7 +33,7 @@ impl Ctx<'_> { // Get the functions that are accessible from non-builtins. for def in self.book.defs.values() { - if !def.builtin && !(used.get(&def.name) == Some(&Used::Main)) { + if !def.is_builtin() && !(used.get(&def.name) == Some(&Used::Main)) { if self.book.ctrs.contains_key(&def.name) { used.insert(def.name.clone(), Used::Ctr); } else { @@ -43,7 +43,7 @@ impl Ctx<'_> { } } for def in self.book.hvm_defs.values() { - if !def.builtin && !(used.get(&def.name) == Some(&Used::Main)) { + if !def.source.is_builtin() && !(used.get(&def.name) == Some(&Used::Main)) { used.insert(def.name.clone(), Used::NonBuiltin); self.book.find_used_definitions_from_hvm_net(&def.body, Used::NonBuiltin, &mut used); } @@ -60,8 +60,11 @@ impl Ctx<'_> { } // Remove unused definitions. - let names = self.book.defs.keys().cloned().chain(self.book.hvm_defs.keys().cloned()).collect::>(); - for def in names { + let defs = self.book.defs.iter().map(|(nam, def)| (nam.clone(), def.source.clone())); + let hvm_defs = self.book.hvm_defs.iter().map(|(nam, def)| (nam.clone(), def.source.clone())); + let names = defs.chain(hvm_defs).collect::>(); + + for (def, src) in names { if let Some(use_) = used.get(&def) { match use_ { Used::Main => { @@ -72,7 +75,7 @@ impl Ctx<'_> { // Prune if `prune_all`, otherwise show a warning. if prune_all { rm_def(self.book, &def); - } else { + } else if !def.is_generated() && !matches!(src, Source::Generated) { self.info.add_rule_warning("Definition is unused.", WarningType::UnusedDefinition, def); } } diff --git a/src/fun/transform/desugar_bend.rs b/src/fun/transform/desugar_bend.rs index 1bf741225..fd9d9c723 100644 --- a/src/fun/transform/desugar_bend.rs +++ b/src/fun/transform/desugar_bend.rs @@ -1,6 +1,6 @@ use crate::{ diagnostics::Diagnostics, - fun::{Ctx, Definition, Name, Rule, Term}, + fun::{Ctx, Definition, Name, Rule, Source, Term}, maybe_grow, }; use indexmap::IndexMap; @@ -15,7 +15,7 @@ impl Ctx<'_> { for def in self.book.defs.values_mut() { let mut fresh = 0; for rule in def.rules.iter_mut() { - if let Err(err) = rule.body.desugar_bend(&def.name, &mut fresh, &mut new_defs, def.builtin) { + if let Err(err) = rule.body.desugar_bend(&def.name, &mut fresh, &mut new_defs, &def.source) { self.info.add_rule_error(err, def.name.clone()); break; } @@ -34,12 +34,12 @@ impl Term { def_name: &Name, fresh: &mut usize, new_defs: &mut IndexMap, - builtin: bool, + source: &Source, ) -> Result<(), String> { maybe_grow(|| { // Recursively encode bends in the children for child in self.children_mut() { - child.desugar_bend(def_name, fresh, new_defs, builtin)?; + child.desugar_bend(def_name, fresh, new_defs, source)?; } // Convert a bend into a new recursive function and call it. @@ -56,11 +56,11 @@ impl Term { // Gather the free variables // They will be implicitly captured by the new function let mut free_vars = step.free_vars(); - free_vars.remove(&Name::new(RECURSIVE_KW)); + free_vars.shift_remove(&Name::new(RECURSIVE_KW)); free_vars.extend(base.free_vars()); free_vars.extend(cond.free_vars()); for bnd in bnd.iter().flatten() { - free_vars.remove(bnd); + free_vars.shift_remove(bnd); } let free_vars = free_vars.into_keys().collect::>(); @@ -87,7 +87,7 @@ impl Term { let body = Term::rfold_lams(body, free_vars.iter().cloned().map(Some)); // Make a definition from the new function - let def = Definition { name: new_nam.clone(), rules: vec![Rule { pats: vec![], body }], builtin }; + let def = Definition::new(new_nam.clone(), vec![Rule { pats: vec![], body }], source.clone()); new_defs.insert(new_nam.clone(), def); // Call the new function in the original term. diff --git a/src/fun/transform/desugar_fold.rs b/src/fun/transform/desugar_fold.rs index f7eba09d3..0ab970835 100644 --- a/src/fun/transform/desugar_fold.rs +++ b/src/fun/transform/desugar_fold.rs @@ -2,7 +2,7 @@ use std::collections::HashSet; use crate::{ diagnostics::Diagnostics, - fun::{Adts, Constructors, Ctx, Definition, Name, Pattern, Rule, Term}, + fun::{Adts, Constructors, Ctx, Definition, Name, Pattern, Rule, Source, Term}, maybe_grow, }; @@ -40,7 +40,7 @@ impl Ctx<'_> { &mut new_defs, &self.book.ctrs, &self.book.adts, - def.builtin, + &def.source, ); if let Err(e) = res { self.info.add_rule_error(e, def.name.clone()); @@ -62,11 +62,11 @@ impl Term { new_defs: &mut Vec, ctrs: &Constructors, adts: &Adts, - builtin: bool, + source: &Source, ) -> Result<(), String> { maybe_grow(|| { for child in self.children_mut() { - child.desugar_fold(def_name, fresh, new_defs, ctrs, adts, builtin)?; + child.desugar_fold(def_name, fresh, new_defs, ctrs, adts, source)?; } if let Term::Fold { .. } = self { @@ -120,7 +120,8 @@ impl Term { let body = Term::rfold_lams(body, with_bnd.iter().cloned()); let body = Term::rfold_lams(body, free_vars.iter().map(|nam| Some(nam.clone()))); let body = Term::lam(Pattern::Var(Some(x_nam)), body); - let def = Definition { name: new_nam.clone(), rules: vec![Rule { pats: vec![], body }], builtin }; + + let def = Definition::new(new_nam.clone(), vec![Rule { pats: vec![], body }], source.clone()); new_defs.push(def); // Call the new function diff --git a/src/fun/transform/desugar_use.rs b/src/fun/transform/desugar_use.rs index b6443c3b0..0fdc2c5cf 100644 --- a/src/fun/transform/desugar_use.rs +++ b/src/fun/transform/desugar_use.rs @@ -21,6 +21,15 @@ impl Book { } } } + + /// Inline copies of the declared bind in `Fold`, `Mat` and `Open` inside `use` expressions. + pub fn desugar_ctr_use(&mut self) { + for def in self.defs.values_mut() { + for rule in def.rules.iter_mut() { + rule.body.desugar_ctr_use(); + } + } + } } impl Term { @@ -36,4 +45,18 @@ impl Term { *self = std::mem::take(nxt); } } + + pub fn desugar_ctr_use(&mut self) { + maybe_grow(|| { + for children in self.children_mut() { + children.desugar_ctr_use(); + } + }); + + if let Term::Use { nam: Some(nam), val, nxt } = self { + if let Term::Var { nam: val } = val.as_ref() { + nxt.subst_ctrs(nam, val); + } + } + } } diff --git a/src/fun/transform/encode_adts.rs b/src/fun/transform/encode_adts.rs index 35183ec48..d937c0c16 100644 --- a/src/fun/transform/encode_adts.rs +++ b/src/fun/transform/encode_adts.rs @@ -18,14 +18,14 @@ impl Book { AdtEncoding::NumScott => { let tag = make_tag(adt_name == ctr_name, ctr_name); let body = encode_ctr_num_scott(fields.iter().map(|f| &f.nam), &tag); - let tag_def = make_tag_def(ctr_idx, &tag, adt); + let tag_def = make_tag_def(ctr_idx, &tag, adt.source.is_builtin()); tags.push((tag, tag_def)); body } }; let rules = vec![Rule { pats: vec![], body }]; - let def = Definition { name: ctr_name.clone(), rules, builtin: adt.builtin }; + let def = Definition::new(ctr_name.clone(), rules, adt.source.clone()); defs.push((ctr_name.clone(), def)); } } @@ -65,7 +65,7 @@ fn encode_ctr_num_scott<'a>(ctr_args: impl DoubleEndedIterator Term::rfold_lams(term, ctr_args.cloned().map(Some)) } -fn make_tag_def(ctr_idx: usize, tag: &Name, adt: &crate::fun::Adt) -> Definition { +fn make_tag_def(ctr_idx: usize, tag: &Name, builtin: bool) -> Definition { let tag_rule = vec![Rule { pats: vec![], body: Term::Num { val: Num::U24(ctr_idx as u32) } }]; - Definition { name: tag.clone(), rules: tag_rule, builtin: adt.builtin } + Definition::new_gen(tag.clone(), tag_rule, builtin) } diff --git a/src/fun/transform/float_combinators.rs b/src/fun/transform/float_combinators.rs index 064e9835f..a19af4747 100644 --- a/src/fun/transform/float_combinators.rs +++ b/src/fun/transform/float_combinators.rs @@ -42,7 +42,7 @@ impl Book { } } - let builtin = def.builtin; + let builtin = def.is_builtin(); let body = &mut def.rule_mut().body; ctx.reset(); ctx.def_size = body.size(); @@ -117,7 +117,7 @@ impl Term { let extracted_term = std::mem::replace(self, comb_ref); let rules = vec![Rule { body: extracted_term, pats: Vec::new() }]; - let rule = Definition { name: comb_name.clone(), rules, builtin }; + let rule = Definition::new_gen(comb_name.clone(), rules, builtin); ctx.combinators.insert(comb_name, (is_safe, rule)); } } diff --git a/src/fun/transform/lift_local_defs.rs b/src/fun/transform/lift_local_defs.rs index a4d4cb817..c0fa3d888 100644 --- a/src/fun/transform/lift_local_defs.rs +++ b/src/fun/transform/lift_local_defs.rs @@ -41,7 +41,7 @@ impl Term { apply_closure(&mut rules, &fvs); - let new_def = Definition { name: local_name.clone(), rules, builtin: false }; + let new_def = Definition::new_gen(local_name.clone(), rules, false); defs.insert(local_name.clone(), new_def); } _ => { diff --git a/src/fun/transform/linearize_matches.rs b/src/fun/transform/linearize_matches.rs index 09fcb15e1..25f0ea0b8 100644 --- a/src/fun/transform/linearize_matches.rs +++ b/src/fun/transform/linearize_matches.rs @@ -412,7 +412,7 @@ pub fn lift_match_vars(match_term: &mut Term) -> &mut Term { for (binds, body) in arms { let mut arm_free_vars = body.free_vars(); for bind in binds { - arm_free_vars.remove(&bind); + arm_free_vars.shift_remove(&bind); } free_vars.push(arm_free_vars.into_keys().collect()); } diff --git a/src/imp/mod.rs b/src/imp/mod.rs index fe513f1bb..a276edfbe 100644 --- a/src/imp/mod.rs +++ b/src/imp/mod.rs @@ -3,7 +3,7 @@ mod order_kwargs; pub mod parser; pub mod to_fun; -use crate::fun::{CtrField, Name, Num, Op}; +use crate::fun::{CtrField, Name, Num, Op, Source}; use indexmap::{IndexMap, IndexSet}; use interner::global::GlobalString; @@ -214,6 +214,7 @@ pub struct Definition { pub name: Name, pub params: Vec, pub body: Stmt, + pub source: Source, } // "type" {name} ":" {variant}* diff --git a/src/imp/order_kwargs.rs b/src/imp/order_kwargs.rs index 93aa5192c..26716a71e 100644 --- a/src/imp/order_kwargs.rs +++ b/src/imp/order_kwargs.rs @@ -1,5 +1,5 @@ use crate::{ - fun::{Book, Name}, + fun::{parser::ParseBook, Name}, imp::{Definition, Expr, Stmt}, }; use indexmap::IndexMap; @@ -7,92 +7,99 @@ use indexmap::IndexMap; impl Definition { /// Traverses the program's definitions and adjusts the order of keyword arguments /// in call/constructor expressions to match the order specified in the function or constructor definition. - pub fn order_kwargs(&mut self, book: &Book) -> Result<(), String> { - self.body.order_kwargs(book).map_err(|e| format!("In function '{}':\n {}", self.name, e)) + pub fn order_kwargs(&mut self, book: &ParseBook) -> Result<(), String> { + let use_map = &mut IndexMap::new(); + self.body.order_kwargs(book, use_map).map_err(|e| format!("In function '{}':\n {}", self.name, e)) } } impl Stmt { - fn order_kwargs(&mut self, book: &Book) -> Result<(), String> { + fn order_kwargs(&mut self, book: &ParseBook, use_map: &mut IndexMap) -> Result<(), String> { match self { Stmt::LocalDef { def, nxt } => { def.order_kwargs(book)?; - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } Stmt::Assign { val, nxt, .. } => { - val.order_kwargs(book)?; + val.order_kwargs(book, use_map)?; if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Ask { val, nxt, .. } => { - val.order_kwargs(book)?; - nxt.order_kwargs(book)?; + val.order_kwargs(book, use_map)?; + nxt.order_kwargs(book, use_map)?; } Stmt::InPlace { val, nxt, .. } => { - val.order_kwargs(book)?; - nxt.order_kwargs(book)?; + val.order_kwargs(book, use_map)?; + nxt.order_kwargs(book, use_map)?; } Stmt::If { cond, then, otherwise, nxt } => { - cond.order_kwargs(book)?; - then.order_kwargs(book)?; - otherwise.order_kwargs(book)?; + cond.order_kwargs(book, use_map)?; + then.order_kwargs(book, use_map)?; + otherwise.order_kwargs(book, use_map)?; if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Match { arg, arms, nxt, .. } => { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; for arm in arms { - arm.rgt.order_kwargs(book)?; + arm.rgt.order_kwargs(book, use_map)?; } if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Switch { arg, arms, nxt, .. } => { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; for arm in arms { - arm.order_kwargs(book)?; + arm.order_kwargs(book, use_map)?; } if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Fold { arg, arms, nxt, .. } => { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; for arm in arms { - arm.rgt.order_kwargs(book)?; + arm.rgt.order_kwargs(book, use_map)?; } if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Bend { bnd: _, arg, cond, step, base, nxt } => { for arg in arg { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; } - cond.order_kwargs(book)?; - step.order_kwargs(book)?; - base.order_kwargs(book)?; + cond.order_kwargs(book, use_map)?; + step.order_kwargs(book, use_map)?; + base.order_kwargs(book, use_map)?; if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::With { typ: _, bod, nxt } => { - bod.order_kwargs(book)?; + bod.order_kwargs(book, use_map)?; if let Some(nxt) = nxt { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } } Stmt::Open { typ: _, var: _, nxt } => { - nxt.order_kwargs(book)?; + nxt.order_kwargs(book, use_map)?; } - Stmt::Use { nam: _, val: bod, nxt } => { - bod.order_kwargs(book)?; - nxt.order_kwargs(book)?; + Stmt::Use { nam, val: bod, nxt } => { + if let Expr::Var { nam: bod } = bod.as_ref() { + use_map.insert(nam.clone(), bod.clone()); + nxt.order_kwargs(book, use_map)?; + use_map.pop(); + } else { + bod.order_kwargs(book, use_map)?; + nxt.order_kwargs(book, use_map)?; + } } - Stmt::Return { term } => term.order_kwargs(book)?, + Stmt::Return { term } => term.order_kwargs(book, use_map)?, Stmt::Err => {} } Ok(()) @@ -100,13 +107,13 @@ impl Stmt { } impl Expr { - fn order_kwargs(&mut self, book: &Book) -> Result<(), String> { + fn order_kwargs(&mut self, book: &ParseBook, use_map: &mut IndexMap) -> Result<(), String> { match self { // Named arguments are only allowed when directly calling a named function. Expr::Call { fun, args, kwargs } => { if !kwargs.is_empty() { if let Expr::Var { nam } = fun.as_ref() { - if let Some(names) = get_args_def_or_ctr(nam, book) { + if let Some(names) = get_args_def_or_ctr(nam, book, use_map) { go_order_kwargs(&names, args, kwargs)?; } else { return Err(format!( @@ -121,54 +128,54 @@ impl Expr { ); } } - fun.order_kwargs(book)?; + fun.order_kwargs(book, use_map)?; for arg in args { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; } for (_, arg) in kwargs { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; } } - Expr::Lam { bod, .. } => bod.order_kwargs(book)?, + Expr::Lam { bod, .. } => bod.order_kwargs(book, use_map)?, Expr::Opr { lhs, rhs, .. } => { - lhs.order_kwargs(book)?; - rhs.order_kwargs(book)?; + lhs.order_kwargs(book, use_map)?; + rhs.order_kwargs(book, use_map)?; } Expr::Lst { els } | Expr::Tup { els } | Expr::Sup { els } => { for el in els { - el.order_kwargs(book)?; + el.order_kwargs(book, use_map)?; } } Expr::LstMap { term, iter, cond, .. } => { - term.order_kwargs(book)?; - iter.order_kwargs(book)?; + term.order_kwargs(book, use_map)?; + iter.order_kwargs(book, use_map)?; if let Some(cond) = cond { - cond.order_kwargs(book)?; + cond.order_kwargs(book, use_map)?; } } - Expr::Ctr { name, args, kwargs } => match get_args_def_or_ctr(name, book) { + Expr::Ctr { name, args, kwargs } => match get_args_def_or_ctr(name, book, use_map) { Some(names) => { go_order_kwargs(&names, args, kwargs)?; for arg in args { - arg.order_kwargs(book)?; + arg.order_kwargs(book, use_map)?; } } _ => return Err(format!("Constructor '{name}' not found.")), }, Expr::Map { entries } => { for entry in entries { - entry.1.order_kwargs(book)?; + entry.1.order_kwargs(book, use_map)?; } } Expr::MapGet { nam: _, key } => { - key.order_kwargs(book)?; + key.order_kwargs(book, use_map)?; } Expr::TreeNode { left, right } => { - left.order_kwargs(book)?; - right.order_kwargs(book)?; + left.order_kwargs(book, use_map)?; + right.order_kwargs(book, use_map)?; } Expr::TreeLeaf { val } => { - val.order_kwargs(book)?; + val.order_kwargs(book, use_map)?; } Expr::Era | Expr::Var { .. } | Expr::Chn { .. } | Expr::Num { .. } | Expr::Str { .. } => {} } @@ -201,11 +208,13 @@ fn go_order_kwargs( Ok(()) } -fn get_args_def_or_ctr(name: &Name, book: &Book) -> Option> { +fn get_args_def_or_ctr(name: &Name, book: &ParseBook, use_map: &IndexMap) -> Option> { + let name = use_map.get(name).unwrap_or(name); + #[allow(clippy::manual_map)] if let Some(adt_nam) = book.ctrs.get(name) { Some(book.adts[adt_nam].ctrs[name].iter().map(|f| f.nam.clone()).collect()) - } else if let Some(def) = book.defs.get(name) { + } else if let Some(def) = book.fun_defs.get(name) { Some(def.rules[0].pats.iter().flat_map(|p| p.binds().flatten().cloned()).collect()) } else { None diff --git a/src/imp/parser.rs b/src/imp/parser.rs index 120cd7108..c78f61e7b 100644 --- a/src/imp/parser.rs +++ b/src/imp/parser.rs @@ -1001,7 +1001,8 @@ impl<'a> PyParser<'a> { let (body, nxt_indent) = self.parse_statement(&mut indent)?; indent.exit_level(); - let def = Definition { name, params, body }; + // Temporary source, should be overwritten later + let def = Definition { name, params, body, source: crate::fun::Source::Generated }; Ok((def, nxt_indent)) } diff --git a/src/imp/to_fun.rs b/src/imp/to_fun.rs index f69df4632..0f14a04eb 100644 --- a/src/imp/to_fun.rs +++ b/src/imp/to_fun.rs @@ -2,11 +2,30 @@ use super::{AssignPattern, Definition, Expr, InPlaceOp, Stmt}; use crate::fun::{ self, builtins::{LCONS, LNIL}, - Name, + parser::ParseBook, + Book, Name, }; +impl ParseBook { + pub fn to_fun(mut self) -> Result { + for (name, mut def) in std::mem::take(&mut self.imp_defs) { + def.order_kwargs(&self)?; + def.gen_map_get(); + + if self.fun_defs.contains_key(&name) { + panic!("Def names collision should be checked at parse time") + } + + self.fun_defs.insert(name, def.to_fun()?); + } + + let ParseBook { fun_defs: defs, hvm_defs, adts, ctrs, import_ctx, .. } = self; + Ok(Book { defs, hvm_defs, adts, ctrs, entrypoint: None, imports: import_ctx.to_imports() }) + } +} + impl Definition { - pub fn to_fun(self, builtin: bool) -> Result { + pub fn to_fun(self) -> Result { let body = self.body.into_fun().map_err(|e| format!("In function '{}': {}", self.name, e))?; let body = match body { StmtToFun::Return(term) => term, @@ -18,7 +37,7 @@ impl Definition { let rule = fun::Rule { pats: self.params.into_iter().map(|param| fun::Pattern::Var(Some(param))).collect(), body }; - let def = fun::Definition { name: self.name, rules: vec![rule], builtin }; + let def = fun::Definition::new(self.name, vec![rule], self.source); Ok(def) } } @@ -371,7 +390,7 @@ impl Stmt { StmtToFun::Return(term) => (None, term), StmtToFun::Assign(pat, term) => (Some(pat), term), }; - let def = def.to_fun(false)?; + let def = def.to_fun()?; let term = fun::Term::Def { nam: def.name, rules: def.rules, nxt: Box::new(nxt) }; if let Some(pat) = nxt_pat { StmtToFun::Assign(pat, term) @@ -420,7 +439,7 @@ impl Expr { Expr::Ctr { name, args, kwargs } => { assert!(kwargs.is_empty()); let args = args.into_iter().map(Self::to_fun); - fun::Term::call(fun::Term::Ref { nam: name }, args) + fun::Term::call(fun::Term::Var { nam: name }, args) } Expr::LstMap { term, bind, iter, cond } => { const ITER_TAIL: &str = "%iter.tail"; diff --git a/src/imports/book.rs b/src/imports/book.rs new file mode 100644 index 000000000..767aca194 --- /dev/null +++ b/src/imports/book.rs @@ -0,0 +1,402 @@ +use super::{BindMap, ImportsMap, PackageLoader}; +use crate::{ + diagnostics::{Diagnostics, DiagnosticsConfig}, + fun::{parser::ParseBook, Adt, Book, Definition, HvmDefinition, Name, Rule, Source, Term}, + imp::{self, Expr, Stmt}, + imports::packages::Packages, +}; +use indexmap::{map::Entry, IndexMap}; +use itertools::Itertools; + +impl ParseBook { + /// Loads and applies imports recursively to a ParseBook, + /// transforming definitions and ADTs to a canonical name, + /// and adding `use` binds so that names are accessible by their alias. + /// + /// # Details + /// + /// The process involves: + /// + /// 1. Loading imports recursively using the provided `loader`. + /// 2. Transforming definitions and ADTs with naming transformations. + /// 3. Adding binds for aliases and old names in their respective definitions. + /// 4. Converting the ParseBook into its functional form. + /// 5. Perform any necessary post-processing. + pub fn load_imports( + self, + mut loader: impl PackageLoader, + diag_config: DiagnosticsConfig, + ) -> Result { + let diag = &mut Diagnostics::new(diag_config); + let pkgs = &mut Packages::new(self); + let mut book = pkgs.load_imports(&mut loader, diag)?; + + book.apply_imports(None, diag, pkgs)?; + eprint!("{}", diag); + + let mut book = book.to_fun()?; + + // Process terms that contains constructors names and can't be updated by `desugar_use`. + book.desugar_ctr_use(); + + Ok(book) + } + + /// Loads the imported books recursively into the importing book, + /// then apply imported names or aliases binds to its definitions. + fn apply_imports( + &mut self, + main_imports: Option<&ImportsMap>, + diag: &mut Diagnostics, + pkgs: &mut Packages, + ) -> Result<(), Diagnostics> { + self.load_packages(main_imports, diag, pkgs)?; + self.apply_import_binds(main_imports, pkgs); + Ok(()) + } + + /// Consumes the book imported packages, + /// applying the imports recursively of every nested book. + fn load_packages( + &mut self, + main_imports: Option<&ImportsMap>, + diag: &mut Diagnostics, + pkgs: &mut Packages, + ) -> Result<(), Diagnostics> { + diag.start_pass(); + + let sources = self.import_ctx.sources().into_iter().cloned().collect_vec(); + + for src in sources { + let Some(package) = pkgs.books.swap_remove(&src) else { continue }; + let mut package = package.into_inner(); + + // Can not be done outside the loop/function because of the borrow checker. + // Just serves to pass only the import map of the first call to `apply_imports_go`. + let main_imports = main_imports.unwrap_or(&self.import_ctx.map); + + package.apply_imports(Some(main_imports), diag, pkgs)?; + + // Rename ADTs and defs, applying binds from old names to new names + package.apply_adts(&src, main_imports); + package.apply_defs(&src, main_imports); + + let Book { defs, hvm_defs, adts, .. } = package.to_fun()?; + + // Add the ADTs to the importing book, + // saving the constructors names to be used when applying ADTs binds. + for (name, adt) in adts { + let adts = pkgs.loaded_adts.entry(src.clone()).or_default(); + adts.insert(name.clone(), adt.ctrs.keys().cloned().collect_vec()); + self.add_imported_adt(name, adt, diag); + } + + // The names on the indexmap are the original ones, so we ignore them + for def in defs.into_values() { + self.add_imported_def(def, diag); + } + + // The names on the indexmap are the original ones, so we ignore them + for def in hvm_defs.into_values() { + self.add_imported_hvm_def(def, diag); + } + } + + diag.fatal(()) + } + + /// Applies a chain of `use bind = src` to every local definition. + /// + /// Must be used after `load_packages` + fn apply_import_binds(&mut self, main_imports: Option<&ImportsMap>, pkgs: &Packages) { + // Can not be done outside the function because of the borrow checker. + // Just serves to pass only the import map of the first call to `apply_imports_go`. + let main_imports = main_imports.unwrap_or(&self.import_ctx.map); + + let mut local_imports = BindMap::new(); + + // Collect local imports binds, starting with `__` if not imported by the main book. + 'outer: for (bind, src) in self.import_ctx.map.binds.iter().rev() { + if self.contains_def(bind) | self.ctrs.contains_key(bind) | self.adts.contains_key(bind) { + // TODO: Here we should show warnings for shadowing of imported names by local def/ctr/adt + // It can be done, but when importing with `ImportType::Single` files in the same folder, + // it gives a false positive warning + continue; + } + + let nam = if main_imports.contains_source(src) { src.clone() } else { Name::new(format!("__{}", src)) }; + + // Checks if the bind is an loaded ADT name, + // If so, add the constructors binds as `bind/ctr` instead. + // As ADTs names are not used in the syntax, we don't bind their names. + for pkg in self.import_ctx.sources() { + if let Some(book) = pkgs.loaded_adts.get(pkg) { + if let Some(ctrs) = book.get(&nam) { + for ctr in ctrs.iter().rev() { + let full_ctr_name = ctr.split("__").nth(1).unwrap_or(ctr.as_ref()); + let ctr_name = full_ctr_name.strip_prefix(src.as_ref()).unwrap(); + let bind = Name::new(format!("{}{}", bind, ctr_name)); + local_imports.insert(bind, ctr.clone()); + } + continue 'outer; + } + } + } + + // Not a constructor, so just insert the bind. + local_imports.insert(bind.clone(), nam); + } + + for (_, def) in self.local_defs_mut() { + def.apply_binds(true, &local_imports); + } + } + + /// Applying the necessary naming transformations to the book ADTs, + /// adding `use ctr = ctr_src` chains to every local definition. + fn apply_adts(&mut self, src: &Name, main_imports: &ImportsMap) { + let adts = std::mem::take(&mut self.adts); + let mut new_adts = IndexMap::new(); + let mut ctrs_map = IndexMap::new(); + + // Rename the ADTs and constructors to their canonical name, + // starting with `__` if not imported by the main book. + for (mut name, mut adt) in adts { + if adt.source.is_local() { + adt.source = Source::Imported; + name = Name::new(format!("{}/{}", src, name)); + + let mangle_name = !main_imports.contains_source(&name); + let mut mangle_adt_name = mangle_name; + + for (ctr, f) in std::mem::take(&mut adt.ctrs) { + let mut ctr_name = Name::new(format!("{}/{}", src, ctr)); + + let mangle_ctr = mangle_name && !main_imports.contains_source(&ctr_name); + + if mangle_ctr { + mangle_adt_name = true; + ctr_name = Name::new(format!("__{}", ctr_name)); + } + + ctrs_map.insert(ctr, ctr_name.clone()); + adt.ctrs.insert(ctr_name, f); + } + + if mangle_adt_name { + name = Name::new(format!("__{}", name)); + } + } + + new_adts.insert(name.clone(), adt); + } + + // Applies the binds for the new constructor names for every definition. + // As ADTs names are not used in the syntax, we don't bind their new names. + for (_, def) in self.local_defs_mut() { + def.apply_binds(true, &ctrs_map); + } + + self.adts = new_adts; + } + + /// Apply the necessary naming transformations to the book definitions, + /// adding `use def = def_src` chains to every local definition. + fn apply_defs(&mut self, src: &Name, main_imports: &ImportsMap) { + let mut canonical_map: IndexMap<_, _> = IndexMap::new(); + + // Rename the definitions to their canonical name + // Starting with `__` if not imported by the main book. + for (_, def) in self.local_defs_mut() { + def.canonicalize_name(src, main_imports, &mut canonical_map); + } + + // Applies the binds for the new names for every definition + for (_, def) in self.local_defs_mut() { + def.apply_binds(false, &canonical_map); + *def.source_mut() = Source::Imported; + } + } +} + +/// Helper functions +impl ParseBook { + pub fn top_level_names(&self) -> impl Iterator { + let imp_defs = self.imp_defs.keys(); + let fun_defs = self.fun_defs.keys(); + let hvm_defs = self.hvm_defs.keys(); + let adts = self.adts.keys(); + let ctrs = self.ctrs.keys(); + + imp_defs.chain(fun_defs).chain(hvm_defs).chain(adts).chain(ctrs) + } + + fn add_imported_adt(&mut self, nam: Name, adt: Adt, diag: &mut Diagnostics) { + if self.adts.get(&nam).is_some() { + let err = format!("The imported datatype '{nam}' conflicts with the datatype '{nam}'."); + diag.add_book_error(err); + } else { + for ctr in adt.ctrs.keys() { + if self.contains_def(ctr) { + let err = format!("The imported constructor '{ctr}' conflicts with the definition '{ctr}'."); + diag.add_book_error(err); + } + match self.ctrs.entry(ctr.clone()) { + Entry::Vacant(e) => _ = e.insert(nam.clone()), + Entry::Occupied(e) => { + let ctr = e.key(); + let err = format!("The imported constructor '{ctr}' conflicts with the constructor '{ctr}'."); + diag.add_book_error(err); + } + } + } + self.adts.insert(nam, adt); + } + } + + fn add_imported_def(&mut self, def: Definition, diag: &mut Diagnostics) { + if !self.has_def_conflict(&def.name, diag) { + self.fun_defs.insert(def.name.clone(), def); + } + } + + fn add_imported_hvm_def(&mut self, def: HvmDefinition, diag: &mut Diagnostics) { + if !self.has_def_conflict(&def.name, diag) { + self.hvm_defs.insert(def.name.clone(), def); + } + } + + fn has_def_conflict(&mut self, name: &Name, diag: &mut Diagnostics) -> bool { + if self.contains_def(name) { + let err = format!("The imported definition '{name}' conflicts with the definition '{name}'."); + diag.add_book_error(err); + true + } else if self.ctrs.contains_key(name) { + let err = format!("The imported definition '{name}' conflicts with the constructor '{name}'."); + diag.add_book_error(err); + true + } else { + false + } + } + + fn local_defs_mut(&mut self) -> impl Iterator { + let fun = self.fun_defs.iter_mut().map(|(nam, def)| (nam, def as &mut dyn Def)); + let imp = self.imp_defs.iter_mut().map(|(nam, def)| (nam, def as &mut dyn Def)); + let hvm = self.hvm_defs.iter_mut().map(|(nam, def)| (nam, def as &mut dyn Def)); + fun.chain(imp).chain(hvm).filter(|(_, def)| def.source().is_local()) + } +} + +/// Common functions for the different definition types +trait Def { + fn canonicalize_name(&mut self, src: &Name, main_imports: &ImportsMap, binds: &mut BindMap) { + let def_name = self.name_mut(); + let mut new_name = Name::new(format!("{}/{}", src, def_name)); + + if !main_imports.contains_source(&new_name) { + new_name = Name::new(format!("__{}", new_name)); + } + + binds.insert(def_name.clone(), new_name.clone()); + *def_name = new_name; + } + + /// Should apply the binds to the definition, + /// and if there are possible constructor names on it, rename rule patterns. + fn apply_binds(&mut self, maybe_constructor: bool, binds: &BindMap); + + fn source(&self) -> &Source; + fn source_mut(&mut self) -> &mut Source; + fn name_mut(&mut self) -> &mut Name; +} + +impl Def for Definition { + fn apply_binds(&mut self, maybe_constructor: bool, binds: &BindMap) { + fn rename_ctr_patterns(rule: &mut Rule, binds: &BindMap) { + for pat in &mut rule.pats { + for bind in pat.binds_mut().flatten() { + if let Some(alias) = binds.get(bind) { + *bind = alias.clone(); + } + } + } + } + + for rule in &mut self.rules { + if maybe_constructor { + rename_ctr_patterns(rule, binds); + } + let bod = std::mem::take(&mut rule.body); + rule.body = bod.fold_uses(binds.iter().rev()); + } + } + + fn source(&self) -> &Source { + &self.source + } + + fn source_mut(&mut self) -> &mut Source { + &mut self.source + } + + fn name_mut(&mut self) -> &mut Name { + &mut self.name + } +} + +impl Def for imp::Definition { + fn apply_binds(&mut self, _maybe_constructor: bool, binds: &BindMap) { + let bod = std::mem::take(&mut self.body); + self.body = bod.fold_uses(binds.iter().rev()); + } + + fn source(&self) -> &Source { + &self.source + } + + fn source_mut(&mut self) -> &mut Source { + &mut self.source + } + + fn name_mut(&mut self) -> &mut Name { + &mut self.name + } +} + +impl Def for HvmDefinition { + /// Do nothing, can not apply binds to a HvmDefinition. + fn apply_binds(&mut self, _maybe_constructor: bool, _binds: &BindMap) {} + + fn source(&self) -> &Source { + &self.source + } + + fn source_mut(&mut self) -> &mut Source { + &mut self.source + } + + fn name_mut(&mut self) -> &mut Name { + &mut self.name + } +} + +impl Term { + fn fold_uses<'a>(self, map: impl Iterator) -> Self { + map.fold(self, |acc, (bind, nam)| Self::Use { + nam: Some(bind.clone()), + val: Box::new(Self::Var { nam: nam.clone() }), + nxt: Box::new(acc), + }) + } +} + +impl Stmt { + fn fold_uses<'a>(self, map: impl Iterator) -> Self { + map.fold(self, |acc, (bind, nam)| Self::Use { + nam: bind.clone(), + val: Box::new(Expr::Var { nam: nam.clone() }), + nxt: Box::new(acc), + }) + } +} diff --git a/src/imports/loader.rs b/src/imports/loader.rs new file mode 100644 index 000000000..636afd349 --- /dev/null +++ b/src/imports/loader.rs @@ -0,0 +1,209 @@ +use super::{BoundSource, Import, ImportType}; +use crate::fun::Name; +use indexmap::IndexMap; +use std::{ + collections::HashSet, + path::{Component, Path, PathBuf}, +}; + +pub type Sources = IndexMap; + +/// Trait to load packages from various sources. +pub trait PackageLoader { + /// Load a package specified by the `import` parameter. + /// + /// # Parameters + /// + /// - `import`: A mutable reference to an `Import` structure, which contains: + /// - `path`: The path to the package or directory to be imported. + /// - `imp_type`: The type of import, which can specify a single name, a list of names, or all names in a path. + /// - `relative`: A boolean indicating if the path is relative to the current directory. + /// - `src`: A `BoundSource` to be updated with the names of the located files. + /// + /// # Behavior + /// + /// The `load` method is responsible for locating and loading the requested package(s). + /// The loaded packages are returned as a `Sources` map, where the key is the package name and the value is its content. + /// Implementers must: + /// + /// - Track already loaded sources to avoid loading and returning them again. + /// - Update `import.src` with the names of the found packages, even if they are not included in the `Sources` map. + /// + /// The implementation should handle the following import types: + /// - **Single**: Load a specific file by its name. + /// - **List**: Load a list of specified files or names from a specific file. + /// - **Glob**: Load all files in a directory or all names from a specific file. + fn load(&mut self, import: &mut Import) -> Result; +} + +/// Default implementation of `PackageLoader` that loads packages from the local directory. +pub struct DefaultLoader { + local_path: PathBuf, + loaded: HashSet, + entrypoint: Name, +} + +impl DefaultLoader { + pub fn new(local_path: &Path) -> Self { + let entrypoint = Name::new(local_path.file_stem().unwrap().to_string_lossy()); + let local_path = local_path.parent().unwrap().to_path_buf(); + Self { local_path, loaded: HashSet::new(), entrypoint } + } + + fn read_file(&mut self, path: &Path, file_path: &str, src: &mut Sources) -> Result, String> { + let normalized = normalize_path(&PathBuf::from(file_path)); + let file_path = Name::new(normalized.to_string_lossy()); + + if self.entrypoint == file_path { + return Err("Can not import the entry point of the program.".to_string()); + }; + + if !self.is_loaded(&file_path) { + self.loaded.insert(file_path.clone()); + + let path = path.with_extension("bend"); + let Some(code) = std::fs::read_to_string(path).ok() else { return Ok(None) }; + src.insert(file_path.clone(), code); + } + + Ok(Some(file_path)) + } + + fn read_file_in_folder( + &mut self, + full_path: &Path, + folder: &str, + file_name: &str, + src: &mut Sources, + ) -> Result, String> { + let full_path = full_path.join(file_name); + + if folder.is_empty() { + self.read_file(&full_path, file_name, src) + } else { + let file_name = &format!("{}/{}", folder, file_name); + self.read_file(&full_path, file_name, src) + } + } + + fn read_path( + &mut self, + base_path: &Path, + path: &Name, + imp_type: &ImportType, + ) -> Result, String> { + let full_path = base_path.join(path.as_ref()); + let mut src = IndexMap::new(); + let (mut file, mut dir) = (None, None); + + if full_path.with_extension("bend").is_file() { + file = self.read_file(&full_path, path.as_ref(), &mut src)?; + } + + if full_path.is_dir() || path.is_empty() { + let mut names = IndexMap::new(); + + match imp_type { + ImportType::Single(file, _) => { + if let Some(name) = self.read_file_in_folder(&full_path, path, file, &mut src)? { + names.insert(file.clone(), name); + } + } + ImportType::List(list) => { + for (file, _) in list { + if let Some(name) = self.read_file_in_folder(&full_path, path, file, &mut src)? { + names.insert(file.clone(), name); + } + } + } + ImportType::Glob => { + for entry in full_path.read_dir().unwrap().flatten() { + let file = PathBuf::from(&entry.file_name()); + + if let Some("bend") = file.extension().and_then(|f| f.to_str()) { + let file = file.file_stem().unwrap().to_string_lossy(); + if let Some(name) = self.read_file_in_folder(&full_path, path, &file, &mut src)? { + names.insert(Name::new(file), name); + } + } + } + } + } + + if !names.is_empty() { + dir = Some(names); + } + } + + let src = match (file, dir) { + (Some(f), None) => Some((BoundSource::File(f), src)), + (None, Some(d)) => Some((BoundSource::Dir(d), src)), + (Some(f), Some(d)) => Some((BoundSource::Either(f, d), src)), + (None, None) => None, + }; + + Ok(src) + } + + fn is_loaded(&self, name: &Name) -> bool { + self.loaded.contains(name) + } +} + +pub const BEND_PATH: &[&str] = &[""]; + +impl PackageLoader for DefaultLoader { + fn load(&mut self, import: &mut Import) -> Result { + let mut sources = Sources::new(); + + let Import { path, imp_type, relative, src } = import; + + let folders = if *relative { + vec![self.local_path.clone()] + } else { + BEND_PATH.iter().map(|p| self.local_path.join(p)).collect() + }; + + for base in folders { + let Some((names, new_pkgs)) = self.read_path(&base, path, imp_type)? else { continue }; + + *src = names; + sources.extend(new_pkgs); + break; + } + + if let BoundSource::None = src { + return Err(format!("Failed to import '{}' from '{}'", imp_type, path).to_string()); + } + + Ok(sources) + } +} + +// Taken from 'cargo/util/paths.rs' +pub fn normalize_path(path: &Path) -> PathBuf { + let mut components = path.components().peekable(); + let mut ret = if let Some(c @ Component::Prefix(..)) = components.peek().cloned() { + components.next(); + PathBuf::from(c.as_os_str()) + } else { + PathBuf::new() + }; + + for component in components { + match component { + Component::Prefix(..) => unreachable!(), + Component::RootDir => { + ret.push(component.as_os_str()); + } + Component::CurDir => {} + Component::ParentDir => { + ret.pop(); + } + Component::Normal(c) => { + ret.push(c); + } + } + } + ret +} diff --git a/src/imports/mod.rs b/src/imports/mod.rs new file mode 100644 index 000000000..32def8ee2 --- /dev/null +++ b/src/imports/mod.rs @@ -0,0 +1,154 @@ +use crate::{ + diagnostics::{Diagnostics, WarningType}, + fun::Name, +}; +use indexmap::{IndexMap, IndexSet}; +use itertools::Itertools; +use std::fmt::Display; + +pub mod book; +pub mod loader; +pub mod packages; + +pub use loader::*; + +pub type BindMap = IndexMap; + +#[derive(Debug, Clone, Default)] +pub struct ImportCtx { + /// Imports declared in the program source. + imports: Vec, + + /// Map from bound names to source package. + map: ImportsMap, +} + +impl ImportCtx { + pub fn add_import(&mut self, import: Import) { + self.imports.push(import); + } + + pub fn to_imports(self) -> Vec { + self.imports + } + + pub fn sources(&self) -> Vec<&Name> { + let mut names = Vec::new(); + for imps in &self.imports { + match &imps.src { + BoundSource::None => {} + BoundSource::File(f) => names.push(f), + BoundSource::Dir(v) => names.extend(v.values()), + BoundSource::Either(_, _) => unreachable!("This should be resolved into `File` or `Dir` by now"), + } + } + names + } +} + +#[derive(Debug, Clone)] +pub struct Import { + pub path: Name, + pub imp_type: ImportType, + pub relative: bool, + pub src: BoundSource, +} + +impl Import { + pub fn new(path: Name, imp_type: ImportType, relative: bool) -> Self { + Self { path, imp_type, relative, src: BoundSource::None } + } +} + +#[derive(Debug, Clone)] +pub enum ImportType { + Single(Name, Option), + List(Vec<(Name, Option)>), + Glob, +} + +impl Display for ImportType { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + ImportType::Single(n, _) => write!(f, "{n}"), + ImportType::List(l) => write!(f, "({})", l.iter().map(|(n, _)| n).join(", ")), + ImportType::Glob => write!(f, "*"), + } + } +} + +#[derive(Debug, Clone)] +pub enum BoundSource { + None, + File(Name), + Dir(IndexMap), + /// If the bound source is ambiguous between a file or a directory + Either(Name, IndexMap), +} + +#[derive(Debug, Clone, Default)] +struct ImportsMap { + binds: BindMap, +} + +impl ImportsMap { + pub fn contains_source(&self, s: &Name) -> bool { + self.binds.values().contains(s) + } + + fn add_bind(&mut self, src: &str, bind: Name, diag: &mut Diagnostics) { + if let Some(old) = self.binds.get(&bind) { + let warn = format!("The import '{src}' shadows the imported name '{old}'"); + diag.add_book_warning(warn, WarningType::ImportShadow); + } + + self.binds.insert(bind, Name::new(src)); + } + + fn add_aliased_bind(&mut self, src: &Name, sub: &Name, alias: Option<&Name>, diag: &mut Diagnostics) { + let src = format!("{}/{}", src, sub); + let aliased = alias.unwrap_or(sub); + self.add_bind(&src, aliased.clone(), diag); + } + + fn add_binds(&mut self, names: &IndexSet, src: &Name, diag: &mut Diagnostics) { + for sub in names { + self.add_aliased_bind(src, sub, None, diag); + } + } + + /// Adds all names to the ImportMap in the form `alias/name`. + /// If one of the names is equal to the file name, adds as `alias` instead. + fn add_file_nested_binds( + &mut self, + src: &Name, + file: &Name, + alias: Option<&Name>, + names: IndexSet, + diag: &mut Diagnostics, + ) { + let aliased = alias.unwrap_or(file); + + self.add_nested_binds(src, aliased, names.iter().filter(|&n| n != file), diag); + + if names.contains(file) { + let src = format!("{}/{}", src, file); + self.add_bind(&src, aliased.clone(), diag); + } + } + + /// Adds all names to the ImportMap in the form `bind/name`. + fn add_nested_binds<'a>( + &mut self, + src: &Name, + bind: &Name, + names: impl Iterator, + diag: &mut Diagnostics, + ) { + for name in names { + let src = format!("{}/{}", src, name); + let bind = Name::new(format!("{bind}/{name}")); + self.add_bind(&src, bind, diag); + } + } +} diff --git a/src/imports/packages.rs b/src/imports/packages.rs new file mode 100644 index 000000000..b77085d56 --- /dev/null +++ b/src/imports/packages.rs @@ -0,0 +1,257 @@ +use super::{loader::PackageLoader, normalize_path, BoundSource, ImportCtx, ImportType, ImportsMap}; +use crate::{ + diagnostics::Diagnostics, + fun::{load_book::do_parse_book, parser::ParseBook, Name}, +}; +use indexmap::{IndexMap, IndexSet}; +use std::{cell::RefCell, collections::VecDeque, path::PathBuf}; + +#[derive(Default)] +pub struct Packages { + /// Map from source name to parsed book. + pub books: IndexMap>, + /// Already loaded ADTs information to be used when applying ADT binds. + pub loaded_adts: IndexMap>>, + /// Queue of books indexes that still needs to load its imports. + load_queue: VecDeque, +} + +impl Packages { + pub fn new(book: ParseBook) -> Self { + Self { + books: IndexMap::from([(book.source.clone(), book.into())]), + load_queue: VecDeque::new(), + loaded_adts: IndexMap::new(), + } + } + + /// Loads each import statement recursively into a Source -> ParseBook map. + /// Inserts into the ImportsMap of each book all the imported names. + pub fn load_imports( + &mut self, + loader: &mut impl PackageLoader, + diag: &mut Diagnostics, + ) -> Result { + diag.start_pass(); + + self.load_imports_go(0, None, loader)?; + + while let Some(idx) = self.load_queue.pop_front() { + let parent_dir = { + let book = self.books[idx].borrow(); + book.source.rsplit_once('/').map(|(s, _)| Name::new(s)) + }; + self.load_imports_go(idx, parent_dir, loader)?; + } + + for idx in 0..self.books.len() { + self.load_binds(idx, diag); + } + + let (_, book) = self.books.swap_remove_index(0).unwrap(); + + diag.fatal(book.into_inner()) + } + + fn load_imports_go( + &mut self, + idx: usize, + dir: Option, + loader: &mut impl PackageLoader, + ) -> Result<(), Diagnostics> { + let mut sources = IndexMap::new(); + + { + let mut book = self.books[idx].borrow_mut(); + let names = &mut book.import_ctx.imports; + + for import in names { + if import.relative { + if let Some(ref dir) = dir { + let path = format!("{}/{}", dir, import.path); + let normalized = normalize_path(&PathBuf::from(path)); + import.path = Name::new(normalized.to_string_lossy()); + } + } + + let loaded = loader.load(import)?; + sources.extend(loaded); + } + } + + for (psrc, code) in sources { + let module = do_parse_book(&code, &PathBuf::from(psrc.as_ref()), ParseBook::default())?; + self.load_queue.push_back(self.books.len()); + self.books.insert(psrc, module.into()); + } + + Ok(()) + } + + /// Maps the `ImportType` of each import to the top level names it relates, + /// checks if it is valid, resolves `BoundSource::Either`, and adds to the book ImportMap. + fn load_binds(&mut self, idx: usize, diag: &mut Diagnostics) { + let book = &mut self.books[idx].borrow_mut(); + let ImportCtx { imports, map } = &mut book.import_ctx; + + for import in imports { + match (&mut import.src, &import.imp_type) { + (BoundSource::Either(src, pkgs), ImportType::Single(nam, alias)) => { + if self.unique_top_level_names(src).contains(nam) { + let err = format!("Both file '{src}.bend' and folder '{src}' contains the import '{nam}'"); + diag.add_book_error(err); + continue; + } + + self.add_file_from_dir(pkgs, nam, alias, map, diag); + import.src = BoundSource::Dir(std::mem::take(pkgs)); + } + + (BoundSource::Either(src, pkgs), ImportType::List(names)) => { + for (name, alias) in names { + let added = self.add_file_from_dir(pkgs, name, alias, map, diag); + + if !added { + if !self.unique_top_level_names(src).contains(name) { + let err = format!("Package '{src}' does not contain the top level name '{name}'"); + diag.add_book_error(err); + continue; + } + + pkgs.insert(name.clone(), src.clone()); + self.add_aliased_bind(src, name, alias, map, diag); + } + } + + import.src = BoundSource::Dir(std::mem::take(pkgs)); + } + + (BoundSource::Either(src, pkgs), ImportType::Glob) => { + let names = self.unique_top_level_names(src); + let mut error = false; + + for nam in pkgs.keys() { + if names.contains(nam) { + let err = format!("Both file '{src}.bend' and folder '{src}' contains the import '{nam}'"); + diag.add_book_error(err); + error = true; + } + } + + if error { + continue; + } + + self.add_glob_from_dir(pkgs, map, diag); + + for sub in &names { + pkgs.insert(sub.clone(), src.clone()); + } + + map.add_binds(&names, src, diag); + + import.src = BoundSource::Dir(std::mem::take(pkgs)); + } + + (BoundSource::File(src), ImportType::Single(name, alias)) => { + if !self.unique_top_level_names(src).contains(name) { + let err = format!("Package '{src}' does not contain the top level name '{name}'"); + diag.add_book_error(err); + continue; + } + + self.add_aliased_bind(src, name, alias, map, diag); + } + + (BoundSource::File(src), ImportType::List(names)) => { + let src_names = self.unique_top_level_names(src); + let mut error = false; + + for (sub, _) in names { + if !src_names.contains(sub) { + let err = format!("Package '{src}' does not contain the top level name '{sub}'"); + diag.add_book_error(err); + error = true; + } + } + + if error { + continue; + } + + for (name, alias) in names { + self.add_aliased_bind(src, name, alias, map, diag); + } + } + + (BoundSource::File(src), ImportType::Glob) => { + let names = self.unique_top_level_names(src); + map.add_binds(&names, src, diag); + } + + (BoundSource::Dir(pkgs), ImportType::Single(nam, alias)) => { + self.add_file_from_dir(pkgs, nam, alias, map, diag); + } + + (BoundSource::Dir(pkgs), ImportType::List(names)) => { + for (nam, alias) in names { + self.add_file_from_dir(pkgs, nam, alias, map, diag); + } + } + + (BoundSource::Dir(pkgs), ImportType::Glob) => { + self.add_glob_from_dir(pkgs, map, diag); + } + + (BoundSource::None, _) => unreachable!(), + } + } + } + + fn add_aliased_bind( + &self, + src: &mut Name, + name: &Name, + alias: &Option, + map: &mut ImportsMap, + diag: &mut Diagnostics, + ) { + let alias = alias.as_ref(); + + if let Some(adt) = self.books.get(src).unwrap().borrow().adts.get(name) { + let names = adt.ctrs.iter().map(|(n, _)| n); + map.add_nested_binds(src, alias.unwrap_or(name), names, diag); + } + + map.add_aliased_bind(src, name, alias, diag); + } + + fn add_file_from_dir( + &self, + pkgs: &IndexMap, + nam: &Name, + alias: &Option, + map: &mut ImportsMap, + diag: &mut Diagnostics, + ) -> bool { + if let Some(src) = pkgs.get(nam) { + let names = self.unique_top_level_names(src); + map.add_file_nested_binds(src, nam, alias.as_ref(), names, diag); + true + } else { + false + } + } + + fn add_glob_from_dir(&self, pkgs: &IndexMap, map: &mut ImportsMap, diag: &mut Diagnostics) { + for (nam, src) in pkgs { + let names = self.unique_top_level_names(src); + map.add_file_nested_binds(src, nam, None, names, diag); + } + } + + fn unique_top_level_names(&self, src: &Name) -> IndexSet { + let bound_book = self.books.get(src).unwrap().borrow(); + bound_book.top_level_names().cloned().collect() + } +} diff --git a/src/lib.rs b/src/lib.rs index e83178d0f..25e1b27c9 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -17,10 +17,11 @@ pub mod diagnostics; pub mod fun; pub mod hvm; pub mod imp; +pub mod imports; pub mod net; mod utils; -pub use fun::load_book::load_file_to_book; +pub use fun::load_book::{load_file_to_book, load_to_book}; pub const ENTRY_POINT: &str = "main"; pub const HVM1_ENTRY_POINT: &str = "Main"; diff --git a/src/main.rs b/src/main.rs index a73c9a2e9..b6fdcd315 100644 --- a/src/main.rs +++ b/src/main.rs @@ -3,6 +3,7 @@ use bend::{ diagnostics::{Diagnostics, DiagnosticsConfig, Severity}, fun::{Book, Name}, hvm::hvm_book_show_pretty, + imports::DefaultLoader, load_file_to_book, run_book, AdtEncoding, CompileOpts, OptLevel, RunOpts, }; use clap::{Args, CommandFactory, Parser, Subcommand}; @@ -230,6 +231,8 @@ pub enum WarningArgs { UnusedDefinition, RepeatedBind, RecursionCycle, + ImportShadow, + MissingMain, } fn main() -> ExitCode { @@ -249,8 +252,9 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let arg_verbose = cli.verbose; let entrypoint = cli.entrypoint.take(); - let load_book = |path: &Path| -> Result { - let mut book = load_file_to_book(path)?; + let load_book = |path: &Path, diag: DiagnosticsConfig| -> Result { + let package_loader = DefaultLoader::new(path); + let mut book = load_file_to_book(path, package_loader, diag)?; book.entrypoint = entrypoint.map(Name::new); if arg_verbose { @@ -287,7 +291,7 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let diagnostics_cfg = set_warning_cfg_from_cli(DiagnosticsConfig::default(), warn_opts); let compile_opts = compile_opts_from_cli(&comp_opts); - let mut book = load_book(&path)?; + let mut book = load_book(&path, diagnostics_cfg)?; let diagnostics = check_book(&mut book, diagnostics_cfg, compile_opts)?; eprintln!("{}", diagnostics); } @@ -296,7 +300,7 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let diagnostics_cfg = set_warning_cfg_from_cli(DiagnosticsConfig::default(), warn_opts); let opts = compile_opts_from_cli(&comp_opts); - let mut book = load_book(&path)?; + let mut book = load_book(&path, diagnostics_cfg)?; let compile_res = compile_book(&mut book, opts, diagnostics_cfg, None)?; eprint!("{}", compile_res.diagnostics); @@ -317,7 +321,7 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let run_opts = RunOpts { linear_readback: linear, pretty, hvm_path: hvm_bin }; - let book = load_book(&path)?; + let book = load_book(&path, diagnostics_cfg)?; if let Some((term, stats, diags)) = run_book(book, run_opts, compile_opts, diagnostics_cfg, arguments, run_cmd)? { @@ -338,7 +342,7 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let diagnostics_cfg = set_warning_cfg_from_cli(DiagnosticsConfig::default(), warn_opts); let opts = compile_opts_from_cli(&comp_opts); - let mut book = load_book(&path)?; + let mut book = load_book(&path, diagnostics_cfg)?; let compile_res = compile_book(&mut book, opts, diagnostics_cfg, None)?; let out_path = ".out.hvm"; @@ -369,7 +373,7 @@ fn execute_cli_mode(mut cli: Cli) -> Result<(), Diagnostics> { let opts = compile_opts_from_cli(&comp_opts); - let mut book = load_book(&path)?; + let mut book = load_book(&path, diagnostics_cfg)?; let diagnostics = desugar_book(&mut book, opts, diagnostics_cfg, None)?; eprint!("{diagnostics}"); @@ -393,6 +397,7 @@ fn set_warning_cfg_from_cli(mut cfg: DiagnosticsConfig, warn_opts: CliWarnOpts) cfg.unused_definition = severity; cfg.repeated_bind = severity; cfg.recursion_cycle = severity; + cfg.import_shadow = severity; } WarningArgs::IrrefutableMatch => cfg.irrefutable_match = severity, WarningArgs::RedundantMatch => cfg.redundant_match = severity, @@ -400,6 +405,8 @@ fn set_warning_cfg_from_cli(mut cfg: DiagnosticsConfig, warn_opts: CliWarnOpts) WarningArgs::UnusedDefinition => cfg.unused_definition = severity, WarningArgs::RepeatedBind => cfg.repeated_bind = severity, WarningArgs::RecursionCycle => cfg.recursion_cycle = severity, + WarningArgs::ImportShadow => cfg.import_shadow = severity, + WarningArgs::MissingMain => cfg.missing_main = severity, // TODO: Should `WarningArgs::All` modify this as well? } } diff --git a/tests/golden_tests.rs b/tests/golden_tests.rs index eb18d17ed..d1334e916 100644 --- a/tests/golden_tests.rs +++ b/tests/golden_tests.rs @@ -1,8 +1,12 @@ use bend::{ compile_book, desugar_book, diagnostics::{Diagnostics, DiagnosticsConfig, Severity}, - fun::{load_book::do_parse_book, net_to_term::net_to_term, term_to_net::Labels, Book, Ctx, Name, Term}, + fun::{ + load_book::do_parse_book_default, net_to_term::net_to_term, term_to_net::Labels, Book, Ctx, Name, Term, + }, hvm::hvm_book_show_pretty, + imports::DefaultLoader, + load_to_book, net::hvm_to_net::hvm_to_net, run_book, AdtEncoding, CompileOpts, RunOpts, }; @@ -67,7 +71,7 @@ fn run_golden_test_dir_multiple(test_name: &str, run: &[&RunFn]) { test_name.rsplit_once(':').unwrap().1 )); - let walker = WalkDir::new(&root).sort_by_file_name().max_depth(2).into_iter().filter_entry(|e| { + let walker = WalkDir::new(&root).sort_by_file_name().max_depth(1).into_iter().filter_entry(|e| { let path = e.path(); path == root || path.is_dir() || (path.is_file() && path.extension().is_some_and(|x| x == "bend")) }); @@ -103,7 +107,7 @@ pub fn run_book_simple( #[test] fn compile_file() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default(); let diagnostics_cfg = DiagnosticsConfig { unused_definition: Severity::Allow, ..Default::default() }; @@ -115,7 +119,7 @@ fn compile_file() { #[test] fn compile_file_o_all() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let opts = CompileOpts::default().set_all(); let diagnostics_cfg = DiagnosticsConfig { recursion_cycle: Severity::Warning, @@ -131,7 +135,7 @@ fn compile_file_o_all() { #[test] fn compile_file_o_no_all() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default().set_no_all(); let diagnostics_cfg = DiagnosticsConfig::default(); let res = compile_book(&mut book, compile_opts, diagnostics_cfg, None)?; @@ -143,7 +147,7 @@ fn compile_file_o_no_all() { fn linear_readback() { run_golden_test_dir(function_name!(), &|code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path, Book::builtins())?; + let book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default().set_all(); let diagnostics_cfg = DiagnosticsConfig::default(); let (term, _, diags) = run_book_simple( @@ -164,7 +168,7 @@ fn run_file() { function_name!(), &[(&|code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path, Book::builtins())?; + let book = do_parse_book_default(code, path)?; let diagnostics_cfg = DiagnosticsConfig { unused_definition: Severity::Allow, ..DiagnosticsConfig::new(Severity::Error, true) @@ -184,13 +188,37 @@ fn run_file() { ) } +#[test] +fn import_system() { + run_golden_test_dir_multiple( + function_name!(), + &[(&|code, path| { + let _guard = RUN_MUTEX.lock().unwrap(); + let diagnostics_cfg = DiagnosticsConfig { + unused_definition: Severity::Allow, + ..DiagnosticsConfig::new(Severity::Error, true) + }; + + let book = load_to_book(path, code, DefaultLoader::new(path), diagnostics_cfg)?; + let run_opts = RunOpts::default(); + + let mut res = String::new(); + + let compile_opts = CompileOpts::default(); + let (term, _, diags) = run_book_simple(book, run_opts, compile_opts, diagnostics_cfg, None)?; + res.push_str(&format!("{diags}{term}\n\n")); + Ok(res) + })], + ) +} + #[test] #[ignore = "while lazy execution is not implemented for hvm32"] fn run_lazy() { run_golden_test_dir(function_name!(), &|_code, _path| { todo!() /* let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path)?; + let book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default_lazy(); let diagnostics_cfg = DiagnosticsConfig { recursion_cycle: Severity::Allow, @@ -225,7 +253,7 @@ fn simplify_matches() { irrefutable_match: Severity::Allow, ..DiagnosticsConfig::new(Severity::Error, true) }; - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let mut ctx = Ctx::new(&mut book, diagnostics_cfg); ctx.check_shared_names(); @@ -258,7 +286,7 @@ fn simplify_matches() { #[test] fn parse_file() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let mut ctx = Ctx::new(&mut book, Default::default()); ctx.set_entrypoint(); ctx.book.encode_adts(AdtEncoding::NumScott); @@ -276,7 +304,7 @@ fn encode_pattern_match() { let mut result = String::new(); for adt_encoding in [AdtEncoding::Scott, AdtEncoding::NumScott] { let diagnostics_cfg = DiagnosticsConfig::default(); - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let mut ctx = Ctx::new(&mut book, diagnostics_cfg); ctx.check_shared_names(); ctx.set_entrypoint(); @@ -317,7 +345,7 @@ fn desugar_file() { unused_definition: Severity::Allow, ..DiagnosticsConfig::new(Severity::Error, true) }; - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; desugar_book(&mut book, compile_opts, diagnostics_cfg, None)?; Ok(book.to_string()) }) @@ -330,7 +358,7 @@ fn hangs() { run_golden_test_dir(function_name!(), &move |code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path, Book::builtins())?; + let book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default().set_all(); let diagnostics_cfg = DiagnosticsConfig::new(Severity::Allow, false); @@ -352,7 +380,7 @@ fn hangs() { #[test] fn compile_entrypoint() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; book.entrypoint = Some(Name::new("foo")); let diagnostics_cfg = DiagnosticsConfig { ..DiagnosticsConfig::new(Severity::Error, true) }; let res = compile_book(&mut book, CompileOpts::default(), diagnostics_cfg, None)?; @@ -365,7 +393,7 @@ fn compile_entrypoint() { fn run_entrypoint() { run_golden_test_dir(function_name!(), &|code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; book.entrypoint = Some(Name::new("foo")); let compile_opts = CompileOpts::default().set_all(); let diagnostics_cfg = DiagnosticsConfig { ..DiagnosticsConfig::new(Severity::Error, true) }; @@ -400,7 +428,7 @@ fn mutual_recursion() { run_golden_test_dir(function_name!(), &|code, path| { let diagnostics_cfg = DiagnosticsConfig { recursion_cycle: Severity::Error, ..DiagnosticsConfig::new(Severity::Allow, true) }; - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let opts = CompileOpts { merge: true, ..CompileOpts::default() }; let res = compile_book(&mut book, opts, diagnostics_cfg, None)?; Ok(format!("{}{}", res.diagnostics, hvm_book_show_pretty(&res.hvm_book))) @@ -415,7 +443,7 @@ fn io() { &[ /* (&|code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path)?; + let book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default_lazy(); let diagnostics_cfg = DiagnosticsConfig::default_lazy(); let Output { status, stdout, stderr } = @@ -427,7 +455,7 @@ fn io() { }), */ (&|code, path| { let _guard = RUN_MUTEX.lock().unwrap(); - let book = do_parse_book(code, path, Book::builtins())?; + let book = do_parse_book_default(code, path)?; let compile_opts = CompileOpts::default(); let diagnostics_cfg = DiagnosticsConfig::default(); let (term, _, diags) = @@ -454,7 +482,7 @@ fn examples() -> Result<(), Diagnostics> { eprintln!("Testing {}", path.display()); let code = std::fs::read_to_string(path).map_err(|e| e.to_string())?; - let book = do_parse_book(&code, path, Book::builtins()).unwrap(); + let book = do_parse_book_default(&code, path).unwrap(); let compile_opts = CompileOpts::default(); let diagnostics_cfg = DiagnosticsConfig::default(); let (term, _, diags) = run_book_simple(book, RunOpts::default(), compile_opts, diagnostics_cfg, None)?; @@ -476,7 +504,7 @@ fn examples() -> Result<(), Diagnostics> { #[test] fn scott_triggers_unused() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let opts = CompileOpts::default(); let diagnostics_cfg = DiagnosticsConfig { unused_definition: Severity::Error, ..DiagnosticsConfig::default() }; @@ -489,7 +517,7 @@ fn scott_triggers_unused() { #[test] fn compile_long() { run_golden_test_dir(function_name!(), &|code, path| { - let mut book = do_parse_book(code, path, Book::builtins())?; + let mut book = do_parse_book_default(code, path)?; let opts = CompileOpts::default().set_all(); let diagnostics_cfg = DiagnosticsConfig { recursion_cycle: Severity::Warning, diff --git a/tests/golden_tests/import_system/import_main.bend b/tests/golden_tests/import_system/import_main.bend new file mode 100644 index 000000000..b6e8cbb72 --- /dev/null +++ b/tests/golden_tests/import_system/import_main.bend @@ -0,0 +1,4 @@ +import lib/import_entry + +def main(): + return * \ No newline at end of file diff --git a/tests/golden_tests/import_system/import_main2.bend b/tests/golden_tests/import_system/import_main2.bend new file mode 100644 index 000000000..4aaecbcbf --- /dev/null +++ b/tests/golden_tests/import_system/import_main2.bend @@ -0,0 +1,4 @@ +import lib/import_entry2 + +def main(): + return * \ No newline at end of file diff --git a/tests/golden_tests/import_system/import_main3.bend b/tests/golden_tests/import_system/import_main3.bend new file mode 100644 index 000000000..3b3feb2ed --- /dev/null +++ b/tests/golden_tests/import_system/import_main3.bend @@ -0,0 +1,4 @@ +import lib/folder/import_entry3 + +def main(): + return * \ No newline at end of file diff --git a/tests/golden_tests/import_system/import_types.bend b/tests/golden_tests/import_system/import_types.bend new file mode 100644 index 000000000..38b531ee1 --- /dev/null +++ b/tests/golden_tests/import_system/import_types.bend @@ -0,0 +1,20 @@ +from lib/types import (Bool, MyTree) +from lib import bool_xor + +def tree_xor(tree): + fold tree: + case MyTree/node: + return bool_xor(tree.lft, tree.rgt); + case MyTree/leaf: + return tree.val; + +main = + let depth = 10 + let tree = bend n = 0 { + when (< n depth): + (MyTree/node (fork (+ n 1)) (fork (+ n 1))) + else: + if (% n 2) { (MyTree/leaf Bool/True) } else { (MyTree/leaf Bool/False) } + } + (tree_xor tree) + diff --git a/tests/golden_tests/import_system/imports.bend b/tests/golden_tests/import_system/imports.bend new file mode 100644 index 000000000..b80ac5d98 --- /dev/null +++ b/tests/golden_tests/import_system/imports.bend @@ -0,0 +1,5 @@ +from lib/nums import (one, two) +from lib/defs import sum + +def main(): + return sum(one, two) \ No newline at end of file diff --git a/tests/golden_tests/import_system/imports_alias.bend b/tests/golden_tests/import_system/imports_alias.bend new file mode 100644 index 000000000..a23f95304 --- /dev/null +++ b/tests/golden_tests/import_system/imports_alias.bend @@ -0,0 +1,5 @@ +from lib/nums import (one as One, two as Two) +from lib/defs import (sum as summation) + +def main(): + return summation(One, Two) \ No newline at end of file diff --git a/tests/golden_tests/import_system/imports_alias_shadow.bend b/tests/golden_tests/import_system/imports_alias_shadow.bend new file mode 100644 index 000000000..f9cc3fbdc --- /dev/null +++ b/tests/golden_tests/import_system/imports_alias_shadow.bend @@ -0,0 +1,4 @@ +from lib/nums import (one as A, two as A) + +def main(): + return A + A \ No newline at end of file diff --git a/tests/golden_tests/import_system/imports_conflict.bend b/tests/golden_tests/import_system/imports_conflict.bend new file mode 100644 index 000000000..dd186ecc7 --- /dev/null +++ b/tests/golden_tests/import_system/imports_conflict.bend @@ -0,0 +1,7 @@ +from lib/a/b import C +from lib/a import b/C + +lib/a/b/C = * + +def main(): + return * \ No newline at end of file diff --git a/tests/golden_tests/import_system/imports_file_and_dir.bend b/tests/golden_tests/import_system/imports_file_and_dir.bend new file mode 100644 index 000000000..bf777af0e --- /dev/null +++ b/tests/golden_tests/import_system/imports_file_and_dir.bend @@ -0,0 +1,4 @@ +from lib/file_and_dir import (x, y) + +def main(): + return (x, y, y/z) diff --git a/tests/golden_tests/import_system/imports_file_and_dir_conflict.bend b/tests/golden_tests/import_system/imports_file_and_dir_conflict.bend new file mode 100644 index 000000000..3314b9619 --- /dev/null +++ b/tests/golden_tests/import_system/imports_file_and_dir_conflict.bend @@ -0,0 +1,4 @@ +from lib/file_and_dir import * + +def main(): + return w diff --git a/tests/golden_tests/import_system/imports_shadow.bend b/tests/golden_tests/import_system/imports_shadow.bend new file mode 100644 index 000000000..83c72fa2f --- /dev/null +++ b/tests/golden_tests/import_system/imports_shadow.bend @@ -0,0 +1,5 @@ +from lib/folder import myFun +from lib import myFun + +def main(): + return myFun \ No newline at end of file diff --git a/tests/golden_tests/import_system/imports_shadow2.bend b/tests/golden_tests/import_system/imports_shadow2.bend new file mode 100644 index 000000000..c0b1c26c2 --- /dev/null +++ b/tests/golden_tests/import_system/imports_shadow2.bend @@ -0,0 +1,5 @@ +from lib import myFun +from lib/folder import myFun + +def main(): + return myFun(lib/myFun/myFun) # use the full name to call a shadowed imported def \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/a.bend b/tests/golden_tests/import_system/lib/a.bend new file mode 100644 index 000000000..7df5b66bd --- /dev/null +++ b/tests/golden_tests/import_system/lib/a.bend @@ -0,0 +1 @@ +type b = C \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/a/b.bend b/tests/golden_tests/import_system/lib/a/b.bend new file mode 100644 index 000000000..eb69a519f --- /dev/null +++ b/tests/golden_tests/import_system/lib/a/b.bend @@ -0,0 +1 @@ +C = * \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/bool_xor.bend b/tests/golden_tests/import_system/lib/bool_xor.bend new file mode 100644 index 000000000..0d14cec64 --- /dev/null +++ b/tests/golden_tests/import_system/lib/bool_xor.bend @@ -0,0 +1,5 @@ +from lib/types import Bool + +(bool_xor Bool/True Bool/False) = Bool/True +(bool_xor Bool/False Bool/True) = Bool/True +(bool_xor * *) = Bool/False diff --git a/tests/golden_tests/import_system/lib/defs.bend b/tests/golden_tests/import_system/lib/defs.bend new file mode 100644 index 000000000..484711892 --- /dev/null +++ b/tests/golden_tests/import_system/lib/defs.bend @@ -0,0 +1,5 @@ +def sum(a, b): + return a + b + +def minus(a, b): + return a - b \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/file_and_dir.bend b/tests/golden_tests/import_system/lib/file_and_dir.bend new file mode 100644 index 000000000..c9245b5d8 --- /dev/null +++ b/tests/golden_tests/import_system/lib/file_and_dir.bend @@ -0,0 +1,3 @@ +x = 1 + +w = 2 \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/file_and_dir/w.bend b/tests/golden_tests/import_system/lib/file_and_dir/w.bend new file mode 100644 index 000000000..11844a5d5 --- /dev/null +++ b/tests/golden_tests/import_system/lib/file_and_dir/w.bend @@ -0,0 +1 @@ +w = 5 diff --git a/tests/golden_tests/import_system/lib/file_and_dir/y.bend b/tests/golden_tests/import_system/lib/file_and_dir/y.bend new file mode 100644 index 000000000..25e99f705 --- /dev/null +++ b/tests/golden_tests/import_system/lib/file_and_dir/y.bend @@ -0,0 +1,3 @@ +y = 3 + +z = 4 \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/folder/import_entry3.bend b/tests/golden_tests/import_system/lib/folder/import_entry3.bend new file mode 100644 index 000000000..af3ae16b2 --- /dev/null +++ b/tests/golden_tests/import_system/lib/folder/import_entry3.bend @@ -0,0 +1 @@ +import ../../../../import_main3 # multiples ../ have no effect past the main file folder \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/folder/myFun.bend b/tests/golden_tests/import_system/lib/folder/myFun.bend new file mode 100644 index 000000000..05300e2ba --- /dev/null +++ b/tests/golden_tests/import_system/lib/folder/myFun.bend @@ -0,0 +1,2 @@ +def myFun(a): + return a \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/import_entry.bend b/tests/golden_tests/import_system/lib/import_entry.bend new file mode 100644 index 000000000..f24cfc6c5 --- /dev/null +++ b/tests/golden_tests/import_system/lib/import_entry.bend @@ -0,0 +1 @@ +import import_main \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/import_entry2.bend b/tests/golden_tests/import_system/lib/import_entry2.bend new file mode 100644 index 000000000..4de1224ff --- /dev/null +++ b/tests/golden_tests/import_system/lib/import_entry2.bend @@ -0,0 +1 @@ +import ../import_main2 \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/myFun.bend b/tests/golden_tests/import_system/lib/myFun.bend new file mode 100644 index 000000000..2340332ea --- /dev/null +++ b/tests/golden_tests/import_system/lib/myFun.bend @@ -0,0 +1,8 @@ +type Type = (A) | (B) + +def myFun(a): + match a: + case Type/A: + return 1 + case Type/B: + return 2 \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/nums.bend b/tests/golden_tests/import_system/lib/nums.bend new file mode 100644 index 000000000..f428a17ac --- /dev/null +++ b/tests/golden_tests/import_system/lib/nums.bend @@ -0,0 +1,7 @@ +def one(): + return 1 + +two = 2 + +def three(): + return 3 \ No newline at end of file diff --git a/tests/golden_tests/import_system/lib/types.bend b/tests/golden_tests/import_system/lib/types.bend new file mode 100644 index 000000000..95858fb5a --- /dev/null +++ b/tests/golden_tests/import_system/lib/types.bend @@ -0,0 +1,5 @@ +type Bool: + True + False + +type MyTree = (node ~lft ~rgt) | (leaf val) diff --git a/tests/snapshots/desugar_file__tree_syntax.bend.snap b/tests/snapshots/desugar_file__tree_syntax.bend.snap index bb451600a..12d0870e6 100644 --- a/tests/snapshots/desugar_file__tree_syntax.bend.snap +++ b/tests/snapshots/desugar_file__tree_syntax.bend.snap @@ -12,6 +12,8 @@ input_file: tests/golden_tests/desugar_file/tree_syntax.bend (fun4) = λa switch a { 0: (Tree/Leaf 0); _: fun4__C0; } +(main) = * + (imp0) = (Tree/Node (Tree/Leaf 0) (Tree/Node (Tree/Leaf 1) (Tree/Node (Tree/Leaf 2) (Tree/Leaf 3)))) (imp1) = (Tree/Leaf (Tree/Node (Tree/Leaf *) (Tree/Leaf *))) @@ -22,8 +24,6 @@ input_file: tests/golden_tests/desugar_file/tree_syntax.bend (imp4) = λa switch a { 0: (Tree/Leaf 0); _: imp4__C0; } -(main) = * - (Tree/Node) = λa λb λc (c Tree/Node/tag a b) (Tree/Leaf) = λa λb (b Tree/Leaf/tag a) diff --git a/tests/snapshots/import_system__import_main.bend.snap b/tests/snapshots/import_system__import_main.bend.snap new file mode 100644 index 000000000..c058f3a3d --- /dev/null +++ b/tests/snapshots/import_system__import_main.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/import_main.bend +--- +Errors: +Can not import the entry point of the program. diff --git a/tests/snapshots/import_system__import_main2.bend.snap b/tests/snapshots/import_system__import_main2.bend.snap new file mode 100644 index 000000000..698bbe8f3 --- /dev/null +++ b/tests/snapshots/import_system__import_main2.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/import_main2.bend +--- +Errors: +Can not import the entry point of the program. diff --git a/tests/snapshots/import_system__import_main3.bend.snap b/tests/snapshots/import_system__import_main3.bend.snap new file mode 100644 index 000000000..f00d33137 --- /dev/null +++ b/tests/snapshots/import_system__import_main3.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/import_main3.bend +--- +Errors: +Can not import the entry point of the program. diff --git a/tests/snapshots/import_system__import_types.bend.snap b/tests/snapshots/import_system__import_types.bend.snap new file mode 100644 index 000000000..5c625738e --- /dev/null +++ b/tests/snapshots/import_system__import_types.bend.snap @@ -0,0 +1,5 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/import_types.bend +--- +lib/types/Bool/False diff --git a/tests/snapshots/import_system__imports.bend.snap b/tests/snapshots/import_system__imports.bend.snap new file mode 100644 index 000000000..c0dd761ce --- /dev/null +++ b/tests/snapshots/import_system__imports.bend.snap @@ -0,0 +1,5 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports.bend +--- +3 diff --git a/tests/snapshots/import_system__imports_alias.bend.snap b/tests/snapshots/import_system__imports_alias.bend.snap new file mode 100644 index 000000000..8fcb86fc5 --- /dev/null +++ b/tests/snapshots/import_system__imports_alias.bend.snap @@ -0,0 +1,5 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_alias.bend +--- +3 diff --git a/tests/snapshots/import_system__imports_alias_shadow.bend.snap b/tests/snapshots/import_system__imports_alias_shadow.bend.snap new file mode 100644 index 000000000..1c6942786 --- /dev/null +++ b/tests/snapshots/import_system__imports_alias_shadow.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_alias_shadow.bend +--- +Errors: +The import 'lib/nums/two' shadows the imported name 'lib/nums/one' diff --git a/tests/snapshots/import_system__imports_conflict.bend.snap b/tests/snapshots/import_system__imports_conflict.bend.snap new file mode 100644 index 000000000..f651bdd44 --- /dev/null +++ b/tests/snapshots/import_system__imports_conflict.bend.snap @@ -0,0 +1,7 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_conflict.bend +--- +Errors: +The imported definition 'lib/a/b/C' conflicts with the definition 'lib/a/b/C'. +The imported constructor 'lib/a/b/C' conflicts with the definition 'lib/a/b/C'. diff --git a/tests/snapshots/import_system__imports_file_and_dir.bend.snap b/tests/snapshots/import_system__imports_file_and_dir.bend.snap new file mode 100644 index 000000000..507d72c8d --- /dev/null +++ b/tests/snapshots/import_system__imports_file_and_dir.bend.snap @@ -0,0 +1,5 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_file_and_dir.bend +--- +(1, (3, 4)) diff --git a/tests/snapshots/import_system__imports_file_and_dir_conflict.bend.snap b/tests/snapshots/import_system__imports_file_and_dir_conflict.bend.snap new file mode 100644 index 000000000..4c28d7844 --- /dev/null +++ b/tests/snapshots/import_system__imports_file_and_dir_conflict.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_file_and_dir_conflict.bend +--- +Errors: +Both file 'lib/file_and_dir.bend' and folder 'lib/file_and_dir' contains the import 'w' diff --git a/tests/snapshots/import_system__imports_shadow.bend.snap b/tests/snapshots/import_system__imports_shadow.bend.snap new file mode 100644 index 000000000..284db8136 --- /dev/null +++ b/tests/snapshots/import_system__imports_shadow.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_shadow.bend +--- +Errors: +The import 'lib/myFun/myFun' shadows the imported name 'lib/folder/myFun/myFun' diff --git a/tests/snapshots/import_system__imports_shadow2.bend.snap b/tests/snapshots/import_system__imports_shadow2.bend.snap new file mode 100644 index 000000000..85e37d51d --- /dev/null +++ b/tests/snapshots/import_system__imports_shadow2.bend.snap @@ -0,0 +1,6 @@ +--- +source: tests/golden_tests.rs +input_file: tests/golden_tests/import_system/imports_shadow2.bend +--- +Errors: +The import 'lib/folder/myFun/myFun' shadows the imported name 'lib/myFun/myFun' diff --git a/tests/snapshots/scott_triggers_unused__test.bend.snap b/tests/snapshots/scott_triggers_unused__test.bend.snap index e94d7ece0..e7221e2cf 100644 --- a/tests/snapshots/scott_triggers_unused__test.bend.snap +++ b/tests/snapshots/scott_triggers_unused__test.bend.snap @@ -2,8 +2,12 @@ source: tests/golden_tests.rs input_file: tests/golden_tests/scott_triggers_unused/test.bend --- -Errors: -In definition 'bool/f/tag': - Definition is unused. -In definition 'bool/t/tag': - Definition is unused. +@bool/f = ((@bool/f/tag a) a) + +@bool/f/tag = 1 + +@bool/t = ((@bool/t/tag a) a) + +@bool/t/tag = 0 + +@main = (((?((0 (* 1)) a) a) b) b)