importing from other files/packages

[jedwards1211]

The language reference doesn't say anything about importing from one ci file to another, are there any import statements yet?

I'm hoping to be able to use it to make packages that depend on each other. I mostly have experience publishing Node.js packages, but would like to be able to make Maven, Pip, etc. packages (maybe vcpkg for C++, no idea if it's any good...) I know modules differ a lot from one language to another so I wouldn't be surprised if Ć lacks any solution for this yet.

[jedwards1211]

of course, I can use conditional compilation to achieve this, though it would be nice to have a simpler way. If I had many files it would become tempting to write some kind of pre- or postprocessor that converts magic comments into the language-specific imports or includes.

[pfusik]

What are you trying to achieve?
You can specify several *.ci files on the cito command line.

[jedwards1211]

Ah I see so it makes a bundle. That's probably fine for my purposes, though if an API gets large enough I imagine there's a benefit to importing/including only the classes one needs.

I'm looking to make a unitized package that exports Length and Angle classes, and then some other packages that parse cave survey data and use those classes to represent the measurements.

For example the @speleotica/unitized npm package would have export {Length, Angle} in the code and the other packages would import {Length, Angle} from '@speleotica/unitized'. Similar for Python I guess. In Java I would add package org.speleotica.unitized; to all the generated files. In C++ I would add a namespace speleotica::unitized { wrapper around the generated code. C would be a problem, I'd like to be able to add a namespace prefix to all the generated functions and types (though I don't know anyone who would want to use this API in C at least).

[pfusik]

I see. One problem with namespaces is that every target language does it differently:

• C: prefix to structures, functions and macros
• C++: namespace some_short_name
• C#: namespace Vendor.Product
• Java: package reverse.domain.name
• JavaScript: an entry in the global dictionary, is there a standard syntax?
• Python: short filename becomes the module name
• Swift: not sure how?

Therefore there's even no Ć construct to define the namespace. It is passed on the command line instead.

Referencing namespaces or modules is another story.

I'm all open for discussion how to solve it best!

[jedwards1211]

JS has had the CommonJS system for a long time now, where you can require('./relative/path/to/file') or require('dependencyName/subpath') (where node_modules/dependencyName was installed by NPM). Exports are done via exports.foo = ... or module.exports = .... Browsers don't support it but people have been using bundlers that bundle together all the required files for the browser for quite awhile.

Now a newer ES Modules syntax has landed in the latest versions of node/browsers, which looks like import Foo from './Foo', export default class Foo { ... } etc.

So there could be an option to output multiple files and have them import from each other by relative paths, as well as an option whether to output CommonJS or ES Modules syntax.

As far as one Ć lib importing from another, it's definitely tricky:

• C: no problem using the right prefixes, but I'm not sure if you can assume the right path for #include directives?
• C++: again no problem using namespaces, but I'm not sure if you can assume the right path for #include directives?
• Java: easy case, the user just has to make sure the dependency is on the classpath, and it can be imported
• C#: using the right namespace is almost the same is importing a package in Java right?
• JavaScript: Ć needs to require or import from whatever package name the dependency was published as
• Python: I know it's possible to import both local files and installed dependencies, but I don't remember the details
• Swift: importing deps looks pretty similar to Java or Python: https://thoughtbot.com/blog/swift-imports

I doubt there would be a problem using the same user-configured name for C++/C# namespaces and Java packages (with the appropriate transformations), but I think we'd want to let the user configure JS and Python packages names separately.

[jedwards1211]

btw what's the command line option for namespaces you mentioned?

[pfusik]

The cito option is -n with an argument. Run cito without arguments to see all available options.

Thanks for your comment! This is a complex problem and I'd like to approach it with small steps.

Let's start with JS modules. My understanding is that there's a widely used CommonJS and an incompatible emerging ES Modules?
I tried to learn about CommonJS but http://www.commonjs.org/specs/ is blank.
What's the naming convention? Your example look like every single class gets exported individually. Is myNamespace.MyClass possible?

[jedwards1211]

The Node.js docs cover CommonJS in https://nodejs.org/dist/latest-v12.x/docs/api/modules.html#modules_require_id
and https://nodejs.org/dist/latest-v12.x/docs/api/modules.html#modules_module_exports.

And ECMAScript modules are covered in https://nodejs.org/dist/latest-v12.x/docs/api/esm.html.

There's no hierarchical naming convention like Java packages or nested namespaces in C++/C#, because no one's putting org.foo.bar.MyClass on the global object anymore. Instead, the import and require statements below just bind whatever names in the local module (file) scope.

myNamespace.MyClass is possible with:

// ES Modules way
import * as myNamespace from 'my-package' // myPackage published to npm
myNamespace.MyClass

// Or
import { MyClass } from 'myPackage'

// CommonJS way
const myNamespace = require('my-package') // myPackage published to npm
myNamespace.MyClass

// Or
const { MyClass } = require('my-package')

Or you can import from a local file instead of an npm package by doing

import * as myNamespace from './path/to/file'

(Same for CommonJS)

The file you're importing from must have

// ES Modules way
export class MyClass { ... }

// CommonJS way
class MyClass { ... }
exports.MyClass = MyClass

Packages published to npm have to have a globally unique name and the convention is kebab-case. But if you own the organization foobar on npm (orgs of course have to have a globally unique name) you can publish "scoped" packages like @foobar/my-package.

[jedwards1211]

The first blocker for me on this issue is actually just that there's no way to make cito take external .ci files into account without bundling them. For example, in my unitized package I would have Length.ci and Angle.ci. Then in my compass-parser package I would have CompassAST.ci, CompassParser.ci, CompassFormatter.ci. I would need to compile all of the compass-parser files against Length.ci and Angle.ci (to prevent unknown identifier errors etc) without including the Length and Angle classes in the output bundle.

If cito had a flag to include files for type checking but not output them, then I could at least add the necessary import and export declarations for each language in my own postprocessing.

[jedwards1211]

Actually I noticed this CLI option:

-I DIR     Add directory to resource search path

Almost sounds like what I'm looking for, but I'm not sure, and I couldn't manage to get it to prevent type not found errors anyway.

[pfusik]

-I DIR only works for resources.

Do you want to author two separate packages that have a common part?
In C++/C#/Java these will be likely in different namespaces, that's why I wanted to solve the problem of declaring and referencing namespaces first.
Thanks for the detailed explanation of the JS case - as I see, either everything ends in the global namespace or the namespaces are defined at import time.

[pfusik]

there's no way to make cito take external .ci files into account without bundling them

Please pull and you can try:

cito -D ES -o unitized.js Length.ci Angle.ci
cito -D ES -o compass-parser.js CompassAST.ci CompassParser.ci CompassFormatter.ci -r Length.ci -r Angle.ci

(-r is short for "references").

add the necessary import and export declarations for each language in my own postprocessing

You can try top-level native blocks, e.g.

// at the top level of CompassParser.ci
#if ES
native { import { Length, Angle } from "unitized" }
#elif CJS
native { const { Length, Angle } = require("unitized") }
#elif CSHARP
...
#endif

[jedwards1211]

Do you want to author two separate packages that have a common part?
In C++/C#/Java these will be likely in different namespaces, that's why I wanted to solve the problem of declaring and referencing namespaces first.

Yup, and I would use different namespaces if I were writing the C++ etc by hand for the packages anyway. It would be possible to insert the appropriate using directives so that the code generation doesn't have to insert scope resolution everywhere a class name is referenced right?

[pfusik]

My point is that I'd like to import a different namespace without resorting to conditional compilation. But let's try #if/native first.

[jedwards1211]

The -r flag works, thanks for implementing stuff so fast! Soon I would like to make a PR for typescript declarations output (.d.ts files). Looks like it won't be too hard, especially since I can omit function bodies (.d.ts files just declare types for a .js file of the same name)

[jedwards1211]

@pfusik ah, agreed, it would be convenient of course to have cito deal with namespace imports. I just assume it will take awhile so we can decide carefully how to let the user configure everything (I think some kind of file to go along with every bundle that configures namespaces for C++/C#, prefix for C, and import paths or packages for JS/Python etc...)

[pfusik]

A "project" file. Cool idea!

[jedwards1211]

yup exactly. that way when building my unitized project for example it would know what exports/namespaces to add, and then when I build other projects I could pass the project file for unitized, and it could use that to determine how to import for each language etc.

It would probably make sense to have a list of source .ci files in the project file as well, that would simplify commands to build that project and to build other projects that depend on it.

[jedwards1211]

All Node.js projects have a package.json file that configures the name and version of the published package and tons of other things, which is basically what gave me this idea

[pfusik]

I consider this an important feature. I'm just still far from knowing how to design it. After so many decades, C and C++ don't have a standard package manager or project file. Ć is on par with them in this area. Suggestions are welcome!

[jedwards1211]

Yeah it's tough. Whenever I get back to working on this stuff I'll play around with putting in the import statements for each target language with macros.

[pfusik]

Another somewhat related topic is integrating cito with other build tools (for Java, C# etc). For example, in RECOIL cito is called from standard Android and Windows 10 projects.

I have enabled https://github.com/pfusik/cito/discussions for general discussions. It might work better than the tracker which is suited to well-defined tasks.

[danon]

If I am to add anything, is that I've been developing applications in PHP for over 5 years now, and it's bloody irritanting and unmaintainable mess that everything from PHP is in global namespace. Please, don't make the same mistake in Ćito, and if there are to be namespaces (and I think they must be), then please put everything that Ćito provides into some kind of namespace. Don't leave anything in global namespace.

Question.

I assume @pfusik you don't want to do dynamic imports? Like in Python, JS and PHP you can do this:

def method():
  print("unloaded")
  from x import y
  print("imported ", y)

function method() {
  console.log("unloaded");
  import("x").then(y => console.log("loaded"));
}

function method() {
  echo "unloaded":
  $x = require('./x');
  echo "loaded" . $x;

Right? Because how would you transpile that to Java/C#/C++, right? No can do (or I am unaware of any such thing). So the only way of any kind of modularity are namespaces, correct? Or am I missing something.

[pfusik]

it's bloody irritanting and unmaintainable mess that everything from PHP is in global namespace.

Could you give some examples? I'm unfamiliar with PHP.

Note that C doesn't have namespaces either and it hasn't stopped people from implementing major operating system kernels for the last five decades, several compilers and 3D engines.
C++ does have namespaces, but its whole standard library is in the namespace std.

do dynamic imports

What do you mean by "dynamic" ?
It's feasible to import symbols into a local scope. C++ supports that with using. Even for languages that don't support it, this could be resolved during the transpilation, e.g.

public void Foo()
{
    import Bar.Quux; // transpiles to nothing
    Quux.DoSth(); // transpiles to Bar.Quux.DoSth();
}

[danon]

it's bloody irritanting and unmaintainable mess that everything from PHP is in global namespace.

Could you give some examples? I'm unfamiliar with PHP.

Virtually every function that exists in PHP is in global namespace.

Note that C doesn't have namespaces either and it hasn't stopped people from implementing major operating system kernels for the last five decades, several compilers and 3D engines. C++ does have namespaces, but its whole standard library is in the namespace std.

Yes, but there is this #include thingy. Which means that you can have two functions named the same way, just #include one file or the other. If there wasn't #include, but instead everything would be declared in the scope (like in java), that would be a totally different story.

do dynamic imports

What do you mean by "dynamic" ? It's feasible to import symbols into a local scope. C++ supports that with using. Even for languages that don't support it, this could be resolved during the transpilation, e.g.

public void Foo()
{
    import Bar.Quux; // transpiles to nothing
    Quux.DoSth(); // transpiles to Bar.Quux.DoSth();
}

No, not true, because in dynamic imports, when there's an exception that exception is thrown at the time of the import.

File source.py::

x = 2
raise Exception("welcome")
y = 3

When you run this code

print("hello")
from source import x
print("there", x)

you will see

hello
Traceback (most recent call last):
  File "C:\Users\Daniel\PycharmProjects\sync\test.py", line 2, in <module>
    from source import x
  File "C:\Users\Daniel\PycharmProjects\sync\source.py", line 2, in <module>
    raise Exception("welcoem")
Exception: welcome

That's how imports work in Python, JS and vanilla PHP. That's exactly what I meant when I asked #65 And opposite would be how imports work in Java and C#.

The C++ as you noticed is an exception. I think that's because you can include only .h files, without actual implementation.