Skip to content

Commit

Permalink
Emit WASM (#3897)
Browse files Browse the repository at this point in the history
Seeks to address #3837 

I have some outstanding questions;

1. As I'm altering the scalaJSModule public API, a double check here
would be welcomed. In particular, I've just called the module property
`emitWasm`. sjrd pretty clearly called the linker API
`withExperimentalUseWebAssembly`. However, if at some point it became
non-experimental, I assume that's a compatibility problem. Advice
welcomed - I've gone with the simplest name possible - `emitWasm`.
2. CI will fail, because node is < 22. The runnable test does work
locally for me. Does mill require, assume or otherwise advertise
compatibility with any particular node version? If not, I'll simply push
the node version forward in the GHA definitions. OItherwise, I'd welcome
advice - I'm not clear what the best way to manage the CI would be.

Those are the two points where I'd welcome advice. I think this is the
path to victory;

- [x] public API check? 
- [x] node version
- [x] documentation
- [x] CI green
  • Loading branch information
Quafadas authored Nov 3, 2024
1 parent d5d56b5 commit 692c0dd
Show file tree
Hide file tree
Showing 10 changed files with 200 additions and 12 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/run-mill-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ jobs:
java-version: ${{ inputs.java-version }}
distribution: temurin

- uses: actions/setup-node@v4
with:
node-version: '22'


- name: Prepare git config
run: |
git config --global user.name "Mill GithHub Actions"
Expand Down
6 changes: 5 additions & 1 deletion docs/modules/ROOT/pages/scalalib/web-examples.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,8 @@ include::partial$example/scalalib/web/6-cross-version-platform-publishing.adoc[]

== Publishing Cross-Platform Scala Modules Alternative

include::partial$example/scalalib/web/7-cross-platform-version-publishing.adoc[]
include::partial$example/scalalib/web/7-cross-platform-version-publishing.adoc[]

== Scala.js WebAssembly Example

include::partial$example/scalalib/web/8-wasm.adoc[]
42 changes: 42 additions & 0 deletions example/scalalib/web/8-wasm/build.mill
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package build
import mill._, scalalib._, scalajslib._
import mill.scalajslib.api._


object wasm extends ScalaJSModule {
override def scalaVersion = "2.13.14"

override def scalaJSVersion = "1.17.0"

override def moduleKind = ModuleKind.ESModule

override def moduleSplitStyle = ModuleSplitStyle.FewestModules

override def scalaJSExperimentalUseWebAssembly = true
}

// This build defines a single `ScalaJSModule` that uses the `WASM` backend of the scala JS linker.
// The release notes that introduced scalaJS wasm are here;
// https://www.scala-js.org/news/2024/09/28/announcing-scalajs-1.17.0/
// and are worth reading. They include information such as the scala JS requirements to successfully emit wasm,
// the flags needed to run in browser and the minimum node version (22) required to actually run the wasm output.

/** Usage

> ./mill show wasm.fastLinkJS # mac/linux
{
...
..."jsFileName": "main.js",
...
"dest": ".../out/wasm/fastLinkJS.dest"
}

> node --experimental-wasm-exnref out/wasm/fastLinkJS.dest/main.js # mac/linux
hello wasm!

*/

// Here we see that scala JS emits a single WASM module, as well as a loader and main.js file.
// `main.js` is the entry point of the program, and calls into the wasm module.


7 changes: 7 additions & 0 deletions example/scalalib/web/8-wasm/wasm/src/hello.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package wasm

object wasm {
def main(args: Array[String]): Unit = {
println("hello wasm!")
}
}
27 changes: 23 additions & 4 deletions scalajslib/src/mill/scalajslib/ScalaJSModule.scala
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ trait ScalaJSModule extends scalalib.ScalaModule { outer =>
moduleSplitStyle = moduleSplitStyle(),
outputPatterns = scalaJSOutputPatterns(),
minify = scalaJSMinify(),
importMap = scalaJSImportMap()
importMap = scalaJSImportMap(),
experimentalUseWebAssembly = scalaJSExperimentalUseWebAssembly()
)
}

Expand Down Expand Up @@ -191,7 +192,8 @@ trait ScalaJSModule extends scalalib.ScalaModule { outer =>
moduleSplitStyle: ModuleSplitStyle,
outputPatterns: OutputPatterns,
minify: Boolean,
importMap: Seq[ESModuleImportMapping]
importMap: Seq[ESModuleImportMapping],
experimentalUseWebAssembly: Boolean
)(implicit ctx: mill.api.Ctx): Result[Report] = {
val outputPath = ctx.dest

Expand All @@ -212,7 +214,8 @@ trait ScalaJSModule extends scalalib.ScalaModule { outer =>
moduleSplitStyle = moduleSplitStyle,
outputPatterns = outputPatterns,
minify = minify,
importMap = importMap
importMap = importMap,
experimentalUseWebAssembly = experimentalUseWebAssembly
)
}

Expand Down Expand Up @@ -293,6 +296,21 @@ trait ScalaJSModule extends scalalib.ScalaModule { outer =>
/** Whether to emit a source map. */
def scalaJSSourceMap: T[Boolean] = Task { true }

/**
* Specifies whether to use the experimental WebAssembly backend.. Requires scalaJS > 1.17.0
* When using this setting, the following properties must also hold:
*
* - `moduleKind = ModuleKind.ESModule`
* - `moduleSplitStyle = ModuleSplitStyle.FewestModules`
*
* @note
* Currently, the WebAssembly backend silently ignores `@JSExport` and
* `@JSExportAll` annotations. This behavior may change in the future,
* either by making them warnings or errors, or by adding support for them.
* All other language features are supported.
*/
def scalaJSExperimentalUseWebAssembly: T[Boolean] = Task { false }

/** Name patterns for output. */
def scalaJSOutputPatterns: T[OutputPatterns] = Task { OutputPatterns.Defaults }

Expand Down Expand Up @@ -370,7 +388,8 @@ trait TestScalaJSModule extends ScalaJSModule with TestModule {
moduleSplitStyle = moduleSplitStyle(),
outputPatterns = scalaJSOutputPatterns(),
minify = scalaJSMinify(),
importMap = scalaJSImportMap()
importMap = scalaJSImportMap(),
experimentalUseWebAssembly = scalaJSExperimentalUseWebAssembly()
)
}

Expand Down
6 changes: 4 additions & 2 deletions scalajslib/src/mill/scalajslib/worker/ScalaJSWorker.scala
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,8 @@ private[scalajslib] class ScalaJSWorker extends AutoCloseable {
moduleSplitStyle: api.ModuleSplitStyle,
outputPatterns: api.OutputPatterns,
minify: Boolean,
importMap: Seq[api.ESModuleImportMapping]
importMap: Seq[api.ESModuleImportMapping],
experimentalUseWebAssembly: Boolean
)(implicit ctx: Ctx.Home): Result[api.Report] = {
bridge(toolsClasspath).link(
runClasspath = runClasspath.iterator.map(_.path.toNIO).toSeq,
Expand All @@ -185,7 +186,8 @@ private[scalajslib] class ScalaJSWorker extends AutoCloseable {
moduleSplitStyle = toWorkerApi(moduleSplitStyle),
outputPatterns = toWorkerApi(outputPatterns),
minify = minify,
importMap = importMap.map(toWorkerApi)
importMap = importMap.map(toWorkerApi),
experimentalUseWebAssembly = experimentalUseWebAssembly
) match {
case Right(report) => Result.Success(fromWorkerApi(report))
case Left(message) => Result.Failure(message)
Expand Down
10 changes: 10 additions & 0 deletions scalajslib/test/resources/wasm/src/app/App.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package app

import scala.scalajs.js
import scala.scalajs.js.annotation._

object App {
def main(args: Array[String]): Unit = {
println("hello, wasm!")
}
}
87 changes: 87 additions & 0 deletions scalajslib/test/src/mill/scalajslib/WasmTests.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
package mill.scalajslib

import mill.api.Result
import mill.define.Discover
import mill.testkit.UnitTester
import mill.testkit.TestBaseModule
import utest._
import mill.scalajslib.api._
import mill.T

object WasmTests extends TestSuite {
val remapTo = "https://cdn.jsdelivr.net/gh/stdlib-js/array-base-linspace@esm/index.mjs"

object Wasm extends TestBaseModule with ScalaJSModule {
override def scalaVersion = sys.props.getOrElse("TEST_SCALA_2_13_VERSION", ???)

override def scalaJSVersion = "1.17.0"

override def moduleKind = ModuleKind.ESModule

override def moduleSplitStyle = ModuleSplitStyle.FewestModules

override def scalaJSExperimentalUseWebAssembly: T[Boolean] = true

override lazy val millDiscover = {
import mill.main.TokenReaders.given
Discover[this.type]
}
}

object OldWasmModule extends TestBaseModule with ScalaJSModule {
override def scalaVersion = sys.props.getOrElse("TEST_SCALA_2_13_VERSION", ???)
override def scalaJSVersion = "1.16.0"

override def moduleKind = ModuleKind.ESModule
override def moduleSplitStyle = ModuleSplitStyle.FewestModules

override def scalaJSExperimentalUseWebAssembly: T[Boolean] = true

override lazy val millDiscover = {
import mill.main.TokenReaders.given
Discover[this.type]
}
}

val millSourcePath = os.Path(sys.env("MILL_TEST_RESOURCE_DIR")) / "wasm"

val tests: Tests = Tests {
test("should emit wasm") {
val evaluator = UnitTester(Wasm, millSourcePath)
val Right(result) =
evaluator(Wasm.fastLinkJS)
val publicModules = result.value.publicModules.toSeq
val path = result.value.dest.path
val main = publicModules.head
assert(main.jsFileName == "main.js")
val mainPath = path / "main.js"
assert(os.exists(mainPath))
val wasmPath = path / "main.wasm"
assert(os.exists(wasmPath))
val wasmMapPath = path / "main.wasm.map"
assert(os.exists(wasmMapPath))
}

test("wasm is runnable") {
val evaluator = UnitTester(Wasm, millSourcePath)
val Right(result) = evaluator(Wasm.fastLinkJS)
val path = result.value.dest.path
os.proc("node", "--experimental-wasm-exnref", "main.js").call(
cwd = path,
check = true,
stdin = os.Inherit,
stdout = os.Inherit,
stderr = os.Inherit
)

}

test("should throw for older scalaJS versions") {
val evaluator = UnitTester(OldWasmModule, millSourcePath)
val Left(Result.Exception(ex, _)) = evaluator(OldWasmModule.fastLinkJS)
val error = ex.getMessage
assert(error == "Emitting wasm is not supported with Scala.js < 1.17")
}

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ private[scalajslib] trait ScalaJSWorkerApi {
moduleSplitStyle: ModuleSplitStyle,
outputPatterns: OutputPatterns,
minify: Boolean,
importMap: Seq[ESModuleImportMapping]
importMap: Seq[ESModuleImportMapping],
experimentalUseWebAssembly: Boolean
): Either[String, Report]

def run(config: JsEnvConfig, report: Report): Unit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ class ScalaJSWorkerImpl extends ScalaJSWorkerApi {
moduleSplitStyle: ModuleSplitStyle,
outputPatterns: OutputPatterns,
minify: Boolean,
dest: File
dest: File,
experimentalUseWebAssembly: Boolean
)
private def minorIsGreaterThanOrEqual(number: Int) = ScalaJSVersions.current match {
case s"1.$n.$_" if n.toIntOption.exists(_ < number) => false
Expand Down Expand Up @@ -153,7 +154,15 @@ class ScalaJSWorkerImpl extends ScalaJSWorkerApi {
if (minorIsGreaterThanOrEqual(16)) withOutputPatterns.withMinify(input.minify)
else withOutputPatterns

val linker = StandardImpl.clearableLinker(withMinify)
val withWasm =
(minorIsGreaterThanOrEqual(17), input.experimentalUseWebAssembly) match {
case (_, false) => withMinify
case (true, true) => withMinify.withExperimentalUseWebAssembly(true)
case (false, true) =>
throw new Exception("Emitting wasm is not supported with Scala.js < 1.17")
}

val linker = StandardImpl.clearableLinker(withWasm)
val irFileCacheCache = irFileCache.newCache
(linker, irFileCacheCache)
}
Expand All @@ -180,7 +189,8 @@ class ScalaJSWorkerImpl extends ScalaJSWorkerApi {
moduleSplitStyle: ModuleSplitStyle,
outputPatterns: OutputPatterns,
minify: Boolean,
importMap: Seq[ESModuleImportMapping]
importMap: Seq[ESModuleImportMapping],
experimentalUseWebAssembly: Boolean
): Either[String, Report] = {
// On Scala.js 1.2- we want to use the legacy mode either way since
// the new mode is not supported and in tests we always use legacy = false
Expand All @@ -195,7 +205,8 @@ class ScalaJSWorkerImpl extends ScalaJSWorkerApi {
moduleSplitStyle = moduleSplitStyle,
outputPatterns = outputPatterns,
minify = minify,
dest = dest
dest = dest,
experimentalUseWebAssembly = experimentalUseWebAssembly
))
val irContainersAndPathsFuture = PathIRContainer.fromClasspath(runClasspath)
val testInitializer =
Expand Down

0 comments on commit 692c0dd

Please sign in to comment.