Execute as a script

[dn2007hw]

Script execution made simple with SML/NJ

Submitted By: Dayanandan Natarajan
Supervisor: Joe Wells
School of Mathematical and Computer Sciences
Heriot-Watt University

Objective

Primary objective of this change is to give the programmers and developers the capability of running a SML program as a script. Programmers and developers can write a SML program and run the program like a script over the command prompt.

Files modified

smlnj/base/cm/main/cm-boot.sml
smlnj/base/system/smlnj/internal/boot-env-fn.sml
smlnj/base/compiler/TopLevel/interact/interact.sig
smlnj/base/compiler/TopLevel/interact/interact.sml

Change details

1. interact.sig & interact.sml

A new function (useScriptFile) is added to Backend.Interact structure, which takes the file name and its content as a stream and process the stream by passing it to EvalLoop.evalStream.

a) New function declaration is added to interact.sig,

val useStream : TextIO.instream -> unit
val useScriptFile : string * TextIO.instream -> unit (* Addded by DAYA *)
val evalStream : TextIO.instream * Environment.environment -> Environment.environment

b) New function definition is added to interact.sml,

    fun useScriptFile (fname, stream) = ( 

      (EvalLoop.evalStream (fname, stream))
        handle exn => ( 
          EvalLoop.uncaughtExnMessage exn
          )  
      )

2. cm-boot.sml

The following changes were made in cm-boot.sml to recognise the new command line parameter passed from script.

a) In function args, line added to recognise the new command-line parameter ‘--script’, and a new function ‘nextargscript’ is called to initiate the process of the file.

        fun args ([], _) = ()
          | args ("-a" :: _, _) = nextarg autoload'
          | args ("-m" :: _, _) = nextarg make'
          | args (["-H"], _) = (help NONE; quit ())
          | args ("-H" :: _ :: _, mk) = (help NONE; nextarg mk)
          | args (["-S"], _) = (showcur NONE; quit ())
          | args ("-S" :: _ :: _, mk) = (showcur NONE; nextarg mk)
          | args (["-E"], _) = (show_envvars NONE; quit ())
          | args ("-E" :: _ :: _, mk) = (show_envvars NONE; nextarg mk)
          | args ("--script" :: _, _) = (nextargscript ())  (* line added by DAYA *)
          | args ("@CMbuild" :: rest, _) = mlbuild rest
          | args (["@CMredump", heapfile], _) = redump_heap heapfile
          | args (f :: rest, mk) =
          (carg (String.substring (f, 0, 2)
             handle General.Subscript => "",
             f, mk, List.null rest);
           nextarg mk)

        and nextarg mk =
        let val l = SMLofNJ.getArgs ()
        in SMLofNJ.shiftArgs (); args (l, mk)
        end

        (* nextargscript added by DAYA *)
        and nextargscript () =
        let val l = SMLofNJ.getArgs ()
        in SMLofNJ.shiftArgs (); processFileScript (hd l); quit ()
        end

b) In function init(), the new function (useScriptFile) is added as one of the parameter passed,

fun init (bootdir, de, er, useStream, useScriptFile, useFile, errorwrap, icm) = let

c) In function procCmdLine (), new function processFileScript is added to process the script file, function will check for whether the file passed on is a script file starting with ‘#!’ thru another new function checkSharpbang, consumes the first line thru another new function eatuntilneline and pass the remaining content of the file to function useScriptFile.

          (* DAYA change starts here *)
            fun eatuntilnewline (instream : TextIO.instream): bool = let
                val c = TextIO.input1 instream
                in
                    case TextIO.lookahead instream of
                        SOME #"\n" => true
                        | SOME c => eatuntilnewline instream
                        | NONE => false
                end

            fun checkSharpbang (instream : TextIO.instream): bool = let
                val c = TextIO.input1 instream
                in
                    case c of
                        SOME #"#" => (
                            case TextIO.lookahead instream of
                                SOME #"!" => eatuntilnewline instream
                                | SOME c => false
                                | NONE => false
                                )
                        | SOME c => false
                        | NONE => false
                end

            fun processFileScript (fname) = let
                val stream = TextIO.openIn fname
                val isscript = checkSharpbang stream
                in
                    if (isscript) = false  
                    then    ( Say.say [ "!* Script file doesn't start with #!. \n" ] ) 
                    else    ( useScriptFile (fname, stream) )
                end
            (* DAYA change ends here *)

3. boot-env-fn.sml

In functor BootEnvF, cminit function declaration is amended to include the newly added function useScriptFile.

functor BootEnvF (datatype envrequest = AUTOLOAD | BARE
          val architecture: string
          val cminit : string * DynamicEnv.env * envrequest
                   * (TextIO.instream -> unit)(* useStream *)
                   * (string * TextIO.instream -> unit) (* useScriptFile *)
                   * (string -> unit) (* useFile *)
                   * ((string -> unit) -> (string -> unit))
                                      (* errorwrap *)
                   * ({ manageImport:
                      Ast.dec * EnvRef.envref -> unit,
                    managePrint:
                      Symbol.symbol * EnvRef.envref -> unit,
                    getPending : unit -> Symbol.symbol list }
                  -> unit)
                   -> (unit -> unit) option
          val cmbmake: string * bool -> unit) :> BOOTENV = struct

Writing a script

The script should start with ‘#!’ in the first line followed by the environment location, command-line parameters ‘-Ssml’ and ‘—script’, a new line and then followed by the SML code or program.

Example script named ‘sample’,
--------------beginning of the script-------------
#!/usr/bin/env -Ssml –script
;(--SML--)
val () = print "Hello World\n";
--------------end of the script---------------------

Running a script

The script ‘sample’ can be executed from Linux terminal or command prompt as a regular OS script as below provided it is given execution permission,

$ ./sample

SML/NJ Version used

Our development and testing is based on Standard ML of New Jersey (32-bit) v110.99.3 on an Intel based macOS 10.13.16.

Test Details

Test Script #1

#!/usr/bin/env -Ssml --script
;(* SML code starts here *)
val x = "Hello World x\n";
val () = print x;
val y = "Hello World y\n";
val () = print y;
val z = "Hello World z\n";
val () = print z;

Test Result #1

$ ./sample
Standard ML of New Jersey (32-bit) v110.99.3 [built: Mon Apr 10 18:03:19 2023]
val x = "Hello World x\n" : string
Hello World x
val y = "Hello World y\n" : string
Hello World y
val z = "Hello World z\n" : string
Hello World z
$

Test Script #2 (sample script with forced error)

#!/usr/bin/env -Ssml --script
;(--SML--)
val x = "Hello World x\n";
val () = print x;
val y = "Hello World y\n";
val () == print y;
val z = "Hello World z\n";
val () = print z;

Test Result #2
$ ./sample
Standard ML of New Jersey (32-bit) v110.99.3 [built: Sat Apr 15 18:38:28 2023]
val x = "Hello World x\n" : string
Hello World x
val y = "Hello World y\n" : string
./exml07:7.18-8.4 Error: syntax error: deleting SEMICOLON VAL

uncaught exception Compile [Compile: "syntax error"]
raised at: ../compiler/Parse/main/smlfile.sml:19.24-19.46
../compiler/TopLevel/interact/evalloop.sml:45.54
../compiler/TopLevel/interact/evalloop.sml:306.20-306.23

[dmacqueen]

We need a fairly detailed description of what this does and how it works, and any new interfaces/signatures, and whether the top-level pervasive environment is changed. When an SML program is invoked as a "script", is it loaded and run in the REPL? Is the SML program that is run as a script an SML source file or is it a stand-alone program with an associated heap-image created by exportFn? In other words, is the goal to make the sml command behave like a Unix shell?

[dn2007hw]

HI Dave, The pull request is still in draft, I will be adding all the details before I submit the request.

[dmacqueen]

Ok, that’s what I expected, but I thought I’d give a potentially useful “prompt” about what we would like to see as documentation of the changes. Also keep in mind that if and when it is merged into legacy, we are likely to also want to merge into the smlnj/smlnj (“development”, but really the continuing release) repository. Also, I have been looking into the state of the compiler+CM “architecture”, which as become rather muddled. So I am interested in whether your work will impinge on architectural issues. We need to clarify the relation between CM and the interactive REPL, and investigate the idea of a “batch compilation” where stand-alone programs are built without using the REPL, its top-level environment, and exportFn. Dave

On Apr 15, 2023, at 10:50 AM, dn2007hw ***@***.***> wrote: HI Dave, The pull request is still in draft, I will be adding all the details before I submit the request. — Reply to this email directly, view it on GitHub <#276 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGXNPKOAH4KOM6R2YM4TO3XBLNW3ANCNFSM6AAAAAAW7RHZI4>. You are receiving this because you commented.

David MacQueen ***@***.***

[dmacqueen]

Thanks for that summary. This makes it fairly clear what you are doing. I’m not sure how your “silenceCompiler” function works. Dave

On Apr 15, 2023, at 10:58 AM, dn2007hw ***@***.***> wrote: Execute-as-a-script.txt <https://github.com/smlnj/legacy/files/11240286/Execute-as-a-script.txt> — Reply to this email directly, view it on GitHub <#276 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGXNPILMHHTDWE2QRRIE73XBLOUPANCNFSM6AAAAAAW7RHZI4>. You are receiving this because you commented.

David MacQueen ***@***.***

[dn2007hw]

silenceCompiler function is not part of this patch, I have removed the reference. It's part of next patch, I will be adding the details in that.

[JohnReppy]

We should think about the name of the command-line option. We currently do not use names like --script. I would suggest @SMLscript (using the @SML prefix is a way to avoid colliding with options that the script might want to handle).

Also, the code can be tightened up a bit and should be made more robust. For example, it might be more direct to process the first line of the script using TextIO.StreamIO.inputLine

(* if the first line of the input stream begins with "#!", then consume the line
 * and return `true`; otherwise return `false` and do not advance the input.
 *)
fun checkSharpbang inS = let
      val inS' = TextIO.getInstream inS
      in
        case TextIO.StreamIO.inputLine inS'
         of SOME(firstLn, inS'') =>
              if String.isPrefix "#!" firstLn
                then (TextIO.setInstream(inS, inS''); true)
                else false
          | _ => false
        (* end case *)
      end

[Skyb0rg007]

This seems like a great feature to add, but I wanted to add my 2 cents about how to handle the shebang.
I think the best way to handle the "#!" is in the lexer: if the first 2 bytes of a file are "#!", then skip until the first "\n" like a line comment.

I think it's also important to not require a command-line flag.
On Linux, shebangs are only split into (executable, argument), and because sml could be installed in many different locations it would be good practice to use /usr/bin/env.
That doesn't leave any room for script arguments.

Tools such as nix-shell or languages like Rust (via cargo) support command-line options via comments.

#! /usr/bin/env cargo
//! ```cargo
//! [dependencies]
//! serde = "1.0.215"
//! ```
use serde::ser::{Serialize};

fn main () {
  println!("Hello from Rust!");
}

This could be integrated into SML/NJ by parsing the first comment in the file, and extracting arguments if the comment is of a certain form.
It could also allow for loading CM modules if executed as a script.

#! /usr/bin/env sml
(*#script -Cmlrisc.disable-jump-chain-elim $smlnj-lib.cm $pp-lib.cm *)

(* Executing this file runs `sml  -Cmlrisc.disable-jump-chain-elim '$smlnj-lib.cm' '$pp-lib.cm' <file.sml>`

val () = print "Hello from SML/NJ!\n"

Handling the shebang in the lexer also makes it possible to write SML files that act as both libraries and scripts.
For example, it would be possible to write:

#! /usr/bin/env sml

structure MyModule : sig
  val foo : unit -> unit
end = struct
  fun internal () = "Internal"

  fun foo () = print (internal ())

  fun test () =
    if internal () = "Internal" then () else raise Fail "Test failure"
  val () = if !SMLofNJ.isScript then test () else ()
end

Running the file would execute the script, but it could also be loaded like a normal file that defines a structure.