[Test] [bug] "test_runtime.py" failing on Windows

[xumingkuan]

Describe the bug
ti.exe may not work properly on Windows in some cases -- test_runtime.py::test_without_init (which introduced in #1347) fails on my end because it calls ti.exe.

Log/Screenshots

tests/python/test_runtime.py::test_without_init [Taichi] mode=development
[Taichi] <dev mode>, llvm 8.0.1, commit f2bd9829, python 3.7.4

*******************************************
**      Taichi Programming Language      **
*******************************************
C:\Users\xmk\AppData\Local\Temp\tmpqrx9npc8 is not a valid command!
usage:
    baseline            |-> Archive current benchmark result as baseline
    benchmark           |-> Run Python tests in benchmark mode
    build               |-> Build C++ files
    changelog           |-> Display changelog of current version
    debug               |-> Debug a single script
    dist                |-> Build package and test in release mode
    doc                 |-> Build documentation
    example             |-> Run an example by name (or name.py)
    format              |-> Reformat modified source files
    format_all          |-> Reformat all source files
    gif                 |-> Convert mp4 file to gif in the same directory
    regression          |-> Display benchmark regression test result
    release             |-> Make source code release
    run                 |-> Run a single script
    task                |-> Run a specific task
    test                |-> Run the tests
    update              |-> Update the Taichi codebase
    video               |-> Make a video using *.png files in the current directory
    video_crop          |-> Crop video in the same directory
    video_scale         |-> Scale video resolution in the same directory
    video_speed         |-> Speed up video in the same directory

Taichi CLI

positional arguments:
  command     command from the above list to run

optional arguments:
  -h, --help  show this help message and exit
>>> Running time: 0.00s
FAILED

=========================================== FAILURES ===========================================
______________________________________ test_without_init _______________________________________

    def test_without_init():
        # We want to check if Taichi works well without ``ti.init()``.
        # But in test ``ti.init()`` will always be called in last ``@ti.all_archs``.
        # So we have to create a new Taichi instance, i.e. test in a sandbox.
        content = '''
    import taichi as ti
    assert ti.cfg.arch == ti.cpu
    
    x = ti.var(ti.i32, (2, 3))
    assert x.shape == (2, 3)
    
    x[1, 2] = 4
    assert x[1, 2] == 4
    '''
        filename = make_temp_file()
        with open(filename, 'w') as f:
            f.write(content)
>       assert os.system(f'{sys.executable} {filename}') == 0
E       assert 1 == 0
E         -1
E         +0

tests\python\test_runtime.py:23: AssertionError
============================== 1 failed, 3 passed in 3.59 seconds ==============================
>>> Running time: 4.21s

To Reproduce
The failing test:

import taichi as ti
from taichi import make_temp_file
import sys, os


def test_without_init():
    # We want to check if Taichi works well without ``ti.init()``.
    # But in test ``ti.init()`` will always be called in last ``@ti.all_archs``.
    # So we have to create a new Taichi instance, i.e. test in a sandbox.
    content = '''
import taichi as ti
assert ti.cfg.arch == ti.cpu

x = ti.var(ti.i32, (2, 3))
assert x.shape == (2, 3)

x[1, 2] = 4
assert x[1, 2] == 4
'''
    filename = make_temp_file()
    with open(filename, 'w') as f:
        f.write(content)
    assert os.system(f'{sys.executable} {filename}') == 0

This works:

import taichi as ti
from taichi import make_temp_file
import sys, os


def test_without_init():
    # We want to check if Taichi works well without ``ti.init()``.
    # But in test ``ti.init()`` will always be called in last ``@ti.all_archs``.
    # So we have to create a new Taichi instance, i.e. test in a sandbox.
    content = '''
import taichi as ti
assert ti.cfg.arch == ti.cpu

x = ti.var(ti.i32, (2, 3))
assert x.shape == (2, 3)

x[1, 2] = 4
assert x[1, 2] == 4
'''
    filename = make_temp_file()
    with open(filename, 'w') as f:
        f.write(content)
    assert os.system(f'python -m taichi {filename}') == 0

(Corresponding log:)

tests/python/test_runtime.py::test_without_init [Taichi] mode=development
[Taichi] <dev mode>, llvm 8.0.1, commit f2bd9829, python 3.7.4

*******************************************
**      Taichi Programming Language      **
*******************************************
C:\Users\xmk\AppData\Local\Temp\tmpr6112yd1 is not a valid command!
usage:
    baseline            |-> Archive current benchmark result as baseline
    benchmark           |-> Run Python tests in benchmark mode
    build               |-> Build C++ files
    changelog           |-> Display changelog of current version
    debug               |-> Debug a single script
    dist                |-> Build package and test in release mode
    doc                 |-> Build documentation
    example             |-> Run an example by name (or name.py)
    format              |-> Reformat modified source files
    format_all          |-> Reformat all source files
    gif                 |-> Convert mp4 file to gif in the same directory
    regression          |-> Display benchmark regression test result
    release             |-> Make source code release
    run                 |-> Run a single script
    task                |-> Run a specific task
    test                |-> Run the tests
    update              |-> Update the Taichi codebase
    video               |-> Make a video using *.png files in the current directory
    video_crop          |-> Crop video in the same directory
    video_scale         |-> Scale video resolution in the same directory
    video_speed         |-> Speed up video in the same directory

Taichi CLI

positional arguments:
  command     command from the above list to run

optional arguments:
  -h, --help  show this help message and exit
>>> Running time: 0.00s
PASSED

If you have local commits (e.g. compile fixes before you reproduce the bug), please make sure you first make a PR to fix the build errors and then report the bug.

[archibate]

Maybe we should use runpy instead of ad-hoc sys.executable?