Module import fails when auto_reload is active

[subokita]

I have two piece of code (the structure has been simplified for clarity sake)
First in base.py

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# module: init0

from abc   import ABCMeta
from sanic import Sanic

class BaseService( metaclass = ABCMeta ):
    def create_app( self ) -> Sanic:
        app = Sanic( __name__ )
        return app

# app = BaseService().create_app()

# app.run( host  = '0.0.0.0',
#      port  = 5000,
#      debug = True,
#     )

print('[DONE]')

Second in run_test.py

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# module: init0

from init0.base import BaseService

def main():
    app = BaseService().create_app()
    app.run( host  = '0.0.0.0',
         port  = 5000,
         debug = True,
        )
    return

if __name__ == '__main__':
    main()

If I were to run python -m init0.run_test with debug = False then everything works perfectly, however if it's debug = True, then it'd throw me ModuleNotFoundError: No module named 'init0'

Is it some sort of loading error somewhere that I need to configure beforehand?

Thanks a lot in advance

[sjsadowski]

What does your __init__.py look like for that directory?

[subokita]

What does your __init__.py look like for that directory?

Completely empty. I updated my Sanic recently from 0.8 to 18.x along with my Python 3.6 to 3.7 or something, and somehow this issue appeared.

[yunstanford]

How does your project hierarchy look like ?

[subokita]

.
├── init0
│   ├── __init__.py
│   ├── __pycache__
│   ├── base.py
│   └── run_test.py
└── venv
    ├── bin
    ├── include
    └── lib

I just run from python -m init0.runtest

[yunstanford]

I don't think your python interpreter know your init0 module unless you've appended that to your sys.path

[subokita]

@yunstanford Hmm, but it works okay when it's not in debug mode, though.

[yunstanford]

ok, i think it's sth. wrong with the auto_reload logic..

You can set (workers > 1 or debug=false) for verifying, since the auto_reload will be disabled in such cases.

[subokita]

Yeap, by setting it to

app.run( host  = '0.0.0.0',
         port  = 5000,
         debug = True,
         workers = 2,
        )

the error no longer occurs.

[gangtao]

I hit the same error, adding workers >1 works for me as well!

[sjsadowski]

@huge-success/sanic-core-devs Anyone want to take a whack at this?

[ashleysommer]

@subokita @gangtao
Are you on Linux/Mac, or Windows?
The auto_reload logic is a bit different between Unix/Posix based OS, and Windows, so it would be good to know which we are dealing with before diving into debugging this.

[subokita]

@ashleysommer

OSX Mojave 10.14.4 Beta (18E194d)
Python     3.7.2
sanic      18.12.0
uvloop     0.12.1
gevent     1.4.0

[yunstanford]

To be clear, this issue is not because of windows.

When invoking python interpreter with -m option, the current directory will be added to the start of sys.path.

However, The reload_logic use https://github.com/huge-success/sanic/blob/master/sanic/reloader_helpers.py#L36-L57 to start worker process. That command line will look like python /path/to/start.py, because if invoking python interpreter with -m option, the first element of sys.argv will be the full path to the module file.

The reload implementation is not robust (and buggy), as mentioned before.

[yunstanford]

I'd also prefer to re-implement the reload logic as @vltr discussed here, #1346
And I personally use gunicorn for reloading in dev mode.

[ashleysommer]

I've written a basic workaround that fixes the issue described in this thread. #1501

This is a simple change to get bugs fixed in the current auto_reloader for the 2019.03 sanic release.
I agree it would be good to replace this current auto_reloader in the future to be more robust one, that will potentially be implemented as a plugin.