Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Molsen/add pex root option #206

Merged
merged 13 commits into from
Apr 28, 2016
15 changes: 11 additions & 4 deletions pex/bin/pex.py
Original file line number Diff line number Diff line change
Expand Up @@ -503,11 +503,18 @@ def make_relative_to_root(path):
return os.path.normpath(path.format(pex_root=ENV.PEX_ROOT))


def main():
def main(args=None, log_callback=None):
if log_callback is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not crazy about this bit, how about just:

def main(args, log=log, exiter=sys.exit):
...
  exiter(pex.run(...))
...
if __name__ == '__main__':
  main(sys.argv[1:])

and then eliminating all of the if log_callback & if args checks + the raise SystemExit in favor of just passing in test-context values for these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not put mutable objects in as the default options, which is why I went with None. As far as the sys.argv[1:] being placed in main I'd rather not put that logic somewhere it can't be easily tested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand your statements or reasoning here.

"I'd rather not put mutable objects in as the default options, which is why I went with None." -> what default option here is mutable? log is a function defined above. sys.exit is also a function. if anything, the whole 'log can be either be a global or local depending on the value of log_callback' strikes me as most peculiar.

"As far as the sys.argv[1:] being placed in main I'd rather not put that logic somewhere it can't be easily tested." -> somehow I feel like the slicing of sys.argv isn't a super critical thing to test? also, by precedent injection of args is actually a super common pattern in twitter.common.app. what's the theoretical harm here in your mind? or why not alternatively main(sys.argv)?

I'm a firm -1 on avoiding needless clutter in main() for tests and particularly not a fan of this approach as-is - would prefer that you either clean this up in-line with the suggestions or find an alternate route.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mutable object was in ref to sys.argv. I generally think that the slice should still be in testable code it would make a difference for example if you were to change option frameworks to docopt or argparse. However, that being said its not super critical and I've moved it.

Also removed the log=None default.

Spoke with you offline and you generally seem okay with SystemExit approach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking through this, updating log in main() doesn't actually help since several other methods are using log as a global as well. I'll update the code later with an alternative that allows you to update the logger for all methods.

log = log_callback

parser, resolver_options_builder = configure_clp()

# split arguments early because optparse is dumb
args = sys.argv[1:]
if args is None:
# split arguments early because optparse is dumb
args = sys.argv[1:]
else:
args = args[:]

try:
separator = args.index('--')
args, cmdline = args[:separator], args[separator + 1:]
Expand Down Expand Up @@ -542,7 +549,7 @@ def main():

log('Running PEX file at %s with args %s' % (pex_builder.path(), cmdline), v=options.verbosity)
pex = PEX(pex_builder.path(), interpreter=pex_builder.interpreter)
sys.exit(pex.run(args=list(cmdline)))
raise SystemExit(pex.run(args=list(cmdline)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change still feels weird to me - and I think to others it will also. since the two are fundamentally identical, why not just stick with the more intuitive/common form of sys.exit?

fwiw, you shouldn't need to explicitly raise here to be able to handle SystemExit in your tests:

>>> try:
...     sys.exit(1)
... except SystemExit:
...     print('not exiting!')
...     
... 
not exiting!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong objection. I'll update the code to use sys.exit rather than raise SystemExit.



if __name__ == '__main__':
Expand Down
38 changes: 38 additions & 0 deletions pex/testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@
import sys
import tempfile
import zipfile
from collections import namedtuple
from textwrap import dedent

from .bin.pex import main
from .common import safe_mkdir, safe_rmtree
from .compatibility import nested
from .installer import EggInstaller, Packager
Expand Down Expand Up @@ -173,6 +175,42 @@ def write_simple_pex(td, exe_contents, dists=None, coverage=False):
return pb


class IntegResults(namedtuple('results', 'output return_code exception')):
"""Convenience object to return integration run results."""

def assert_success(self):
assert self.exception is None and self.return_code is None

def assert_failure(self):
assert self.exception or self.return_code


def run_pex_command(args, env=None):
"""Simulate running pex command for integration testing.

This is different from run_simple_pex in that it calls the pex command rather
than running a generated pex. This is useful for testing end to end runs
with specific command line arguments or env options.
"""
def logger_callback(_output):
def mock_logger(msg, v=None):
_output.append(msg)

return mock_logger

exception = None
error_code = None
output = []

try:
main(args=args, log_callback=logger_callback(output))
except SystemExit as e:
error_code = e.code
except Exception as e:
exception = e
return IntegResults(output, error_code, exception)


# TODO(wickman) Why not PEX.run?
def run_simple_pex(pex, args=(), env=None):
po = subprocess.Popen(
Expand Down
19 changes: 18 additions & 1 deletion tests/test_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from twitter.common.contextutil import environment_as, temporary_dir

from pex.compatibility import WINDOWS
from pex.testing import run_simple_pex_test
from pex.testing import run_pex_command, run_simple_pex_test
from pex.util import named_temporary_file


Expand All @@ -23,6 +23,23 @@ def test_pex_raise():
run_simple_pex_test(body, coverage=True)


def test_pex_root():
with temporary_dir() as tmp_home:
with environment_as(HOME=tmp_home):
with temporary_dir() as td:
with temporary_dir() as output_dir:
env = os.environ.copy()
env['PEX_INTERPRETER'] = '1'

output_path = os.path.join(output_dir, 'pex.pex')
args = ['pex', '-o', output_path, '--not-zip-safe', '--pex-root={0}'.format(td)]
results = run_pex_command(args=args, env=env)
results.assert_success()
assert ['pex.pex'] == os.listdir(output_dir), 'Expected built pex file.'
assert [] == os.listdir(tmp_home), 'Expected empty temp home dir.'
assert 'build' in os.listdir(td), 'Expected build directory in tmp pex root.'


def test_pex_interpreter():
with named_temporary_file() as fp:
fp.write(b"print('Hello world')")
Expand Down