Make fn work like lambda and remove lambda (#1228)

* with-decorator: Allow a `setv` form as the form to be decorated

This feature is of dubious value by itself, but it's necessary to allow `defn` to create a lambda instead of a `def`.

* Make `fn` work the same as `lambda`

That is, allow it to generate a `lambda` instead of a `def` statement if the function body is just an expression.

I've removed two uses of with_decorator in hy.compiler because they'd require adding another case to HyASTCompiler.compile_decorate_expression and they have no ultimate effect, anyway.

In a few tests, I've added a meaningless statement in `fn` bodies to force generation of a `def`.

I've removed `test_fn_compiler_empty_function` rather than rewrite it because it seems like a pain to maintain and not very useful.

* Remove `lambda`, now that `fn` does the same thing

NEWS

@@ -3,9 +3,11 @@ Changes from 0.12.1
    [ Language Changes ]
    * `let` has been removed. Python's scoping rules do not make a proper
      implementation of it possible. Use `setv` instead.
+   * `lambda` has been removed, but `fn` now does exactly what `lambda` did
    * Added bytestring literals, which create `bytes` objects under Python 3
      and `str` objects under Python 2
    * Commas and underscores are allowed in numeric literals
+   * with-decorator: Allow a `setv` form as the form to be decorated
    * xor: If exactly one argument is true, return it
 
    [ Bug Fixes ]

docs/extra/anaphoric.rst

@@ -236,7 +236,7 @@ xi
 
 Usage ``(xi body ...)``
 
-Returns a function with parameters implicitly determined by the presence in the body of xi parameters. An xi symbol designates the ith parameter (1-based, e.g. x1, x2, x3, etc.), or all remaining parameters for xi itself. This is not a replacement for lambda. The xi forms cannot be nested.
+Returns a function with parameters implicitly determined by the presence in the body of xi parameters. An xi symbol designates the ith parameter (1-based, e.g. x1, x2, x3, etc.), or all remaining parameters for xi itself. This is not a replacement for fn. The xi forms cannot be nested.
 
 This is similar to Clojure's anonymous function literals (``#()``).
 

docs/language/api.rst

@@ -406,8 +406,8 @@ do
 
 ``do`` is used to evaluate each of its arguments and return the
 last one. Return values from every other than the last argument are discarded.
-It can be used in ``lambda`` or ``list-comp`` to perform more complex logic as
-shown in one of the following examples.
+It can be used in ``list-comp`` to perform more complex logic as shown in one
+of the following examples.
 
 Some example usage:
 
@@ -1116,13 +1116,15 @@ that ``import`` can be used.
     (import [sys [*]])
 
 
-lambda / fn
+fn
 -----------
 
-``lambda`` and ``fn`` can be used to define an anonymous function. The parameters are
-similar to ``defn``: the first parameter is vector of parameters and the rest is the
-body of the function. ``lambda`` returns a new function. In the following example, an
-anonymous function is defined and passed to another function for filtering output.
+``fn``, like Python's ``lambda``, can be used to define an anonymous function.
+Unlike Python's ``lambda``, the body of the function can comprise several
+statements. The parameters are similar to ``defn``: the first parameter is
+vector of parameters and the rest is the body of the function. ``fn`` returns a
+new function. In the following example, an anonymous function is defined and
+passed to another function for filtering output.
 
 .. code-block:: clj
 
@@ -1642,6 +1644,9 @@ will be 4 (``1+1 + 1+1``).
     => (addition 1 1)
     8
 
+In addition to ``defn`` forms, ``with-decorator`` can be used with ``defclass``
+and ``setv`` forms. In the latter case, the generated Python code uses an
+ordinary function call rather than decorator syntax.
 
 #@
 ~~

hy/cmdline.py

@@ -164,11 +164,11 @@ def ideas_macro():
 
 
 ;;; filtering a list w/ a lambda
-(filter (lambda [x] (= (% x 2) 0)) (range 0 10))
+(filter (fn [x] (= (% x 2) 0)) (range 0 10))
 
 
 ;;; swaggin' functional bits (Python rulez)
-(max (map (lambda [x] (len x)) ["hi" "my" "name" "is" "paul"]))
+(max (map (fn [x] (len x)) ["hi" "my" "name" "is" "paul"]))
 
 """)])
 

hy/compiler.py

@@ -1417,12 +1417,28 @@ def compile_assoc_expression(self, expr):
     def compile_decorate_expression(self, expr):
         expr.pop(0)  # with-decorator
         fn = self.compile(expr.pop(-1))
-        if not fn.stmts or not (isinstance(fn.stmts[-1], ast.FunctionDef) or
-                                isinstance(fn.stmts[-1], ast.ClassDef)):
+        if fn.stmts and isinstance(fn.stmts[-1], (ast.FunctionDef,
+                                                  ast.ClassDef)):
+            decorators, ret, _ = self._compile_collect(expr)
+            fn.stmts[-1].decorator_list = (decorators +
+                                           fn.stmts[-1].decorator_list)
+            return ret + fn
+        elif fn.stmts and isinstance(fn.stmts[-1], ast.Assign):
+            # E.g., (with-decorator foo (setv f (fn [] 5)))
+            # We can't use Python's decorator syntax, but we can get the
+            # same effect.
+            decorators, ret, _ = self._compile_collect(expr)
+            for d in decorators:
+                fn.stmts[-1].value = ast.Call(func=d,
+                                              args=[fn.stmts[-1].value],
+                                              keywords=[],
+                                              starargs=None,
+                                              kwargs=None,
+                                              lineno=expr.start_line,
+                                              col_offset=expr.start_column)
+            return fn
+        else:
             raise HyTypeError(expr, "Decorated a non-function")
-        decorators, ret, _ = self._compile_collect(expr)
-        fn.stmts[-1].decorator_list = decorators + fn.stmts[-1].decorator_list
-        return ret + fn
 
     @builds("with*")
     @checkargs(min=2)
@@ -2285,17 +2301,15 @@ def compile_set(self, expression):
                             col_offset=expression.start_column)
         return ret
 
-    @builds("lambda")
     @builds("fn")
     @checkargs(min=1)
     def compile_function_def(self, expression):
-        called_as = expression.pop(0)
+        expression.pop(0)
 
         arglist = expression.pop(0)
         if not isinstance(arglist, HyList):
             raise HyTypeError(expression,
-                              "First argument to `{}' must be a list".format(
-                                  called_as))
+                              "First argument to `fn' must be a list")
 
         (ret, args, defaults, stararg,
          kwonlyargs, kwonlydefaults, kwargs) = self._parse_lambda_list(arglist)
@@ -2367,7 +2381,7 @@ def compile_function_def(self, expression):
             defaults=defaults)
 
         body = self._compile_branch(expression)
-        if not body.stmts and called_as == "lambda":
+        if not body.stmts:
             ret += ast.Lambda(
                 lineno=expression.start_line,
                 col_offset=expression.start_column,
@@ -2513,7 +2527,6 @@ def compile_macro(self, expression):
             if kw in expression[0]:
                 raise HyTypeError(name, "macros cannot use %s" % kw)
         new_expression = HyExpression([
-            HySymbol("with_decorator"),
             HyExpression([HySymbol("hy.macros.macro"), name]),
             HyExpression([HySymbol("fn")] + expression),
         ]).replace(expression)
@@ -2536,7 +2549,6 @@ def compile_reader(self, expression):
                                "for reader macro name" % type(name).__name__))
         name = HyString(name).replace(name)
         new_expression = HyExpression([
-            HySymbol("with_decorator"),
             HyExpression([HySymbol("hy.macros.reader"), name]),
             HyExpression([HySymbol("fn")] + expression),
         ]).replace(expression)

hy/core/shadow.hy

@@ -65,7 +65,7 @@
   "Helper for shadow comparison operators"
   (if (< (len args) 2)
     (raise (TypeError "Need at least 2 arguments to compare"))
-    (reduce (lambda [x y] (and x y))
+    (reduce (fn [x y] (and x y))
             (list-comp (op x y)
                        [(, x y) (zip args (cut args 1))]))))
 (defn < [&rest args]

hy/extra/anaphoric.hy

@@ -135,9 +135,9 @@
   "Returns a function with parameters implicitly determined by the presence in
    the body of xi parameters. An xi symbol designates the ith parameter
    (1-based, e.g. x1, x2, x3, etc.), or all remaining parameters for xi itself.
-   This is not a replacement for lambda. The xi forms cannot be nested. "
+   This is not a replacement for fn. The xi forms cannot be nested. "
   (setv flatbody (flatten body))
-  `(lambda [;; generate all xi symbols up to the maximum found in body
+  `(fn [;; generate all xi symbols up to the maximum found in body
             ~@(genexpr (HySymbol (+ "x"
                                     (str i)))
                        [i (range 1

tests/compilers/test_ast.py

@@ -225,13 +225,13 @@ def test_ast_bad_defclass():
 
 def test_ast_good_lambda():
     "Make sure AST can compile valid lambda"
-    can_compile("(lambda [])")
-    can_compile("(lambda [] 1)")
+    can_compile("(fn [])")
+    can_compile("(fn [] 1)")
 
 
 def test_ast_bad_lambda():
     "Make sure AST can't compile invalid lambda"
-    cant_compile("(lambda)")
+    cant_compile("(fn)")
 
 
 def test_ast_good_yield():
@@ -369,9 +369,11 @@ def test_ast_expression_basics():
 
 def test_ast_anon_fns_basics():
     """ Ensure anon fns work. """
-    code = can_compile("(fn (x) (* x x))").body[0]
+    code = can_compile("(fn (x) (* x x))").body[0].value
+    assert type(code) == ast.Lambda
+    code = can_compile("(fn (x) (print \"multiform\") (* x x))").body[0]
     assert type(code) == ast.FunctionDef
-    code = can_compile("(fn (x))").body[0]
+    can_compile("(fn (x))")
     cant_compile("(fn)")
 
 
@@ -430,7 +432,7 @@ def test_lambda_list_keywords_kwargs():
 def test_lambda_list_keywords_kwonly():
     """Ensure we can compile functions with &kwonly if we're on Python
     3, or fail with an informative message on Python 2."""
-    kwonly_demo = "(fn [&kwonly a [b 2]] (print a b))"
+    kwonly_demo = "(fn [&kwonly a [b 2]] (print 1) (print a b))"
     if PY3:
         code = can_compile(kwonly_demo)
         for i, kwonlyarg_name in enumerate(('a', 'b')):

tests/compilers/test_compiler.py

@@ -57,24 +57,6 @@ def _make_expression(*args):
     def setUp(self):
         self.c = compiler.HyASTCompiler('test')
 
-    def test_fn_compiler_empty_function(self):
-        ret = self.c.compile_function_def(
-            self._make_expression(HySymbol("fn"), HyList()))
-        self.assertEqual(ret.imports, {})
-
-        self.assertEqual(len(ret.stmts), 1)
-        stmt = ret.stmts[0]
-        self.assertIsInstance(stmt, ast.FunctionDef)
-        self.assertIsInstance(stmt.args, ast.arguments)
-        self.assertEqual(stmt.args.vararg, None)
-        self.assertEqual(stmt.args.kwarg, None)
-        self.assertEqual(stmt.args.defaults, [])
-        self.assertEqual(stmt.decorator_list, [])
-        self.assertEqual(len(stmt.body), 1)
-        self.assertIsInstance(stmt.body[0], ast.Pass)
-
-        self.assertIsInstance(ret.expr, ast.Name)
-
     def test_compiler_bare_names(self):
         """
         Check that the compiler doesn't drop bare names from code branches

tests/importer/test_importer.py

@@ -12,7 +12,8 @@ def test_basics():
 
 def test_stringer():
     "Make sure the basics of the importer work"
-    _ast = import_buffer_to_ast("(defn square [x] (* x x))", '')
+    _ast = import_buffer_to_ast(
+        "(defn square [x] (print \"hello\") (* x x))", '')
     assert type(_ast.body[0]) == ast.FunctionDef
 
 

tests/native_tests/language.hy

@@ -75,7 +75,7 @@
        (except [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
   (try (eval '(defn get [] (print "hello")))
        (except [e [TypeError]] (assert (in "Can't assign to a builtin" (str e)))))
-  (try (eval '(defn lambda [] (print "hello")))
+  (try (eval '(defn fn [] (print "hello")))
        (except [e [TypeError]] (assert (in "Can't assign to a builtin" (str e))))))
 
 
@@ -133,10 +133,6 @@
 
 (defn test-alias-names-in-errors []
   "NATIVE: tests that native aliases show the correct names in errors"
-  (try (eval '(lambda))
-       (except [e [Exception]] (assert (in "lambda" (str e)))))
-  (try (eval '(fn))
-       (except [e [Exception]] (assert (in "fn" (str e)))))
   (try (eval '(setv 1 2 3))
        (except [e [Exception]] (assert (in "setv" (str e)))))
   (try (eval '(def 1 2 3))
@@ -343,11 +339,11 @@
              "level")))
 
 
-(defn test-lambda []
-  "NATIVE: test lambda operator"
-  (setv square (lambda [x] (* x x)))
+(defn test-fn []
+  "NATIVE: test fn operator"
+  (setv square (fn [x] (* x x)))
   (assert (= 4 (square 2)))
-  (setv lambda_list (lambda [test &rest args] (, test args)))
+  (setv lambda_list (fn [test &rest args] (, test args)))
   (assert (= (, 1 (, 2 3)) (lambda_list 1 2 3))))
 
 
@@ -1424,7 +1420,7 @@
 (defmacro identify-keywords [&rest elts]
   `(list
     (map
-     (lambda (x) (if (is-keyword x) "keyword" "other"))
+     (fn (x) (if (is-keyword x) "keyword" "other"))
      ~elts)))
 
 (defn test-keywords-and-macros []

tests/native_tests/with_decorator.hy

@@ -1,19 +1,40 @@
-(defn foodec [func]
-  (lambda [] (+ 1 1)))
+(defn test-decorated-1line-function []
+  (defn foodec [func]
+    (fn [] (+ (func) 1)))
+  (with-decorator foodec
+    (defn tfunction []
+      (* 2 2)))
+  (assert (= (tfunction) 5)))
 
 
-(with-decorator foodec
-  (defn tfunction []
-    (* 2 2)))
+(defn test-decorated-multiline-function []
+  (defn bazdec [func]
+    (fn [] (+ (func) "x")))
+  (with-decorator bazdec
+    (defn f []
+      (setv intermediate "i")
+      (+ intermediate "b")))
+  (assert (= (f) "ibx")))
 
 
-(defn bardec [cls]
-  (setv cls.my_attr 123)
-  cls)
+(defn test-decorated-class []
+  (defn bardec [cls]
+    (setv cls.attr2 456)
+    cls)
+  (with-decorator bardec
+    (defclass cls []
+      [attr1 123]))
+  (assert (= cls.attr1 123))
+  (assert (= cls.attr2 456)))
+
+
+(defn test-decorated-setv []
+  (defn d [func]
+    (fn [] (+ (func) "z")))
+  (with-decorator d
+    (setv f (fn [] "hello")))
+  (assert (= (f) "helloz")))
 
-(with-decorator bardec
-  (defclass cls []
-    [my_attr 456]))
 
 (defn test-decorator-clobbing []
   "NATIVE: Tests whether nested decorators work"
@@ -24,8 +45,3 @@
       (with-decorator dec2
         (defn f [] 1)))
     (assert (= (f) 4))))
-
-(defn test-decorators []
-  "NATIVE: test decorators."
-  (assert (= (tfunction) 2))
-  (assert (= cls.my_attr 123)))