store the actual source code of a method

[astrieanna]

Julia has a lot of useful REPL things like methods and help.
However, I still end up needing to just look at the code to see what it's doing.
It would be cool to be able to do:

julia> methods(base)
# methods for generic function base
base(base::Integer,n::Integer,pad::Integer) at intfuncs.jl:290
base(symbols::Array{Uint8,N},n::Integer,p::Integer) at intfuncs.jl:291
base(base_or_symbols::Union(Integer,Array{Uint8,N}),n::Integer) at intfuncs.jl:292

julia> implementation(base,1)
base(base::Integer, n::Integer, pad::Integer) = _base(dig_syms,int(base),unsigned(abs(n)),pad,n<0)

julia> implementation(base,3)
base(base_or_symbols::Union(Integer,Array{Uint8}), n::Integer) = base(base_or_symbols, n, 1)

It's kind of like the jump-to code stuff in some IDEs, but in the REPL, and only showing the one function.
This obviously has limitations, since you can't just scroll up and see the implementation of _base or search for the definition of dig_syms in that file, but it does let you see what the default values are.

You can already see the signature for _base, which makes the implementations of base more meaningful. (without needing to switch from the REPL to a text editor)

julia> methods(Base._base)
# methods for generic function _base
_base(symbols::Array{Uint8,N},b::Int32,x::Unsigned,pad::Int32,neg::Bool) at intfuncs.jl:278

Considering that the line numbers/files are already included in the output of methods, it seems like it should be straightforward to grab the appropriate lines of code from the file.

[StefanKarpinski]

If we stored the source (compressed) of each method definition when running interactively, this could be done rather easily and work correctly even when source files change and even when source line annotation isn't quite perfect. That would also help with the #265 (see also this discussion) since you could use the source to recompile things. We could also store the AST in compressed form instead – six vs. half a dozen.

[JeffBezanson]

No, it won't help with #265. We already have all the information, it just doesn't look like source code anymore. If you want to look at the original code, the best way is to read it from the file.

[milktrader]

Agree with @JeffBezanson and this would be a departure from R where typing the function without parens barfs out the source code. If the source is more than a few lines long, it becomes unusable (there is no way to scroll through the output afaik).

[wlbksy]

+1 with read from file.
Better fix #2594 , it really kills windows users when start is a native cmd command and notepad cannot highlight syntax nor show/jump to line number.
edit is really a useful function, if we could fix it/make it better.

[malmaud]

Just to understand, is the feeling here that there shouldn't be a function to echo the source code of methods, but rather to rely on 'edit' for that purpose? That might not play well with the IPython web notebook that's interacting with a remote Julia kernel, since I think the file will open on the kernel's machine instead of the client's.

[StefanKarpinski]

For what it's worth, in interactive modes, I'd like to keep source code around, rather than rely on what's in files. I want to be able to see the source code of things that were input via the repl too. See also #3988.

[hayd]

pager support #6921 may negate some concerns on the usefulness of this.

I find this very convenient in ipython (via func??), although - unlike could be the case of julia - some functions are hidden (e.g. built-in/cython functions).

[mbauman]

Also, @less func(x) does almost exactly what @astrieanna asks for (with paging), but it's dependent upon an external $PAGER.

[ivarne]

@less func(x) does not work for interactive functions, but that seems to be a problem that should be rephrased in a different issue.

[hgkamath]

If the following two features were available

• getting pre-lowered AST code i.e. source from the repl or at runtime
• redefining/clearing variables/types/functions/types

then it may be possible to do run time code listing/traversal/manipulation/generation.

julia> q=:( function a(i::Int) ; return i+4 ; end ; b=4 ; println(a(b))  )
quote 
    function a(i::Int) # none, line 1:
        return i + 4
    end
    begin 
        b = 4
        println(a(b))
    end
end

julia> function exprdescend(ex) ; if (isa(ex,Expr)) ; println("Descending Expr:",ex) ; println("head:",ex.head); println("args:",ex.args) ; println("type:",ex.typ)  ; for i in ex.args ; exprdescend(i) ; end ; else ; println("*:",typeof(ex),":",ex)  ; end  ;  end
// # ''try it ... long output''

in response to comment : JeffBezanson commented on Mar 20, 2013
"No, it won't help with #265. We already have all the information, it just doesn't look like source code anymore. If you want to look at the original code, the best way is to read it from the file."

So you think this is possible ? What happens where there are include/require statements and other things ?

// # str = read_whole_file_into_a_string(filename)
julia> str="for i in [1,2,3,4] ; println(i) ; end "
julia> s=parse(str)
:(for i = [1,2,3,4] # line 1:
        println(i)
    end)
julia> exprdescend(s)
Descending Expr:for i = [1,2,3,4] # line 1:
    println(i)
end
head:for
args:{:(i = [1,2,3,4]),quote  # line 1:
    println(i)
end}
type:Any
Descending Expr:i = [1,2,3,4]
head:=
args:{:i,:([1,2,3,4])}
type:Any
*:Symbol:i
Descending Expr:[1,2,3,4]
head:vcat
args:{1,2,3,4}
type:Any
*:Int64:1
*:Int64:2
*:Int64:3
*:Int64:4
Descending Expr:begin  # line 1:
    println(i)
end
head:block
args:{:( # line 1:),:(println(i))}
type:Any
*:LineNumberNode: # line 1:
Descending Expr:println(i)
head:call
args:{:println,:i}
type:Any
*:Symbol:println
*:Symbol:i

[ihnorton]

@hgkamath I'm afraid I don't understand the question, but it seems better-suited for the users mailing list. Please do read the metaprogramming and reflection sections of the manual.

[simonbyrne]

Another use case would be @generated functions (from https://groups.google.com/d/topic/julia-users/4pkWhcap1Zg/discussion).

[fredrikekre]

Oh, did I solve this in #22007 ?

[StefanKarpinski]

Sort of – I still think that in REPL mode we should stash the original source for functions for display.

[stevengj]

There is also the question of displaying anonymous functions, where it would be nice to be able to display the original AST.

julia> x -> x+1
(::#5) (generic function with 1 method)

is not super useful. (See also discourse.)

[vtjnash]

in REPL mode

we already basically do, it's just annoying to access:

let h = Base.active_repl.interface.modes[1].hist,
    replno = match(r"REPL\[(\d+)\]", $filename)

    replno === nothing || h.history[h.start_idx + parse(Int, replno[1])]
end

[StefanKarpinski]

"annoying to access" == "not useful enough to be considered solved"

[jebej]

I admit that it would be useful. I frequently debug by modifying a function in and running different variations in different REPLs to compare them. Sometimes I lose track of which REPL corresponds to which variation. Printing the function that is currently defined would then be nice to have.

[Roger-luo]

A use case in Yao provided by https://github.com/MasonProtter/LegibleLambdas.jl

Every block has a argument of number of qubits, and we don't want to write it repeatedly, so it can be auto-curried when you don't feed this number, e.g

julia> using Yao

julia> control(2, 1=>X)
(n -> control(n, 2, 1 => X gate))

So the user will be aware of this is not a block, it need this number of qubits for further evaluation. Before we have LegibleLambdas, it is quite confusing with just a number like #42. This also happens to Flux when the Optimizers returns an lambda before.

But there are a lot corner cases that we can't support in LegibleLambdas, it would be nice, that we could directly get these information in REPL with the support from compiler instead extern package.

[StefanKarpinski]

It still seems perverse to me that we support this

julia> @code_native 1 + 2
	.section	__TEXT,__text,regular,pure_instructions
; ┌ @ int.jl:53 within `+'
	leaq	(%rdi,%rsi), %rax
	retq
; └
; ┌ @ int.jl:53 within `<invalid>'
	nopw	%cs:(%rax,%rax)
; └

and this

julia> @code_llvm 1 + 2

;  @ int.jl:53 within `+'
define i64 @"julia_+_13402"(i64, i64) {
top:
  %2 = add i64 %1, %0
  ret i64 %2
}

and this

julia> @code_typed 1 + 2
CodeInfo(
1 ─ %1 = Base.add_int(x, y)::Int64
└──      return %1
) => Int64

and this

julia> @code_lowered 1 + 2
CodeInfo(
1 ─ %1 = Base.add_int(x, y)
└──      return %1
)

but not this

julia> @code_source 1 + 2
ERROR: LoadError: UndefVarError: @code_source not defined
in expression starting at REPL[23]:1

We can introspect all different possible versions of a method except for the one that everyone using Julia knows how to read and write. I'm gonna mark this as "help wanted" to indicate that it would be a welcomed addition to the language to have an optional mode where we remember the source representation of a function. It could be turned on by default in interactive mode but turned off by default in non-interactive mode.

[ilyagr]

Similar functionality is now implemented by https://github.com/timholy/CodeTracking.jl, which is part of Revise.jl. I played with it a little, and while it is not perfect, it works more often than not. The documentation says it's much better when also using Revise.

This was suggested in this discussion.

[FelixBenning]

This might be excellent in combination with automatic differentiation (like https://github.com/FluxML/Zygote.jl), since you can then show the derivative as julia code

[vtjnash]

@less and @edit do this now, as well as packages like Debugger and Cthulhu

[stevengj]

@less and @edit do not work with anonymous functions or with functions defined interactively, only with functions defined in files.

[KristofferC]

https://github.com/timholy/CodeTracking.jl has some tools for this.