Adding Julia extension instructions to template

[snairdesai]

The purpose of this issue (#76) is to provide additional user documentation in README.md on our Julia extension, which was integrated within gentzkow/template in #67. I noticed when pulling these changes to our selective-exposure project that the README.md file does not currently have instructions on running Julia scripts or initializing the Julia environment within miniconda. This process differs slightly from those we use for other softwares, because Julia is not yet fully integrated within conda-forge. I will shortly post a proposed revision to the README.md file describing this procedure.

cc @jc-cisneros @gentzkow

[snairdesai]

I propose we edit the following sections of the README.md file (new additions are in bold):

---

Requirements

Note: The application requirements and setup instructions outlined below are intended to serve general users. To build the repository as-is, the following applications are required:

• R
• Stata
• Python
• git
• git lfs
• LyX
• A TeX distribution for your local OS (for example, MacTeX for MacOS).

You may download the latest versions of each. By default, the Setup instructions below will assume their usage. Note that some of these applications must also be invocable from the command line. See the Command Line Usage section for details on how to set this up. Note that if you wish to run Julia scripts in your repository, you will additionally need to install Julia and set up its command line usage. Julia is currently not required to build the repository as-is.

You must set up a personal GitHub account to clone private repositories on which you are a collaborator. For public repositories (such as template), Git will suffice. You may need to set up Homebrew if git and git-lfs are not available on your local computer.

If you are using MacOS, ensure your terminal is operating in bash rather than the default zsh. MacOS users who are running template on an Apple Silicon chip will instead want to use Rosetta as their default terminal. You can find instructions on how to shift from zsh to Rosetta here and here. WindowsOS users (with Version 10 or higher) will need to switch to bash from PowerShell. To do this, you can run bash from within a PowerShell terminal (you must have installed git first).

Once you have met these OS and application requirements, clone a team repository from GitHub and proceed to Setup.

---

Adding Packages

Note: These instructions are relevant for users who have installed miniconda. If you have not done so, consult /setup/dependencies.md.

Python

Add any required packages to /setup/conda_env.yaml. If possible add the package version number as well. If there is a package that is not available from conda, add this to the pip section of the yaml file. In order to not re-run the entire environment setup you can download these individual files from conda with the command:

conda install -c conda-forge --name <environment name> <package_name=version_number>

R

Add any required packages that are available via CRAN to /setup/conda_env.yaml. These must be prepended with r-. If there is a package that is only available from GitHub and not from CRAN, add this package to /setup/setup_r.r (after copying this script from /extensions). These individual packages can be added in the same way as Python packages above (with the r- prepend).

Stata

Install Stata dependencies using /setup/download_stata_ado.do (copy download_stata_ado.do from /extensions to /setup first). We keep all non-base Stata ado files in the lib subdirectory, so most non-base Stata ado files will be versioned. To add additional Stata dependencies, use the following bash command from the setup subdirectory:

stata-mp -e download_stata_ado.do

Julia

First, add any required Julia packages to julia_conda_env.jl. Follow the same steps described in Setup to build and activate your conda environment, being sure to uncomment the line referencing julia in /setup/conda_env.yaml before building the environment. Once the environment is activated, run the following in terminal:

julia julia_conda_env.jl

---

Command Line Usage

For instructions on how to set up command line usage, refer to the repo wiki.

By default, the repository assumes these executable names for the following applications:

application : executable

python      : python
git-lfs     : git-lfs
lyx         : lyx
r           : Rscript
stata       : stata-mp (this will need to be updated if using a version of Stata that is not Stata-MP)
julia       : julia

Default executable names can be updated in config_user.yaml. For further details, see the User Configuration section.

---

Windows Differences

The only addition here should be updating the WindowsOS executable name for julia, but we should check this by running on @jc-cisneros's Windows to ensure everything compiles as expected.

[snairdesai]

Additional flags for @jc-cisneros and I to complete, in addition to the proposed README.md revisions above:

• Add Julia as an additional executable to /setup/config_user_template.yaml. This is relevant only for non-conda users, but it makes sense to add in this script regardless.
• Revise the Command Line Interface section of the Wiki to describe Julia CLI setup for those not using conda.
• Ensure the Julia extension works in the absence of a conda environment. This will likely require minor edits to ~/setup/dependencies.md (i.e., to provide instructions for manual installations of Julia packages - in this case, simply by running julia julia_conda_env.jl in terminal) and ~/setup/check_setup.py (i.e., to ensure CLI for Julia is enabled for users without an active conda environments).
• Ensure the Julia setup works on WindowsOS.
• In a separate issue, we may consider reproducing template to run directly using Julia rather than python (i.e., adding an extensions\julia subfolder similar to what currently exists for STATA and R. If we choose to do this at some point, we would also need to revise Running Package Scripts in Other Languages in the README.md.

We can either accomplish the above tasks here, or create a new issue.

cc @jc-cisneros

[jc-cisneros]

@snairdesai thanks for this! The additions make sense. The only one I would change is Note that these packages must be available via conda-forge. My understanding if the user chooses to use miniconda, then we are using two package managers in the following way:

• When the conda environment is built, it will contain an installation of Julia with no additional packages.
• When running julia julia_conda_env.jl, we are using Julia Pkg package manager. These packages get installed in the Julia directory connected to the Julia installation the user called in the command. In other words, if the conda environment is activated, these packages will get installed in the conda directory and not affect a local Julia installation.

The reason we are keeping track of the integration of Julia packages with Conda is that in the process above we are solving dependencies separately for Python/R packages and Julia packages. This will not solve dependencies or harmonize the paths across Python/R/Julia (this gave us issues when using a matplotlib backend previously).

[snairdesai]

Thanks @jc-cisneros! Got it -- makes sense. I'll remove this text from the README.md. Any preference on whether we should tackle the above items directly in this issue, or move them to elsewhere to handle at a later time?

[jc-cisneros]

@snairdesai Regarding the steps outlined in #76 (comment), I would split them into separate issues that are handled in parallel to this one. In particular, I would suggest one issue dealing with using template with the user's own version of Julia, another one to report the results from any test with Windows OS, and a last one (if considered necessary) adding a version of template that is run with Julia. All of these parallel issues would feed input to make changes on the instructions, which is the scope of this issue.

[snairdesai]

Summary + Deliverables

In this issue (#76), we updated the README to include additional instructions for Julia users (see 48040d7 or here for a formatted version) and added instructions on manual package installations for non-conda users (see 4cfb3eb).

Following a discussion with @jc-cisneros, we determined that the additional action items referenced above should be addressed in separate issues, which we will open and resolve as more bandwidth arises. I will proceed to open a PR with @jc-cisneros as a reviewer.