Migrate to calling igraph method

[TomKellyGenetics]

An implementation in the igraph C library is being developed. Once this can be called in https://github.com/igraph/r-igraph then this version can be used instead of calling https://github.com/vtraag/leidenalg in python from reticulate.

igraph/igraph#1259

This will improve performance for igraph objects and retain compatibility with code that calls leiden in R.

[TomKellyGenetics]

The community_leiden function has been implemented in python-igraph. Once it is available in r-igraph, it will be called as a dependency here. Dependency on reticulate can then be removed.

igraph/python-igraph#273
igraph/rigraph#346

This package will be retained only for compatibility with existing code (as the r-igraph function should serve the same purpose).

[SamGG]

Hi,
Any idea about leiden_community will be available in rigraph?
Best?

[TomKellyGenetics]

See the relevant issues in the "igraph" repository. I only know that @vtraag has been working on this for many months as adding leidenalg to igraph has been a large project that requires changes to the core igraph C libraries. My understanding is that is close but please be patient. Open-source developers are very busy and often maintaining projects that are no longer their main work.

This issue serves primarily to remind myself to update this package to reflect this changes when updates to igraph are available. This update cannot be submitted to CRAN until the igraph version of leiden_community is already on CRAN.

Originally, I considered migrating from calling python with reticulate to calling the C++ version with RCpp. Now that it's being implemented in r-igraph this will not be necessary to improve performance.

[SamGG]

I agree and will also wait for this.

[vtraag]

I just finished the implementation of the Leiden algorithm in the R interface, see PR igraph/rigraph#399. Feel free to pull that and test further.

The implementation in igraph is considerably faster than the leidenalg package, but it only supports undirected graphs and CPM/modularity (note that for CPM the direction is irrelevant anyway). Fancier stuff, such as bipartite clustering (as requested here) and multiplex clustering more generally, is not available from the igraph implementation. Weights are supported though, so that should not be a problem (cf. satijalab/seurat#2574).

[TomKellyGenetics]

Thanks for the update @vtraag, I'm sure this took a lot of effort to implement.

I will need to wait for the igraph::cluster_leiden to be available on CRAN before releasing anything that depends on it. If I have time, I will try testing the development version with your changes on GitHub.

That information is very helpful. In that case, we can migrate this package to call cluster_leiden for CPM and (for undirected) modularity quality functions. I will keep the other methods calling leidenalg in Python via reticulate to provide the additional options. I'd prefer to remove the strict python dependency altogether but in this case, it seems there is a benefit to keep it when these other methods are used.

To clarify, the issue on Seurat is about whether to change the default behaviours (which will disrupt a lot of code from novice users). I think their Louvain clustering function uses weights but the Leiden algorithm call I added originally did not (although this leiden R package now supports this).

[SamGG]

@vtraag Thanks for your work. I am trying to compile it under Win10. But right now it's not clear to me where I should get the cigraph and the rigraph codes. Could you tell me?

[vtraag]

@SamGG I think that compilation under Win 10 might be more challenging (I have not done that for the R interface yet). Under other platforms, I believe you should be able to pull it and compile it using devtools as follows:

remotes::install_github("igraph/rigraph#399")

which refers to PR igraph/rigraph#399.

[SamGG]

Thanks for your feedback. Yes, Win10 is not easy, but that's my workhorse. Hopefully, MSYS2 is quite helpful and ease a lot of tasks. No problem to compile cigraph, but I got this error when executing "make" in the rigraph folder.

$ cigraph/tools/stimulus.py \
>            -f cigraph/interfaces/functions.def \
>            -i tools/stimulus/auto.R.in \
>            -o R/auto.R \
>            -t tools/stimulus/types-R.def \
>            -l RR
Traceback (most recent call last):
  File "cigraph/tools/stimulus.py", line 1487, in <module>
    main()
  File "cigraph/tools/stimulus.py", line 201, in main
    cg=cl(functions, types)
  File "cigraph/tools/stimulus.py", line 335, in __init__
    CodeGenerator.__init__(self, func, types)
  File "cigraph/tools/stimulus.py", line 223, in __init__
    newtypes=parser.parse(ff)
  File "cigraph/tools/stimulus.py", line 118, in parse
    tok=lex.token()
  File "cigraph/tools/stimulus.py", line 77, in token
    while line[-1]=="\\":
IndexError: string index out of range

Any idea?

[vtraag]

@SamGG Can I propose to discuss this on the igraph community forum? This might be more generally useful to people, and otherwise it is only found in an (otherwise unrelated) issue.

[TomKellyGenetics]

Just a note here that igraph/rigraph#399 has been merged into the development branch of the 'igraph' R package. I will migrate to calling this where necessary. We should be able to test installing it off GitHub before it's on CRAN.

[TomKellyGenetics]

I've had some issues setting up the development version of igraph but I've managed to compile and install it now in R 4.0.2. The main issue was linking to libraries on Mac OS 10.15 Catalina. I've got a local install for testing the igraph functions and comparing the performance to calling python leidenalg with reticulate.

To clarify, I misunderstood before: "igraph_community_leiden" is the C file running Leiden within igraph. The R function "cluster_leiden" will be tested for calling Modularity or CPM cost functions when appropriate.

I think it is safe to assume a performance benefit here. A quick test run is promising!

    library("igraph")
    #> 
    #> Attaching package: 'igraph'
    #> The following objects are masked from 'package:stats':
    #> 
    #>     decompose, spectrum
    #> The following object is masked from 'package:base':
    #> 
    #>     union
    library("leiden")
    #> conda environment r-reticulate installed
    #> python modules igraph and leidenalg installed
    adj_mat <- matrix(round(runif(10000, 0, 1)), 100, 100)
    snn_graph <- as.undirected(graph_from_adjacency_matrix(adj_mat))
    system.time({
    igraph::membership(cluster_leiden(snn_graph, objective_function = "modularity"))
    })
    #>    user  system elapsed 
    #>   0.004   0.000   0.004
    system.time({
    leiden(snn_graph, partition_type = "ModularityVertexPartition")
    })
    #>    user  system elapsed 
    #>   0.523   0.005   0.529

<sup>Created on 2020-11-05 by the [reprex package](https://reprex.tidyverse.org) (v0.3.0)</sup>

[TomKellyGenetics]

"cluster_leiden" from igraph is now supported in the development version of the "leiden" R package. See the updated vignette for details: https://htmlpreview.github.io/?https://github.com/TomKellyGenetics/leiden/blob/dev/doc/benchmarking.html

Note this requires cloning igraph and installing from source. Once the development version of "igraph" is installed you can check this with.

library("igraph")
?cluster_leiden

If the help page for cluster_leiden is available, then you can build the vignettes here:

devtools::install(dependencies = FALSE, build_vignettes = FALSE)
devtools::build_vignettes(install = FALSE, dependencies = FALSE)

This will not be necessary when igraph is updated on CRAN. When it is, I will release leiden version 0.4.0 to support compatibility for it while continuing to have extended functions available in Python.

Further details from local testing as it may be hard to install with remote dependencies on igraph::cluster_leiden.

Successful install:

> devtools::install()
* installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
* installing *source* package ‘leiden’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (leiden)

Successful tests:

> devtools::test()
Loading leiden
python modules igraph and leidenalg installed
Testing leiden
✓ |  OK F W S | Context
✓ |   9       | running Leiden on a bipartite igraph object [0.5 s]                   
✓ |  20       | running Leiden on an igraph object [39.9 s]                           
✓ |   7       | running Leiden on an adjacency matrix [0.2 s]                         
✓ |   8       | running Leiden on an dense and sparse matrices [6.7 s]                
✓ |   9       | running Leiden on unweighted objects                                  
✓ |  12       | running Leiden on weighted objects [0.3 s]                            

══ Results ═══════════════════════════════════════════════════════════════════════════
Duration: 49.4 s

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 65 ]

Successful checks:

> devtools::check(document = FALSE, args = c('--as-cran'))

── Building ─────────────────────────────────────────────────────────────── leiden ──
Setting env vars:
● CFLAGS    : -Wall -pedantic -fdiagnostics-color=always
● CXXFLAGS  : -Wall -pedantic -fdiagnostics-color=always
● CXX11FLAGS: -Wall -pedantic -fdiagnostics-color=always
─────────────────────────────────────────────────────────────────────────────────────
✓  checking for file ‘/Users/tom/Downloads/packages/leiden/DESCRIPTION’ (401ms)
─  preparing ‘leiden’: (3.9s)
✓  checking DESCRIPTION meta-information ...
─  installing the package to build vignettes
✓  creating vignettes (3m 53.9s)
─  checking for LF line-endings in source and make files and shell scripts (803ms)
─  checking for empty or unneeded directories
   Removed empty directory ‘leiden/tests/testthat/_snaps’
─  building ‘leiden_0.4.0.9001.tar.gz’
   
── Checking ─────────────────────────────────────────────────────────────── leiden ──
Setting env vars:
● _R_CHECK_CRAN_INCOMING_USE_ASPELL_: TRUE
● _R_CHECK_CRAN_INCOMING_REMOTE_    : FALSE
● _R_CHECK_CRAN_INCOMING_           : FALSE
● _R_CHECK_FORCE_SUGGESTS_          : FALSE
● NOT_CRAN                          : true
── R CMD check ─────────────────────────────────────────────────────────────────
─  using log directory ‘/Users/tom/Downloads/packages/leiden.Rcheck’ (383ms)
─  using R version 4.0.2 (2020-06-22)
─  using platform: x86_64-apple-darwin17.0 (64-bit)
─  using session charset: UTF-8
─  using options ‘--no-manual --as-cran’
✓  checking for file ‘leiden/DESCRIPTION’
─  checking extension type ... Package
─  this is package ‘leiden’ version ‘0.4.0.9001’
─  package encoding: UTF-8
✓  checking package namespace information ...
✓  checking package dependencies (8.6s)
✓  checking if this is a source package
✓  checking if there is a namespace
✓  checking for executable files (514ms)
✓  checking for hidden files and directories ...
✓  checking for portable file names ...
✓  checking for sufficient/correct file permissions
✓  checking serialization versions
✓  checking whether package ‘leiden’ can be installed (22.2s)
✓  checking installed package size ...
✓  checking package directory ...
✓  checking for future file timestamps (1.3s)
✓  checking ‘build’ directory
✓  checking DESCRIPTION meta-information ...
✓  checking top-level files
✓  checking for left-over files ...
✓  checking index information (500ms)
✓  checking package subdirectories (521ms)
✓  checking R files for non-ASCII characters ...
✓  checking R files for syntax errors ...
✓  checking whether the package can be loaded (9.5s)
✓  checking whether the package can be loaded with stated dependencies (7.1s)
✓  checking whether the package can be unloaded cleanly (8.2s)
✓  checking whether the namespace can be loaded with stated dependencies (1.4s)
✓  checking whether the namespace can be unloaded cleanly (1.3s)
✓  checking dependencies in R code (7.6s)
✓  checking S3 generic/method consistency (7.8s)
✓  checking replacement functions (10.6s)
✓  checking foreign function calls (12.3s)
✓  checking R code for possible problems (28.2s)
✓  checking Rd files ...
✓  checking Rd metadata ...
✓  checking Rd line widths ...
✓  checking Rd cross-references ...
✓  checking for missing documentation entries (11.2s)
✓  checking for code/documentation mismatches (26.2s)
✓  checking Rd \usage sections (14.3s)
✓  checking Rd contents ...
✓  checking for unstated dependencies in examples ...
✓  checking installed files from ‘inst/doc’ ...
✓  checking files in ‘vignettes’ ...
✓  checking examples (11.2s)
✓  checking for unstated dependencies in ‘tests’ ...
─  checking tests (375ms)
✓  Running ‘spelling.R’
✓  Running ‘testthat.R’ [43s/63s] (1m 3s)
✓  checking for unstated dependencies in vignettes (1m 3.9s)
✓  checking package vignettes in ‘inst/doc’ ...
✓  checking re-building of vignette outputs (3m 6.3s)
✓  checking for non-standard things in the check directory
✓  checking for detritus in the temp directory ...
   
   
── R CMD check results ────────────────────────────────── leiden 0.4.0.9001 ────
Duration: 7m 24.4s

0 errors ✓ | 0 warnings ✓ | 0 notes ✓

R CMD check succeeded

[TomKellyGenetics]

@vtraag Great work on updating Python leidenalg and integrating leiden into igraph. Do you have any idea when the next version of R-igraph will be released on CRAN?

[vtraag]

We are working on it. Hopefully it will be released in a couple of weeks, but I can't make any promises.

[TomKellyGenetics]

@vtraag Note that you should be able to install the development version of R igraph using remotes::install_github("igraph/rigraph@master"), see also igraph/rigraph#423 (comment)

@vtraag If you could report those errors during installation/compilation in the rigraph repository, that would be great.

Ok a few minor issues with building igraph on Mac OS Catalina from source on the dev branch. I'm going to leave them here in case anyone else runs into them. I think they don't require changes to the repository with one exception (the version number). You're probably aware of the rest but I'll leave it for future reference if others need to do it (or if I run into issues again later).

Otherwise, it's working great. Thanks for developing a great package.

Multiplex graphs are now supported in both the "multiplex" branch (which depends not on the dev version of igraph) and the "dev" branch (which does). If I have time to write a vignette for multiplex graphs these will likely be releases 0.3.6 and 0.4.0 respectively on R (before and after igraph is updated).

1. I had to install the development igraph packages again because the development version is not updated. It is still version 1.2.5 so the 1.2.6 release on CRAN was installed instead when I updated other R packages. The version number of the version number in the DESCRIPTION needs to be updated to avoid this. You can do this by changing the Makefile.

-REALVERSION=1.2.5
-VERSION=1.2.5
+REALVERSION=1.2.6.9001
+VERSION=1.2.6.9001

None of the other changes were necessary to install the CRAN version 1.2.6. The version number here means that updating CRAN packages will not install another version when the development version is already installed (until version 1.2.7 or later which supports cluster_leiden is released. I'm not sure that incrementing the version number is worth reporting as an issue to r-igraph.

The remaining issues below are presumably problems with my set up to compile the C igraph core from source (rather than downloading pre-compiled libraries for Mac from CRAN).

I have similar errors on a Linux server but this also seems to be missing dependencies (I don’t have sudo access to install them). This is okay as I can test “cluster_leiden” on my Mac.

[TomKellyGenetics]

Further requirements to install the “dev” version.

2. calling make requires python 2. I used a conda environment to do this.

$ conda create -n py2 python=2.7
$ conda activate py2
$ conda list
# packages in environment at /Users/tom/Library/r-miniconda/envs/py2:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                haf1e3a3_3    conda-forge
c-ares                    1.16.1               haf1e3a3_3    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2019.11.28       py27h8c360ce_1    conda-forge
cmake                     3.18.3               hfc1b5b8_0    conda-forge
expat                     2.2.9                hb1e8313_2    conda-forge
krb5                      1.17.1               h75d18d8_3    conda-forge
libcurl                   7.71.1               h9bf37e3_8    conda-forge
libcxx                    11.0.0               h439d374_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libffi                    3.2.1             hb1e8313_1007    conda-forge
libllvm11                 11.0.0               hf85e3d2_0    conda-forge
libnghttp2                1.41.0               h7580e61_2    conda-forge
libssh2                   1.9.0                h8a08a2b_5    conda-forge
libuv                     1.40.0               h22f3db7_0    conda-forge
llvm                      11.0.0               h1341992_0    conda-forge
ncurses                   6.2                  hb1e8313_2    conda-forge
openssl                   1.1.1h               haf1e3a3_0    conda-forge
pip                       20.1.1             pyh9f0ad1d_0    conda-forge
python                    2.7.15          h8e446fc_1011_cpython    conda-forge
python_abi                2.7                     1_cp27m    conda-forge
readline                  8.0                  h0678c8f_2    conda-forge
rhash                     1.3.6             haf1e3a3_1001    conda-forge
setuptools                44.0.0                   py27_0    conda-forge
sqlite                    3.33.0               h960bd1c_1    conda-forge
tk                        8.6.10               hb0a8c7a_1    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
zlib                      1.2.11            h7795811_1010    conda-forge

If a build fails you can start over with this, removing compiled files and starting fresh.ca

git clean -fdx . 
./configure
make

Note: I can install from GitHub in this environment with remotes::install_github("igraph/rigraph@master") but this installs version 1.2.5 from master branch which doesn't have cluster_leiden.

> igraph::cluster_leiden
Error: 'cluster_leiden' is not an exported object from 'namespace:igraph'

Installing the dev branch doesn't work from GitHub. I think it needs to be cloned and run make to compile the C functions first.

> remotes::install_github("igraph/rigraph@dev")
Using github PAT from envvar GITHUB_PAT
Error: Failed to install 'unknown package' from GitHub:
  HTTP error 404.
  Not Found

4. The following environment variables are required to be in the environment (defined here in ~/.R/Makevars). Some of this may be superfluous but it is a working solution. Make sure not to run devtools::install in a conda environment as this will cause issues. GLPK and GMP libraries do not need to be installed (actually adding these to the PATH breaks the installation). The libgfortran.4.dylib or libgfortran.4.so file needs to be renamed or removed if libgfortran.5.dylib or libgfortran.4.so is available.

You can locate them with:

find /usr/local/ -name libgfortran.4.dylib
find $HOME -name libgfortran.4.dylib 

Here is my ~/.R/Makevars file.

FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin18/8.2.0/ -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm

F77=/usr/local/bin/gfortran
FC=/usr/local/bin/gfortran
FLIBS="-L/usr/local/bin/gfortran"

CC=clang
export CXX=clang++

CPPFLAGS="-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -isystem /usr/local/include -isystem  /Users/tom/Downloads/packages/rigraph/cigraph/optional/simpleraytracer -isystem /Users/tom/Downloads/packages/rigraph/cigraph/optional/glpk -I/usr/local/opt/flex/include -I/usr/local/opt/bison/include -I/usr/local/opt/glpk/include -L/usr/local/opt/flex/lib -I/usr/local/opt/bison/lib -L/usr/local/opt/glpk/lib -I/usr/local/include -L/usr/local/lib -I/usr/local/Cellar/gmp/6.2.0/include -L/usr/local/Cellar/gmp/6.2.0/lib -I/opt/local/include -L/opt/local/lib -I/usr/local/opt/icu4c/include -L/usr/local/opt/icu4c/lib -I/usr/local/include/ -L/usr/local/lib"
CFLAGS="-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -isystem /usr/local/include -isystem  /Users/tom/Downloads/packages/rigraph/cigraph/optional/simpleraytracer -isystem /Users/tom/Downloads/packages/rigraph/cigraph/optional/glpk -I/usr/local/opt/flex/include -I/usr/local/opt/bison/include -I/usr/local/opt/glpk/include -L/usr/local/opt/flex/lib -I/usr/local/opt/bison/lib -L/usr/local/opt/glpk/lib -I/usr/local/include -L/usr/local/lib -I/usr/local/Cellar/gmp/6.2.0/include -L/usr/local/Cellar/gmp/6.2.0/lib -I/opt/local/include -L/opt/local/lib -I/usr/local/opt/icu4c/include -L/usr/local/opt/icu4c/lib"  

CPLUS_INCLUDE_PATH=/usr/local/opt/llvm/include/c++/v1:/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include     

CPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include  

DYLD_FALLBACK_LIBRARY_PATH=/Users/tom/Library/r-miniconda/lib

I also installed flex gmp bison glpk llvm and libgfortran with "brew" and add these to my /.bashrc file.

export PATH="/usr/local/opt:$PATH" 
export PATH="/usr/local/opt/llvm/bin:$PATH"

export LIBRARY_PATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib:/usr/local/lib'
export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib      

export PATH="/usr/local/opt/flex/bin:$PATH"
export PATH="/usr/local/opt/bison/bin:$PATH"

export LDFLAGS="-L/usr/local/opt/flex/lib -L/usr/local/opt/bison/lib -L/usr/local/opt/glpk/lib -L/usr/local/opt/gmp/lib"

Here is a summary of the environment env that builds successfully on Mac OS 10.15 Catalina.

CONDA_BACKUP_DEBUG_FFLAGS=-march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-tracking-assignments
CONDA_BACKUP_DEBUG_FORTRANFLAGS=-march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe -Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-tracking-assignments
CONDA_BACKUP_FFLAGS=-march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe
CONDA_BACKUP_FORTRANFLAGS=-march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe
CONDA_BACKUP_GFORTRAN=/Users/tom/Library/r-miniconda/envs/py2/bin/x86_64-apple-darwin13.4.0-gfortran
CONDA_BACKUP_HOST=x86_64-apple-darwin13.4.0
CONDA_DEFAULT_ENV=base
CONDA_EXE=/Users/tom/Library/r-miniconda/bin/conda
CONDA_PREFIX=/Users/tom/Library/r-miniconda
CONDA_PROMPT_MODIFIER=(base) 
CONDA_PYTHON_EXE=/Users/tom/Library/r-miniconda/bin/python
CONDA_SHLVL=1
CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
CPL_ZIP_ENCODING=UTF-8
DISPLAY=/private/tmp/com.apple.launchd.RK0vw9KBUS/org.macosforge.xquartz:0
F77=/usr/local/bin/gfortran
FC=/usr/local/bin/gfortran
FLIBS=-L/usr/local/bin/gfortran
GDAL_DATA=/Users/tom/Library/r-miniconda/share/gdal
GOPATH=/Users/tom/.go
GOROOT=/usr/local/opt/go/libexec
GSETTINGS_SCHEMA_DIR=/Users/tom/Library/r-miniconda/share/glib-2.0/schemas
GSETTINGS_SCHEMA_DIR_CONDA_BACKUP=
HOME=/Users/tom
LANG=en_NZ.UTF-8
LC_CTYPE=UTF-8
LESS=-R
LOGNAME=tom
LSCOLORS=Gxfxcxdxbxegedabagacad
OLDPWD=/Users/tom/Downloads/packages/rigraph/cigraph/src
PAGER=less
PATH=/Users/tom/Library/r-miniconda/bin:/Users/tom/Library/r-miniconda/bin:/usr/local/bin:/usr/local/opt/flex/bin:/usr/local/opt/bison/bin:/Users/tom/Library/r-miniconda/bin:/Users/tom/Library/r-miniconda/condabin:/usr/local/opt/[email protected]/bin:/usr/local/opt/[email protected]/bin:/Users/tom/anaconda3/bin:/Users/tom/.cargo/bin:/usr/local/opt/ruby/bin:/.gem/ruby/2.6.0/bin:/usr/local/opt/llvm/bin:/usr/local/opt/llvm/bin:/usr/local/opt/gnu-sed/libexec/gnubin:/Users/tom/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/tom/.go/bin:/usr/local/opt/go/libexec/bin
PROJ_LIB=/Users/tom/Library/r-miniconda/share/proj
PROJ_NETWORK=ON
PWD=/Users/tom/Downloads/packages/rigraph
SHELL=/bin/zsh
SHLVL=1
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.UCJKaiHpfA/Listeners
TERM=xterm-256color
TERM_PROGRAM=Apple_Terminal
TERM_PROGRAM_VERSION=433
TERM_SESSION_ID=FD67F203-6B6A-45EF-A210-01253052BFC6
TMPDIR=/var/folders/2r/qnb884952pqf5f36bk2g99rr0000gp/T/
USER=tom
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
ZSH=/Users/tom/.oh-my-zsh
_=/usr/bin/ENV
_CE_CONDA=
_CE_M=

New versions of roxygen2 may return errors. A workaround is using older versions and calling pkgbuild::compile_dll.
r-lib/roxygen2#822

I found this wasn't necessary with the latest version if the header, libraries, and dlls for dependencies were found.

Successful install

git submodule init
git submodule update
./configure 

config.log

git clean -fdx .
make

make.log

R -e "devtools::document(); devtools::install(); devtools::test()"

install.log

Successful tests

Attaching package: ‘igraph’

The following objects are masked from ‘package:stats’:

    decompose, spectrum

The following object is masked from ‘package:base’:

    union

Loading igraph
Testing igraph
✔ |  OK F W S | Context
✔ |  20       | Constructor modifiers [0.5 s]                                                                                                                                                               
✔ |  14       | Graph ids [0.9 s]                                                                                                                                                                           
✔ |   2       | VS/ES indexing [0.1 s]                                                                                                                                                                      
✔ |  26       | New isomorphism API [0.4 s]                                                                                                                                                                 
✔ |  18       | Make API                                                                                                                                                                                    
✔ |   8       | make_graph                                                                                                                                                                                  
✔ |   6       | New layout API                                                                                                                                                                              
✔ |   9       | Notable graphs                                                                                                                                                                              
✔ |   2       | Old data type and VS/ES                                                                                                                                                                     
✔ |   4       | Random walks                                                                                                                                                                                
✔ |   1       | igraph_version                                                                                                                                                                              
✔ |   3       | Data version and conversion                                                                                                                                                                 
✔ |  10       | Detailed printing of vs and es [0.2 s]                                                                                                                                                      
✔ |   4       | Vertex and edge sequence quirks                                                                                                                                                             
✔ |  50     1 | Vertex and edge sequences [1.4 s]                                                                                                                                                           
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Skip (test-vs-es.R:97:1): we can use vs/es with broken refs
Reason: empty test
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔ | 227       | VS/ES operators [7.9 s]                                                                                                                                                                     
✔ |  17       | Weak references [1.1 s]                                                                                                                                                                     
✔ |   8       | add_edges [0.2 s]                                                                                                                                                                           
✔ |   5       | add_vertices                                                                                                                                                                                
✔ |  78       | adjacency spectral embedding [4.5 s]                                                                                                                                                        
✔ |   7       | all.st.cuts                                                                                                                                                                                 
✔ |  14       | alpha_centrality [0.2 s]                                                                                                                                                                    
✔ |   9       | are_adjacent                                                                                                                                                                                
✔ |  12       | arpack [0.1 s]                                                                                                                                                                              
✔ |   1       | articulation_points                                                                                                                                                                         
✔ |  10       | as.directed                                                                                                                                                                                 
✔ |   6       | as.undirected                                                                                                                                                                               
✔ |   6       | assortativity [0.4 s]                                                                                                                                                                       
✔ |  34       | attributes                                                                                                                                                                                  
✔ | 406       | authority_score [0.8 s]                                                                                                                                                                     
✔ |   4       | mean_distance                                                                                                                                                                               
✔ |  21       | sample_pa [0.1 s]                                                                                                                                                                           
✔ |   6       | betweenness                                                                                                                                                                                 
✔ |   5       | biconnected_components                                                                                                                                                                      
✔ |  27       | bipartite_projection [0.2 s]                                                                                                                                                                
✔ |  27       | sample_bipartite [0.2 s]                                                                                                                                                                    
✔ |   5       | Bonacich's power centrality [0.5 s]                                                                                                                                                         
✔ |   1       | Bug 1019624                                                                                                                                                                                 
✔ |   1       | Bug 1032819                                                                                                                                                                                 
✔ |   1       | Bug 1033045                                                                                                                                                                                 
✔ |   5       | Bug 1073705 [0.1 s]                                                                                                                                                                         
✔ |   1       | Bug 1073800                                                                                                                                                                                 
✔ |   1       | canonical_permutation                                                                                                                                                                       
✔ |   5       | cliques [0.1 s]                                                                                                                                                                             
✔ |   5       | closeness                                                                                                                                                                                   
✔ |   6       | components                                                                                                                                                                                  
✔ |  36     1 | communities [0.8 s]                                                                                                                                                                         
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Skip (test_communities.R:72:3): communities function works
Reason: No GLPK library
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔ |   2       | constraint                                                                                                                                                                                  
✔ |   3       | contract                                                                                                                                                                                    
✔ |  11       | Correlated E-R random graphs                                                                                                                                                                
✔ |   7       | count_multiple                                                                                                                                                                              
✔ |   9       | decompose [5.8 s]                                                                                                                                                                           
✔ |   7       | degree                                                                                                                                                                                      
✔ |   7       | sample_degseq [0.5 s]                                                                                                                                                                       
✔ |   1       | delete_edges                                                                                                                                                                                
✔ |   1       | delete_vertices                                                                                                                                                                             
✔ |   5       | diameter                                                                                                                                                                                    
✔ |  10       | Dimensionality selection                                                                                                                                                                    
✔ |   3       | dominator_tree                                                                                                                                                                              
✔ |   8       | Dot-product random graphs                                                                                                                                                                   
✔ |   4       | dyad_census                                                                                                                                                                                 
✔ |   2       | edge_betweenness                                                                                                                                                                            
✔ |   8       | cluster_edge_betweenness                                                                                                                                                                    
✔ |   4       | edge_connectivity                                                                                                                                                                           
✔ |   4       | edge names                                                                                                                                                                                  
✔ | 1001       | eigen_centrality [2.8 s]                                                                                                                                                                   
✔ |   3       | farthest_vertices                                                                                                                                                                           
✔ |   8       | cluster_fast_greedy                                                                                                                                                                         
✔ |   1       | sample_forestfire [1.5 s]                                                                                                                                                                   
✔ |   4       | as_adj [0.1 s]                                                                                                                                                                              
✔ | 302       | as_adj_list [0.4 s]                                                                                                                                                                         
✔ |   4       | all_shortest_paths                                                                                                                                                                          
✔ |   4       | get_diameter                                                                                                                                                                                
✔ |   1       | ends                                                                                                                                                                                        
✔ |   1       | edgelist                                                                                                                                                                                    
✔ |   6       | as_incidence_matrix                                                                                                                                                                         
✔ |   2       | shortest_paths                                                                                                                                                                              
✔ |   4       | girth                                                                                                                                                                                       
✔ |   6       | adhesion                                                                                                                                                                                    
✔ |  17       | graph.adjancency [0.1 s]                                                                                                                                                                    
✔ |   2       | graph_from_adj_list [0.1 s]                                                                                                                                                                 
✔ |   2       | graph.atlas                                                                                                                                                                                 
✔ |   5       | BFS                                                                                                                                                                                         
✔ |   1       | make_bipartite_graph                                                                                                                                                                        
✔ |   1       | complementer                                                                                                                                                                                
✔ |   2       | compose                                                                                                                                                                                     
✔ |   1       | coreness                                                                                                                                                                                    
✔ |   3       | graph_from_data_frame                                                                                                                                                                       
✔ |   2       | make_de_bruijn_graph                                                                                                                                                                        
✔ |   2       | edge_density                                                                                                                                                                                
✔ |   3       | graph_from_edgelist                                                                                                                                                                         
✔ |   4       | Eigenproblems                                                                                                                                                                               
✔ |   4       | graph_from_literal                                                                                                                                                                          
✔ |   5       | isomorphism_class                                                                                                                                                                           
✔ |   4       | make_kautz_graph                                                                                                                                                                            
✔ |  10       | knn                                                                                                                                                                                         
✔ |   5       | max_flow                                                                                                                                                                                    
✔ |   4       | min_cut                                                                                                                                                                                     
✔ | 203       | graph.subisomorphic, lad [1.3 s]                                                                                                                                                            
✔ |   6       | graph.subisomorphic.vf2                                                                                                                                                                     
✔ |   0     1 | graphNEL conversion                                                                                                                                                                         
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Skip (test_graphNEL.R:6:3): graphNEL conversion works
Reason: No graph package
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔ |   9       | Graphlets [0.3 s]                                                                                                                                                                           
✔ |   1       | Hierarchical random graphs [0.3 s]                                                                                                                                                          
✔ |  21       | Hierarchical stochastic block models [0.1 s]                                                                                                                                                
✔ |   8       | igraph_options                                                                                                                                                                              
✔ |   1       | ivs                                                                                                                                                                                         
✔ |  73       | Indexing [0.3 s]                                                                                                                                                                            
✔ |  19       | Assignments via indexing                                                                                                                                                                    
✔ |   1       | Indexing                                                                                                                                                                                    
✔ |   2       | is_bipartite                                                                                                                                                                                
✔ |   6       | is_chordal                                                                                                                                                                                  
✔ |  19       | iterators                                                                                                                                                                                   
✔ |   4       | cluster_label_prop                                                                                                                                                                          
✔ | 108       | Spectral embedding of the Laplacian [0.2 s]                                                                                                                                                 
✔ |   1       | largest_cliques                                                                                                                                                                             
✔ |   2       | largest_ivs                                                                                                                                                                                 
✔ |   2       | Fruchterman-Reingold layout                                                                                                                                                                 
✔ |   4       | Kamada-Kawai layouts                                                                                                                                                                        
✔ |  12       | layout_with_mds [6.0 s]                                                                                                                                                                     
✔ |  13       | merge_coords [3.6 s]                                                                                                                                                                        
✔ | 125       | cluster_leading_eigen [1.5 s]                                                                                                                                                               
✔ |   6       | cluster_leiden                                                                                                                                                                              
✔ |   3       | Maximal cliques [5.7 s]                                                                                                                                                                     
✔ |   1       | min_st_separators                                                                                                                                                                           
✔ |   1       | min_separators                                                                                                                                                                              
✔ |   2       | modularity_matrix                                                                                                                                                                           
✔ |   8       | motifs [0.6 s]                                                                                                                                                                              
✔ |   4       | cluster_louvain                                                                                                                                                                             
✔ |  15       | ego                                                                                                                                                                                         
✔ | 100       | neighbors [0.2 s]                                                                                                                                                                           
✔ |  13       | operators                                                                                                                                                                                   
✔ |   8       | infix operators                                                                                                                                                                             
✔ |  38       | operators on named graphs [0.2 s]                                                                                                                                                           
✔ |   0     2 | cluster_optimal                                                                                                                                                                             
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Skip (test_optimal.community.R:6:3): cluster_optimal works
Reason: No GLPK library

Skip (test_optimal.community.R:27:3): weighted cluster_optimal works
Reason: No GLPK library
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔ |   1       | Pajek file format                                                                                                                                                                           
✔ |  16       | print.igraph [0.2 s]                                                                                                                                                                        
✔ |   3       | Prefix sum tree                                                                                                                                                                             
✔ |  10       | Various samplers                                                                                                                                                                            
✔ |   4       | Stochastic block models                                                                                                                                                                     
✔ |  32       | Local scan statistics [24.6 s]                                                                                                                                                              
✔ |  12       | sdf                                                                                                                                                                                         
✔ |   7       | Seeded graph matching [0.3 s]                                                                                                                                                               
✔ |   1       | SIR epidemics model on a network                                                                                                                                                            
✔ |   8       | Sampling points from a sphere [0.1 s]                                                                                                                                                       
✔ |   5       | transitivity [0.1 s]                                                                                                                                                                        
✔ |  11       | Triangles                                                                                                                                                                                   
✔ |   2       | unfold_tree                                                                                                                                                                                 
✔ |   8       | cluster_walktrap                                                                                                                                                                            
✔ |  50       | sample_smallworld [1.4 s]                                                                                                                                                                   

══ Results ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 83.0 s

── Skipped tests  ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
● No GLPK library (3)
● No graph package (1)
● empty test (1)

[ FAIL 0 | WARN 0 | SKIP 5 | PASS 3703 ]
> 
> 

Successful test of dev branch of leiden commit 16f51586a967c0f23193ad1598032e9cbd5ffb91

==> devtools::test()

Loading leiden
! Adding files missing in collate: /Users/tom/Downloads/packages/leiden/R/py_objects.R
conda environment r-reticulate installed
python modules igraph and leidenalg installed
Testing leiden
✓ |  OK F W S | Context
✓ |   9       | running Leiden on a bipartite igraph object [0.3 s]                
✓ |  20       | running Leiden on an igraph object [32.7 s]                        
✓ |   7       | running Leiden on an adjacency matrix [0.2 s]                      
✓ |   8       | running Leiden on an dense and sparse matrices [5.2 s]             
✓ |   6       | running Leiden on multiplex igraph objects [0.6 s]                 
✓ |   9       | running Leiden on unweighted objects [0.1 s]                       
✓ |  12       | running Leiden on weighted objects [0.3 s]                         

══ Results ════════════════════════════════════════════════════════════════════════
Duration: 40.2 s

Tests dev version of igraph:

Testing commit 4b036df

==> devtools::test()

Loading leiden
! Adding files missing in collate: /Users/tom/Downloads/packages/leiden/R/py_objects.R
conda environment r-reticulate installed
python modules igraph and leidenalg installed
Testing leiden
✓ |  OK F W S | Context
✓ |   9       | running Leiden on a bipartite igraph object [0.3 s]                
✓ |  20       | running Leiden on an igraph object [32.7 s]                        
✓ |   7       | running Leiden on an adjacency matrix [0.2 s]                      
✓ |   8       | running Leiden on an dense and sparse matrices [5.2 s]             
✓ |   6       | running Leiden on multiplex igraph objects [0.6 s]                 
✓ |   9       | running Leiden on unweighted objects [0.1 s]                       
✓ |  12       | running Leiden on weighted objects [0.3 s]                         

══ Results ════════════════════════════════════════════════════════════════════════
Duration: 40.2 s

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 71 ]
��

[vtraag]

Thanks for the extensive reporting @TomKellyGenetics!

The version number of the version number in the DESCRIPTION needs to be updated

Right, good point, I will look into this.

I can install from GitHub in this environment with remotes::install_github("igraph/rigraph@master") but this installs version 1.2.5 from master branch which doesn't have cluster_leiden

This is rather odd. Indeed, the version is incorrect (as you already pointed out), but it does have cluster_leiden when I install it. Are you sure your installation wasn't overwritten by the CRAN package, or that you somehow were still using another version? Could you perhaps double check this? If this does work correctly, a number of the other points can be ignored (e.g. python).

Installing the dev branch doesn't work from GitHub.

There is some confusion about the master branch and the dev branch. The master branch is automatically generated from the dev branch. That is, we run all steps to build a source package (i.e. make and friends), and upload that to the master branch, so that you can install it using remotes::install_github. The reason you need to specify master is because dev is the default branch, which is nowadays being picked up by install_github. We should document this more clearly.

[TomKellyGenetics]

Indeed, the version is incorrect (as you already pointed out), but it does have cluster_leiden when I install it.

I may have misunderstood which branch to use. I tried both but neither worked at the time. In hindsight the version number may have thrown me off (and I already had version 1.2.5 installed from CRAN so I assumed the new version was needed).

I've checked here and you're right that both branches now support cluster_leiden. Thanks for explaining the difference. I first tried to install it a few weeks ago when there were unmerged commits on dev so I might have mistaken they were necessary (looks like it's for CI).

https://github.com/igraph/rigraph/blob/dev/R/community.R
https://github.com/igraph/rigraph/blob/master/R/community.R

Presumably now that I've fixed the dependencies to compile on Mac, it would install with devtools from remote as well as the local directory.

[TomKellyGenetics]

I've forked the remote and increased the version number for testing (as it won't install a lower version). I can install master branch from GitHub on our Linux server.

It successfully increases the version number and I can call the help on the function which displays. It won't run the function though which is odd since I can see it in the NAMESPACE.

I had to run devtools::document() locally to install it on Mac and this is probably why I cloned the repo in the first place (and also to debug the missing dependencies).

** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (igraph)
> 
> library("igraph")
> igraph::cluster_leiden()
Error: 'cluster_leiden' is not an exported object from 'namespace:igraph'
> ?igraph::cluster_leiden
> igraph::cluster_leiden(graph)
Error: 'cluster_leiden' is not an exported object from 'namespace:igraph'
> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS:   /opt/local/R-3.6.1/lib/R/lib/libRblas.so
LAPACK: /opt/local/R-3.6.1/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] igraph_1.2.6.9001 leiden_0.3.1     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5        compiler_3.6.1    prettyunits_1.1.1 remotes_2.2.0    
 [5] tools_3.6.1       testthat_2.3.2    digest_0.6.27     pkgbuild_1.1.0   
 [9] pkgload_1.1.0     jsonlite_1.7.1    memoise_1.1.0     lattice_0.20-38  
[13] pkgconfig_2.0.3   rlang_0.4.8       Matrix_1.2-18     cli_2.0.2        
[17] curl_4.3          withr_2.3.0       desc_1.2.0        fs_1.3.1         
[21] devtools_2.3.2    rprojroot_1.3-2   grid_3.6.1        reticulate_1.13  
[25] glue_1.4.2        R6_2.5.0          processx_3.4.1    fansi_0.4.1      
[29] sessioninfo_1.1.1 callr_3.4.4       magrittr_2.0.1    backports_1.1.10 
[33] ps_1.4.0          ellipsis_0.3.1    usethis_1.6.3     assertthat_0.2.1 
[37] crayon_1.3.4     

I'm running R 4.0.2 on Mac and R 3.6.1 on Linux but I think it works on either version. Anyway I've got it under control on my Mac for now and I think the precompiled release on CRAN wouldn't be affected by this issue.

[TomKellyGenetics]

This may be way the CRAN version 1.2.6 of igraph was installed instead. When building another package, depedencies are installed. Here it recognises thath the development version is installed (as I've increased the version number of igraph from 1.2.5 to 1.2.6.9001).

> devtools::document()
Updating graphsim documentation
ℹ Loading graphsim
Writing NAMESPACE
Writing NAMESPACE
> devtools::install(build_vignettes = TRUE)
Skipping 1 packages not available: paxtoolsr
Skipping 1 packages ahead of CRAN: igraph
✓  checking for file ‘/Users/tom/Downloads/packages/graphsim/DESCRIPTION’ (391ms)
─  preparing ‘graphsim’: (6s)
✓  checking DESCRIPTION meta-information ...

[vtraag]

A new development version is now available that uses correct version numbering, you should be able to install it using remotes::install_github("igraph/rigraph@master"). The current version at the time of writing is 1.2.6.9100. This version will automatically be increased with each additional commit that is made. It passes CI on Windows, macOS and Linux for the R 4.0 (i.e. release), and also passes on Linux for R-devel and R 3.4, 3.5 and 3.6 (the other OS are only tested on R-release).

[vtraag]

R igraph 1.2.7 was just released, containing the Leiden algorithm implementation.

[TomKellyGenetics]

This version is now on CRAN with release: https://github.com/TomKellyGenetics/leiden/releases/tag/0.4.2