Positron RPC timed out when loading complex objects like SummarizedExperiment objects

[ZhimingYe]

System details:

Positron and OS details:

Positron Version: 2024.08.0 (Universal) build 24 on macOS 14.5 (ARM, Apple M1 Chip)
Server Version: Downloaded from positron-reh-linux-x64-2024.08.0-24.tar.gz release file. (Ubuntu 22.04.3 LTS)
Other attached packages:
[1] zellkonverter_1.10.1 SeuratObject_4.1.3 Seurat_4.3.0.1

Interpreter details:

R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Describe the issue:

When loading complex data objects (like S4 object with many slots and spare matrix) via SSH remote connection, all auxiliary functions of Positron for R, such as preview, become unresponsive.

Supplement: See the following comment, once I supposed is a bug from SSH connection, but it can re-produced in local ubuntu desktop environment without ssh. So it is not produced by SSH connection. It looks like #3628, but SummarizedExperiment object is more complex.

Steps to reproduce the issue:

Dataset download URL: https://datasets.cellxgene.cziscience.com/283b05b7-e336-4fb4-a36e-cec0627c426b.h5ad

library(Seurat)

library(zellkonverter)

OriginDS0<-readH5AD("~/redoSC/283b05b7-e336-4fb4-a36e-cec0627c426b.h5ad",reader = "R")

Afther running these, when executing the next command with Command + Enter, there is no response, also, clicking on any object in the Variable pane also has no effect.

After multiple attempts, I suspect this issue might be due to the server and client waiting for a large data packet? (I'm not a professional, so I can only describe it this way—please bear with me). For instance, it takes a long time (much longer than running the same code in RStudio) before the next line can be executed, and str(OriginDS0) returns results after multiple requests. In some cases, after loading a file, clicking on a variable in the Variable pane occasionally results in an RPC error.

Example1: No response in both executing code and viewing variables

PositronTestCase.mp4

Example2: After a long wait (longer than RStudio takes to execute the same code), the commands submitted with Command + Enter are executed all at once.

I come from a biomedical background and haven't received formal training in computer science. Therefore, I can only express my problem in such a rough manner.

Expected or desired behavior:

In RStudio, the same code doesn't cause similar issues, and the preview variable window is very smooth (which might also rule out network connection problems).

RstudioTestCase.mp4

Were there any error messages in the UI, Output panel, or Developer Tools console?

Error expanding variable item: RPC timed out after 5seconds: f"jsonrpc":"2.0""method":"inspect","params":{"path":["OriginDSo"]}} Error (-32603)

[ZhimingYe]

Once I supposed is a bug from SSH connection, but it can re-produced in local ubuntu desktop environment without ssh. So it is not produced by SSH connection. This seem to related to #3628.

PositronUbuntuLocal.mp4

[jmcphers]

This seem to related to #3628.

Yes, I think it's the same issue!

[ZhimingYe]

@jmcphers Thank you for quick reply!
Another suggestion is that, complex object like this, str(object) returns

Formal class 'SingleCellExperiment' [package "SingleCellExperiment"] with 9 slots
  ..@ int_elementMetadata:Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. ..@ rownames       : NULL
  .. .. ..@ nrows          : int 26274
  .. .. ..@ elementType    : chr "ANY"
  .. .. ..@ elementMetadata: NULL
  .. .. ..@ metadata       : list()
  .. .. ..@ listData       :List of 1
  .. .. .. ..$ rowPairs:Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. .. ..@ nrows          : int 26274
  .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. .. ..@ listData       : Named list()
  ..@ int_colData        :Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. ..@ rownames       : NULL
  .. .. ..@ nrows          : int 14057
  .. .. ..@ elementType    : chr "ANY"
  .. .. ..@ elementMetadata: NULL
  .. .. ..@ metadata       : list()
  .. .. ..@ listData       :List of 3
  .. .. .. ..$ reducedDims:Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. .. ..@ nrows          : int 14057
  .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. .. ..@ listData       :List of 3
  .. .. .. .. .. .. ..$ X_pca   : num [1:14057, 1:50] 5.4 7.31 4.41 7.18 6.86 ...
  .. .. .. .. .. .. ..$ X_scANVI: num [1:14057, 1:50] 0.00981 0.01838 0.01236 0.00872 0.02385 ...
  .. .. .. .. .. .. ..$ X_umap  : num [1:14057, 1:2] 2.339 0.161 3.28 3.667 8.402 ...
  .. .. .. ..$ altExps    :Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. .. ..@ nrows          : int 14057
  .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. .. ..@ listData       : Named list()
  .. .. .. ..$ colPairs   :Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. .. ..@ nrows          : int 14057
  .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. .. ..@ listData       : Named list()
  ..@ int_metadata       :List of 1
  .. ..$ version:Classes 'package_version', 'numeric_version'  hidden list of 1
  .. .. ..$ : int [1:3] 1 24 0
  ..@ rowRanges          :Formal class 'CompressedGRangesList' [package "GenomicRanges"] with 5 slots
  .. .. ..@ unlistData     :Formal class 'GRanges' [package "GenomicRanges"] with 7 slots
  .. .. .. .. ..@ seqnames       :Formal class 'Rle' [package "S4Vectors"] with 4 slots
  .. .. .. .. .. .. ..@ values         : Factor w/ 0 levels: 
  .. .. .. .. .. .. ..@ lengths        : int(0) 
  .. .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. ..@ ranges         :Formal class 'IRanges' [package "IRanges"] with 6 slots
  .. .. .. .. .. .. ..@ start          : int(0) 
  .. .. .. .. .. .. ..@ width          : int(0) 
  .. .. .. .. .. .. ..@ NAMES          : NULL
  .. .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. ..@ strand         :Formal class 'Rle' [package "S4Vectors"] with 4 slots
  .. .. .. .. .. .. ..@ values         : Factor w/ 3 levels "+","-","*": 
  .. .. .. .. .. .. ..@ lengths        : int(0) 
  .. .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. ..@ seqinfo        :Formal class 'Seqinfo' [package "GenomeInfoDb"] with 4 slots
  .. .. .. .. .. .. ..@ seqnames   : chr(0) 
  .. .. .. .. .. .. ..@ seqlengths : int(0) 
  .. .. .. .. .. .. ..@ is_circular: logi(0) 
  .. .. .. .. .. .. ..@ genome     : chr(0) 
  .. .. .. .. ..@ elementMetadata:Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. .. .. ..@ nrows          : int 0
  .. .. .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. .. .. ..@ metadata       : list()
  .. .. .. .. .. .. ..@ listData       : Named list()
  .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. ..@ metadata       : list()
  .. .. ..@ elementMetadata:Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. .. .. ..@ rownames       : NULL
  .. .. .. .. ..@ nrows          : int 26274
  .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. ..@ metadata       : list()
  .. .. .. .. ..@ listData       :List of 12
  .. .. .. .. .. ..$ n_cells              : num [1:26274] 0 0 0 0 0 ...
  .. .. .. .. .. ..$ mt                   : logi [1:26274] FALSE FALSE FALSE FALSE FALSE FALSE ...
  .. .. .. .. .. ..$ n_cells_by_counts    : num [1:26274] 0 0 0 0 0 ...
  .. .. .. .. .. ..$ mean_counts          : num [1:26274] 0 0 0 0 0 ...
  .. .. .. .. .. ..$ pct_dropout_by_counts: num [1:26274] 0 0 0 0 0 ...
  .. .. .. .. .. ..$ total_counts         : num [1:26274] 0 0 0 0 0 0 2210 0 0 0 ...
  .. .. .. .. .. ..$ gene_name            : Factor w/ 12471 levels "0610010K14Rik",..: NA NA NA NA NA NA 1 NA NA NA ...
  .. .. .. .. .. ..$ feature_is_filtered  : logi [1:26274] TRUE TRUE TRUE TRUE TRUE TRUE ...
  .. .. .. .. .. ..$ feature_name         : Factor w/ 26274 levels "0610005C13Rik",..: 1 2 3 4 5 21463 6 7 17501 8 ...
  .. .. .. .. .. ..$ feature_reference    : Factor w/ 1 level "NCBITaxon:10090": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. .. .. ..$ feature_biotype      : Factor w/ 1 level "gene": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. .. .. ..$ feature_length       : Factor w/ 9365 levels "56","59","60",..: 3385 1955 837 2797 1227 4905 1726 378 1278 960 ...
  .. .. ..@ elementType    : chr "GRanges"
  .. .. ..@ metadata       : list()
  .. .. ..@ partitioning   :Formal class 'PartitioningByEnd' [package "IRanges"] with 5 slots
  .. .. .. .. ..@ end            : int [1:26274] 0 0 0 0 0 0 0 0 0 0 ...
  .. .. .. .. ..@ NAMES          : chr [1:26274] "ENSMUSG00000109644" "ENSMUSG00000108652" "ENSMUSG00000007777" "ENSMUSG00000086714" ...
  .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. ..@ metadata       : list()
  ..@ colData            :Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. ..@ rownames       : chr [1:14057] "167739" "235221" "237620" "245316" ...
  .. .. ..@ nrows          : int 14057
  .. .. ..@ elementType    : chr "ANY"
  .. .. ..@ elementMetadata: NULL
  .. .. ..@ metadata       : list()
  .. .. ..@ listData       :List of 32
  .. .. .. ..$ n_genes_by_counts                       : int [1:14057] 679 959 505 941 746 699 1214 1874 1938 640 ...
  .. .. .. ..$ total_counts                            : num [1:14057] 965 1343 781 1349 1211 ...
  .. .. .. ..$ pct_counts_mt                           : num [1:14057] 1.451 0.298 2.177 0.667 0.33 ...
  .. .. .. ..$ Conditions                              : Factor w/ 5 levels "Naive","PBS",..: 3 3 3 3 3 3 3 3 3 3 ...
  .. .. .. ..$ Source                                  : Factor w/ 5 levels "Jia","Lab","Liu",..: 3 3 3 3 3 3 3 3 3 3 ...
  .. .. .. ..$ Technology                              : Factor w/ 5 levels "10x","BD","DropSeq",..: 2 2 2 2 2 2 2 2 2 2 ...
  .. .. .. ..$ Strains                                 : Factor w/ 1 level "C57": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ Time                                    : Factor w/ 6 levels "0day","0.0417day",..: 6 6 6 6 6 6 6 6 6 6 ...
  .. .. .. ..$ subtype                                 : Factor w/ 13 levels "Injured","NF1",..: 12 12 12 12 12 12 12 12 12 12 ...
  .. .. .. ..$ total_counts_mt                         : num [1:14057] 14 4 17 9 4 6 6 16 142 33 ...
  .. .. .. ..$ CellTypes_scANVI_pred                   : Factor w/ 6 levels "NP","PEP1","PEP2",..: 2 3 6 6 4 6 1 3 3 6 ...
  .. .. .. ..$ donor_id                                : Factor w/ 34 levels "CSD_1.5h_male_rep1",..: 17 17 17 17 17 17 17 17 17 17 ...
  .. .. .. ..$ organism_ontology_term_id               : Factor w/ 1 level "NCBITaxon:10090": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ tissue_ontology_term_id                 : Factor w/ 1 level "UBERON:0001675": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ tissue_type                             : Factor w/ 1 level "tissue": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ assay_ontology_term_id                  : Factor w/ 5 levels "EFO:0008722",..: 3 3 3 3 3 3 3 3 3 3 ...
  .. .. .. ..$ disease_ontology_term_id                : Factor w/ 2 levels "MONDO:0005244",..: 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ cell_type_ontology_term_id              : Factor w/ 1 level "CL:4023169": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ self_reported_ethnicity_ontology_term_id: Factor w/ 1 level "na": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ development_stage_ontology_term_id      : Factor w/ 1 level "MmusDv:0000052": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ sex_ontology_term_id                    : Factor w/ 2 levels "PATO:0000383",..: 2 2 2 2 2 2 2 2 2 2 ...
  .. .. .. ..$ suspension_type                         : Factor w/ 2 levels "cell","nucleus": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ is_primary_data                         : logi [1:14057] FALSE FALSE FALSE FALSE FALSE FALSE ...
  .. .. .. ..$ cell_type                               : Factor w/ 1 level "trigeminal neuron": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ assay                                   : Factor w/ 5 levels "Drop-seq","10x 3' v3",..: 3 3 3 3 3 3 3 3 3 3 ...
  .. .. .. ..$ disease                                 : Factor w/ 2 levels "peripheral neuropathy",..: 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ organism                                : Factor w/ 1 level "Mus musculus": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ sex                                     : Factor w/ 2 levels "female","male": 2 2 2 2 2 2 2 2 2 2 ...
  .. .. .. ..$ tissue                                  : Factor w/ 1 level "trigeminal ganglion": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ self_reported_ethnicity                 : Factor w/ 1 level "na": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ development_stage                       : Factor w/ 1 level "8 weeks": 1 1 1 1 1 1 1 1 1 1 ...
  .. .. .. ..$ observation_joinid                      : chr [1:14057] "?1b=`fC9hd" "f83h_&FwH=" "6pFQJ4{=sw" "N((d0BK|M*" ...
  ..@ assays             :Formal class 'SimpleAssays' [package "SummarizedExperiment"] with 1 slot
  .. .. ..@ data:Formal class 'SimpleList' [package "S4Vectors"] with 4 slots
  .. .. .. .. ..@ listData       :List of 1
  .. .. .. .. .. ..$ X:Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  .. .. .. .. .. .. .. ..@ i       : int [1:22815349] 655 1361 1633 1826 1836 1881 1890 1923 1940 2013 ...
  .. .. .. .. .. .. .. ..@ p       : int [1:14058] 0 387 883 1153 1644 2065 2458 3067 4026 5116 ...
  .. .. .. .. .. .. .. ..@ Dim     : int [1:2] 26274 14057
  .. .. .. .. .. .. .. ..@ Dimnames:List of 2
  .. .. .. .. .. .. .. .. ..$ : NULL
  .. .. .. .. .. .. .. .. ..$ : NULL
  .. .. .. .. .. .. .. ..@ x       : num [1:22815349] 1.13 1.65 1.13 1.13 1.13 ...
  .. .. .. .. .. .. .. ..@ factors : list()
  .. .. .. .. ..@ elementType    : chr "ANY"
  .. .. .. .. ..@ elementMetadata: NULL
  .. .. .. .. ..@ metadata       : list()
  ..@ NAMES              : NULL
  ..@ elementMetadata    :Formal class 'DFrame' [package "S4Vectors"] with 6 slots
  .. .. ..@ rownames       : NULL
  .. .. ..@ nrows          : int 26274
  .. .. ..@ elementType    : chr "ANY"
  .. .. ..@ elementMetadata: NULL
  .. .. ..@ metadata       : list()
  .. .. ..@ listData       : Named list()
  ..@ metadata           :List of 9
  .. ..$ Source_colors    : chr [1:5(1d)] "#1f77b4" "#ff7f0e" "#279e68" "#d62728" ...
  .. ..$ Time_colors      : chr [1:6(1d)] "#1f77b4" "#ff7f0e" "#279e68" "#d62728" ...
  .. ..$ batch_condition  : chr [1:4(1d)] "Source" "Conditions" "Time" "assay_ontology_term_id"
  .. ..$ citation         : chr "Dataset Version: https://datasets.cellxgene.cziscience.com/283b05b7-e336-4fb4-a36e-cec0627c426b.h5ad curated an"| __truncated__
  .. ..$ default_embedding: chr "X_umap"
  .. ..$ neighbors        :List of 3
  .. .. ..$ connectivities_key: chr "connectivities"
  .. .. ..$ distances_key     : chr "distances"
  .. .. ..$ params            :List of 6
  .. .. .. ..$ method      : chr "umap"
  .. .. .. ..$ metric      : chr "cosine"
  .. .. .. ..$ n_neighbors : int 50
  .. .. .. ..$ n_pcs       : int 50
  .. .. .. ..$ random_state: int 0
  .. .. .. ..$ use_rep     : chr "X_scANVI"
  .. ..$ schema_reference : chr "https://github.com/chanzuckerberg/single-cell-curation/blob/main/schema/5.1.0/schema.md"
  .. ..$ schema_version   : chr "5.1.0"
  .. ..$ title            : chr "iPain Atlas TG Nociceptive Lineage Cells - RNA"

These are all displayed in the variable pane, which is overwhelming for both the IDE and my eyes. RStudio allows these to be expanded into a new window. Could Positron also consider implementing a similar solution? Thank you for your work.

Additionally, I think this could also be a way to reduce the communication load between the ARK kernel and the Positron IDE frontend. For large and complex objects, Positron should only attempt to retrieve information from the backend and present it in a more refined manner when I actively request it by clicking, rather than displaying everything in the variable pane. Moreover, for small-screen devices like my MacBook Air, to be honest, a very small variable pane provides limited help.

like this:

[seeM]

Thanks for the report @ZhimingYe! I'll close this as a duplicate of #3628.

These are all displayed in the variable pane, which is overwhelming for both the IDE and my eyes. RStudio allows these to be expanded into a new window. Could Positron also consider implementing a similar solution? Thank you for your work.

That sounds like a reasonable enhancement request – would you mind creating a separate issue for this?