Convert CSV to Out object

[lrocax]

Hi,

Great package.

A question: I have a CSV that contain 4 columns (objectName,Date,Location,Coord) with several rows. How can I import the objectName and Coord as Out object?

Ex1.zip

Thanks

[vbonhomme]

Where does it come from and what have you tried ? Le dim. 17 mai 2020 17:20, Lluis Roca <[email protected]> a écrit :

…

Hi, Great package. A question: I have a CSV that contain 4 columns (objectName,Date,Location,Coord) with several rows. How can I import the objectName and Coord as Out object? Ex1.zip <https://github.com/MomX/Momocs/files/4640478/Ex1.zip> Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#211>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFWPIPFCLZYDIW2KS43RR76CZANCNFSM4NDNWFNA> .

[vbonhomme]

I now have a real keyboard ! The where does it come from is because of Momit, I'm trying to cover all possible sources for raw data. The what have you tried is because I believe in you ! Le dim. 17 mai 2020 à 19:53, Vincent Bonhomme <[email protected]> a écrit :

…

Where does it come from and what have you tried ? Le dim. 17 mai 2020 17:20, Lluis Roca ***@***.***> a écrit : > Hi, > > Great package. > > A question: I have a CSV that contain 4 columns > (objectName,Date,Location,Coord) with several rows. How can I import the > objectName and Coord as Out object? > > Ex1.zip <https://github.com/MomX/Momocs/files/4640478/Ex1.zip> > > Thanks > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#211>, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABECQFWPIPFCLZYDIW2KS43RR76CZANCNFSM4NDNWFNA> > . >

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[lrocax]

It is an output from a python code that I wrote that is structured as followed [x1 y1; x2 y2; …xn yn]. So, a general solution for CSV file will be awesome.

[petrpajdla]

Generally speaking, the structure you showed in your example is not a simple CSV file, it resembles file structures used with spatial data. I think you first need to read the coordinates and add the metadata to an object with coordinates in the next step...

[vbonhomme]

Hey there, I have got a bitt of code that does that (see below). Apparently this type of structure is not rare so I ll verse it into Momit. Try to survive with this, if you dont ring my bell ! Read your .csv first so that it enters the function as a data.frame. ``` # Dependencies library(tidyverse) library(Momocs) # a function that turns the csv with coordinates # encoded into columns into a proper Out with its fac from_csv_to_Out <- function(df){ df <- as_tibble(df) # cosmetics fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non coord columns -> our $fac coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord columns -> our $coo #split rowwise split(1:nrow(.)) %>% # for each row, turn columns into shape map(~.x %>% # select appropriate columns select(matches("X[[:digit:]]+(x|y)")) %>% # unlist and turn into a shape unlist() %>% na.omit() %>% # required because some missing data matrix(ncol=2, byrow=T)) # build and return the Out Out(coo, fac) } ``` Le lun. 18 mai 2020 à 09:01, Petr Pajdla <[email protected]> a écrit :

…

Generally speaking, the structure you showed in your example is not a simple CSV file, it resembles file structures used with spatial data. I think you first need to read the coordinates and add the metadata to an object with coordinates in the next step... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFWX3BCF7EFY5NX5YULRSDMMPANCNFSM4NDNWFNA> .

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[raz1]

I was running this code using R 4.0.5

library(tidyverse)
library(Momocs)
 
df <- read.csv(file = 'E:/Ex1.csv')
  
from_csv_to_Out <- function(df){
  df <- as_tibble(df) # cosmetics
  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
  coord columns -> our $fac
  coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord
    columns         -> our $coo
  #split rowwise
  split(1:nrow(.)) %>%
    # for each row, turn columns into shape
    map(~.x %>%
          # select appropriate columns
          select(matches("X[[:digit:]]+(x|y)")) %>%
          # unlist and turn into a shape
          unlist() %>%
          na.omit() %>% # required because some missing data
          matrix(ncol=2, byrow=T))
  # build  and return the Out
  Out(coo, fac)
}

and I got this output:

library(tidyverse)
-- Attaching packages ------------------------------------------ tidyverse 1.3.1 --
<U+221A> ggplot2 3.3.4     <U+221A> purrr   0.3.4
<U+221A> tibble  3.1.2     <U+221A> dplyr   1.0.6
<U+221A> tidyr   1.1.3     <U+221A> stringr 1.4.0
<U+221A> readr   1.4.0     <U+221A> forcats 0.5.1
-- Conflicts --------------------------------------------- tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()
> library(Momocs)

Attaching package: ‘Momocs’

The following objects are masked from ‘package:dplyr’:

   arrange, combine, filter, mutate, rename, sample_frac, sample_n,
   select, slice

The following object is masked from ‘package:tidyr’:

   chop

The following object is masked from ‘package:stats’:

   filter

> df <- read.csv(file = 'E:/Ex1.csv')
> from_csv_to_Out <- function(df){
+   df <- as_tibble(df) # cosmetics
+   fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
+   coord columns -> our $fac
Error: unexpected symbol in:
"  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
 coord columns"
>   coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord
+     columns         -> our $coo
Error in columns(.) : could not find function "columns"
>   #split rowwise
>   split(1:nrow(.)) %>%
+     # for each row, turn columns into shape
+     map(~.x %>%
+           # select appropriate columns
+           select(matches("X[[:digit:]]+(x|y)")) %>%
+           # unlist and turn into a shape
+           unlist() %>%
+           na.omit() %>% # required because some missing data
+           matrix(ncol=2, byrow=T))
Error in nrow(.) : object '.' not found
>   # build  and return the Out
>   Out(coo, fac)
Error in Out(coo, fac) : object 'coo' not found
> }
Error: unexpected '}' in "}"

Any ideas how to solve the problem?

[vbonhomme]

the two line with "->" shoulb be commented, ie: ``` from_csv_to_Out <- function(df){ df <- as_tibble(df) # cosmetics fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord #split rowwise split(1:nrow(.)) %>% # for each row, turn columns into shape map(~.x %>% # select appropriate columns select(matches("X[[:digit:]]+(x|y)")) %>% # unlist and turn into a shape unlist() %>% na.omit() %>% # required because some missing data matrix(ncol=2, byrow=T)) # build and return the Out Out(coo, fac) } ``` Le mer. 23 juin 2021 à 14:48, Ely Raz ***@***.***> a écrit :

…

I was running this code using R 4.0.5 library(tidyverse) library(Momocs) df <- read.csv(file = 'E:/Ex1.csv') from_csv_to_Out <- function(df){ df <- as_tibble(df) # cosmetics fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non coord columns -> our $fac coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord columns -> our $coo #split rowwise split(1:nrow(.)) %>% # for each row, turn columns into shape map(~.x %>% # select appropriate columns select(matches("X[[:digit:]]+(x|y)")) %>% # unlist and turn into a shape unlist() %>% na.omit() %>% # required because some missing data matrix(ncol=2, byrow=T)) # build and return the Out Out(coo, fac) } and I got this output: library(tidyverse) -- Attaching packages ------------------------------------------ tidyverse 1.3.1 -- <U+221A> ggplot2 3.3.4 <U+221A> purrr 0.3.4 <U+221A> tibble 3.1.2 <U+221A> dplyr 1.0.6 <U+221A> tidyr 1.1.3 <U+221A> stringr 1.4.0 <U+221A> readr 1.4.0 <U+221A> forcats 0.5.1 -- Conflicts --------------------------------------------- tidyverse_conflicts() -- x dplyr::filter() masks stats::filter() x dplyr::lag() masks stats::lag() > library(Momocs) Attaching package: ‘Momocs’ The following objects are masked from ‘package:dplyr’: arrange, combine, filter, mutate, rename, sample_frac, sample_n, select, slice The following object is masked from ‘package:tidyr’: chop The following object is masked from ‘package:stats’: filter > df <- read.csv(file = 'E:/Ex1.csv') > from_csv_to_Out <- function(df){ + df <- as_tibble(df) # cosmetics + fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non + coord columns -> our $fac Error: unexpected symbol in: " fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non coord columns" > coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord + columns -> our $coo Error in columns(.) : could not find function "columns" > #split rowwise > split(1:nrow(.)) %>% + # for each row, turn columns into shape + map(~.x %>% + # select appropriate columns + select(matches("X[[:digit:]]+(x|y)")) %>% + # unlist and turn into a shape + unlist() %>% + na.omit() %>% # required because some missing data + matrix(ncol=2, byrow=T)) Error in nrow(.) : object '.' not found > # build and return the Out > Out(coo, fac) Error in Out(coo, fac) : object 'coo' not found > } Error: unexpected '}' in "}" Any ideas how to solve the problem? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFS3PLX42EC3E3XCE6TTUHJ3JANCNFSM4NDNWFNA> .

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[raz1]

Thanks for the super fast respond.

I tried this line with this output:

> ex1 <- from_csv_to_Out(df)
 Error in matrix(., ncol = 2, byrow = T) : 
  'data' must be of a vector type, was 'NULL' 

I did something wrong?

[vbonhomme]

from_csv_to_Out expects a filepath I guess Le mer. 23 juin 2021 à 15:08, Ely Raz ***@***.***> a écrit :

…

Thanks for the super fast respond. I tried this line with this output: > ex1 <- from_csv_to_Out(df) Error in matrix(., ncol = 2, byrow = T) : 'data' must be of a vector type, was 'NULL' I did something wrong? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFRUILMLLGQEO5AP3O3TUHMDHANCNFSM4NDNWFNA> .

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[raz1]

The same error

> ex1 <- from_csv_to_Out('E:/Ex1.csv')
 Error in matrix(., ncol = 2, byrow = T) : 
  'data' must be of a vector type, was 'NULL' 

[vbonhomme]

send me "Ex1.csv" > ***@***.*** Le mer. 23 juin 2021 à 15:13, Ely Raz ***@***.***> a écrit :

…

The same error > ex1 <- from_csv_to_Out('E:/Ex1.csv') Error in matrix(., ncol = 2, byrow = T) : 'data' must be of a vector type, was 'NULL' — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFSFOIZAYGCLBZIZIJLTUHMZPANCNFSM4NDNWFNA> .

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[vbonhomme]

wait a min, where does from_csv_to_Out comes from ??? it's not in the released version of Momocs ! send me your csv I can do the import anyway Le mer. 23 juin 2021 à 15:16, Vincent Bonhomme ***@***.***> a écrit :

…

send me "Ex1.csv" > ***@***.*** Le mer. 23 juin 2021 à 15:13, Ely Raz ***@***.***> a écrit : > The same error > > > ex1 <- from_csv_to_Out('E:/Ex1.csv') > Error in matrix(., ncol = 2, byrow = T) : > 'data' must be of a vector type, was 'NULL' > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#211 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABECQFSFOIZAYGCLBZIZIJLTUHMZPANCNFSM4NDNWFNA> > . > -- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[raz1]

It is the file given above by lrocax

[vbonhomme]

this works like a charm on my machine: ```` from_csv_to_Out <- function(df, col){ df <- as_tibble(df) # cosmetics fac <- df %>% select(-col) # all but non coo <- df %>% select(col) %>% #split rowwise split(1:nrow(.)) %>% # for each row, turn columns into shape map(~.x %>% pull() %>% str_remove_all("\\[|\\]") %>% str_replace_all(";", " ") %>% str_split(" ") %>% unlist() %>% as.numeric() %>% matrix(ncol=2, byrow=T) ) # build and return the Out Out(coo, fac) } "~/Desktop/Ex1.csv" %>% read.table(h=T, sep=",") %>% from_csv_to_Out(col=4) ```` Le mer. 23 juin 2021 à 15:20, Ely Raz ***@***.***> a écrit :

…

It is the file given above by lrocax — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABECQFV7CRISFLB4SIL3DBDTUHNR3ANCNFSM4NDNWFNA> .

-- vincentbonhomme.fr <http://www.vincentbonhomme.fr>

[raz1]

Thanks