Skip to content
This repository has been archived by the owner on Jun 30, 2023. It is now read-only.

[FR] Improved Error handling for Functions utilizing make_query() #322

Open
TimBMK opened this issue May 16, 2022 · 0 comments
Open

[FR] Improved Error handling for Functions utilizing make_query() #322

TimBMK opened this issue May 16, 2022 · 0 comments

Comments

@TimBMK
Copy link
Contributor

TimBMK commented May 16, 2022

Describe the solution you'd like

I noticed that the function get_user_profile() fails if the input contains NA values. I think this points to a larger issue of how functions utilizing make_query() deal with errors. One approach would be to build additional checks into make_query to e.g. check for empty input vectors. Another - and possibly more effective - approach would be to improve how the higher level functions deal with error output from make_query. A simple solution may be to always wrap make_query() into try(), so the underlying loop keeps going if one search query fails. This would be especially useful for large data requests, allowing users to get output for all search queries that did'nt fail. In the case of get_user_profile() it would allow to get data for the other 90.000 user profiles (in my case) even though a rogue NA value found itself into the data.

Anything else?

Something like this might do the trick, but this is untested (I think the binding might need some additional failsafe; this is mostly a mental note) . I could make a pull request, implement it for all higher level functions and do some testing. But I wanted to check back with you guys if there was a rationale for not implementing something similar?

function (x, bearer_token = get_bearer()) 
{
    bearer_token <- check_bearer(bearer_token)
    url <- "https://api.twitter.com/2/users"
    new_df <- data.frame()
    slices <- seq(1, length(x), 100)
    for (i in slices) {
        if (length(x) < (i + 99)) {
            end <- length(x)
        }
        else {
            end <- (i + 99)
        }
        cat(paste0("Processing from ", i, " to ", end, "\n"))
        slice <- x[i:end]
        params <- list(ids = paste(slice, collapse = ","), user.fields = "created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,public_metrics,url,username,verified,withheld")
        try({
            dat <- make_query(url = url, params = params, bearer_token = bearer_token, 
                 verbose = TRUE)
            new_df <- dplyr::bind_rows(new_df, dat$data)
         })
    }
    return(new_df)
}
<bytecode: 0x00000192fc15d2e0>
<environment: namespace:academictwitteR>

/// Edited the possible solution to something that should work. Wrapping the whole call + bind block into {} to evaluate wholly with try() seems to be the most simple solution. Let me know what you think though. Happy to implement, but would require some testing

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant