Streaming API incomplete tweet

[bcambel]

Hi Adam,

Great library, thank you very much!

I am having an issue where the received tweet stream is incomplete. In some cases, the stream is properly JSON parsed though. Here is the code that I use

(defn parse-tweet
  [resp tweet]
  (try 
    (let [tw-obj (parse-string tweet true)]
      (when (:text tw-obj)
        (log/info (select-keys tw-obj [:id :text]))))
    (catch Throwable t 
      (log/error (.getMessage t) (format "Can't parse tweet\n%s" tweet)))))

(defn vacuum-twttr [tracking]
  (statuses-filter :params {:track tracking}
             :oauth-creds my-creds
             :callbacks (AsyncStreamingCallback. 
                            (fn [resp tweet]
                                (parse-tweet resp (str tweet)))
                          (comp println response-return-everything)
                          exception-print)))

My assumption is that somehow using str over ByteArrayOutputStream creates the problem. Do you have any suggestions ?

Thanks a lot,
Bahadir

[adamwynne]

I think it being chunked by twitter - someone wrote some library (forgotten
which) that sits on top of mine and does the correct message chunking -
take a look at the issues and it should be there

On Thu, Dec 11, 2014 at 11:03 AM, Bahadir Cambel [email protected]
wrote:

Hi Adam,

Great library, thank you very much!

I am having an issue where the received tweet stream is incomplete. In
some cases, the stream is properly JSON parsed though. Here is the code
that I use

(defn parse-tweet
[resp tweet](try
%28let [tw-obj %28parse-string tweet true%29]
%28when %28:text tw-obj%29
%28log/info %28select-keys tw-obj [:id :text]%29%29%29%29
%28catch Throwable t
%28log/error %28.getMessage t%29 %28format "Can't parse tweetn%s" tweet)))))

(defn vacuum-twttr [tracking](statuses-filter :params {:track tracking}
:oauth-creds my-creds
:callbacks %28AsyncStreamingCallback.
%28fn [resp tweet]
%28parse-tweet resp %28str tweet%29%29%29
%28comp println response-return-everything%29
exception-print%29))

My assumption is that somehow using str over ByteArrayOutputStream
creates the problem. Do you have any suggestions ?

Thanks a lot,
Bahadir

—
Reply to this email directly or view it on GitHub
#58.

[bcambel]

Thanks for the quick response Adam. For future reference, here is the library https://github.com/mccraigmccraig/twitter-streaming-client

[adamwynne]

thanks for pointing it out

On Thu, Dec 11, 2014 at 12:32 PM, Bahadir Cambel [email protected]
wrote:

Thanks for the quick response Adam. For future reference, here is the
library https://github.com/mccraigmccraig/twitter-streaming-client

—
Reply to this email directly or view it on GitHub
#58 (comment)
.