You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
File ~/FinNLP/finnlp/data_sources/social_media/reddit_streaming.py:40, in Reddit_Streaming.download_streaming_all(self, rounds)
38 if rounds > 1:
39 for _ in range(1,rounds):
---> 40 last_id = self._fatch_other_pages(last_id, pbar)
File ~/FinNLP/finnlp/data_sources/social_media/reddit_streaming.py:82, in Reddit_Streaming._fatch_other_pages(self, last_page, pbar)
49 data = {
50 "id": "02e3b6d0d0d7",
51 "variables": {
(...)
79 }
80 }
81 response = self._request_post(url = url, headers= headers, json = data)
---> 82 data = json.loads(response.text)
83 data = data["data"]["subredditInfoByName"]["elements"]["edges"]
84 for d in data:
AttributeError: 'NoneType' object has no attribute 'text'
The text was updated successfully, but these errors were encountered:
protocold
changed the title
Reddit scrapping doesnt work
Reddit scrapping doesnt work - AttributeError: 'NoneType' object has no attribute 'text'
Jul 17, 2023
I simply pasted the example code for Reddit and it errored out..
Downloading by pages...: 0%| | 0/3 [00:00<?, ?it/s]
Downloading by pages...: 33%|███████████████████████████████ | 1/3 [00:02<00:04, 2.22s/it]
AttributeError Traceback (most recent call last)
Cell In[12], line 11
4 config = {
5 "use_proxy": "us_free",
6 "max_retry": 5,
7 "proxy_pages": 2,
8 }
10 downloader = Reddit_Streaming(config)
---> 11 downloader.download_streaming_all(pages)
12 selected_columns = ["created", "title"]
13 downloader.dataframe[selected_columns].head(10)
File ~/FinNLP/finnlp/data_sources/social_media/reddit_streaming.py:40, in Reddit_Streaming.download_streaming_all(self, rounds)
38 if rounds > 1:
39 for _ in range(1,rounds):
---> 40 last_id = self._fatch_other_pages(last_id, pbar)
File ~/FinNLP/finnlp/data_sources/social_media/reddit_streaming.py:82, in Reddit_Streaming._fatch_other_pages(self, last_page, pbar)
49 data = {
50 "id": "02e3b6d0d0d7",
51 "variables": {
(...)
79 }
80 }
81 response = self._request_post(url = url, headers= headers, json = data)
---> 82 data = json.loads(response.text)
83 data = data["data"]["subredditInfoByName"]["elements"]["edges"]
84 for d in data:
AttributeError: 'NoneType' object has no attribute 'text'
The text was updated successfully, but these errors were encountered: