Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem working with Squid web proxy #35

Closed
aburnheimer opened this issue Dec 1, 2012 · 2 comments · May be fixed by #36
Closed

Problem working with Squid web proxy #35

aburnheimer opened this issue Dec 1, 2012 · 2 comments · May be fixed by #36

Comments

@aburnheimer
Copy link

I'm having a problem going through Squid despite the changes bgreenlee suggested 3 years ago in Issue #2.

Using Twitter::JSONStream as shown at https://gist.github.com/4179836.js?file=Listen_in_on_Campfire_room.rb , I get the following results - with a little bit of annotation sprinkled in at bottom of JSONStream#send_request, and temporarily backing out the "full url" fix bgreenlee suggested, please read on:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-f
orm-urlencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :param
s=>{}, :user_agent=>"TwitterStream", :host=>"streaming.campfirenow.com", :
proxy=>"http://...proxy_host_name removed...:3128", :auth=>"1234567890abc
def123467890abcdef123456789:X", :ssl=>true, :oauth=>{}, :path=>"/room/123
456/live.json", :filters=>[]}
send_request: ["GET /room/123456/live.json HTTP/1.1", "Host: streaming.ca
mpfirenow.com", "Accept: */*", "User-Agent: TwitterStream", "Authorizatio
n: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD", "\r\n
"]
Tried 11 times to connect.

Meanwhile, using curl, I can generate the exact same headers, and it works...

[aburnheimer@hostname ~]$ echo $http_proxy
http://...proxy_host_name removed...:3128
[aburnheimer@hostname ~]$ echo $HTTP_PROXY
http://...proxy_host_name removed...:3128
[aburnheimer@hostname ~]$ curl -k -v -A "TwitterStream" -u 123467890abcdef12346
7890abcdef123456789:X -x $http_proxy https://streaming.campfirenow.com/room/123
467/live.json
* About to connect() to proxy ...proxy_host_name removed... port 3128 (#0)
*   Trying proxy_ip removed... connected
* Connected to proxy_host_name removed (proxy_ip removed) port 3128 (#0)
* Establish HTTP proxy tunnel to streaming.campfirenow.com:443
* Server auth using Basic with user '123467890abcdef123467890abcdef123456789'
> CONNECT streaming.campfirenow.com:443 HTTP/1.1
> Host: streaming.campfirenow.com:443
> User-Agent: TwitterStream
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*   subject: CN=*.campfirenow.com,OU=Domain Control Validated - RapidSSL(R),OU=S
ee www.rapidssl.com/resources/cps (c)11,OU=GT23036785,O=*.campfirenow.com,C=US,
serialNumber=JoGozIJmZwhtvHRQCuYEMvwbe6AYqcjR
*   start date: Dec 24 15:40:13 2011 GMT
*   expire date: Jan 25 19:52:44 2014 GMT
*   common name: *.campfirenow.com
*   issuer: CN=RapidSSL CA,O="GeoTrust, Inc.",C=US
* Server auth using Basic with user '123467890abcdef123467890abcdef123456789'
> GET /room/123467/live.json HTTP/1.1
> Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD
> User-Agent: TwitterStream
> Host: streaming.campfirenow.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sat, 01 Dec 2012 00:41:52 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{"room_id":123467,"created_at":"2012/12/01 00:41:54 +0000","body":"Lorem ipsum
dolor sit amet, consectetur adipiscing elit. Donec a diam lectus. Sed sit amet i
psum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante hendre
rit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit ame
t vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. D
onec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit
amet, c

Even just advice on where or how to investigate further would be appreciated. I've been annotating twitter-stream-0.1.16 and eventmachine-1.0.0 to try and determine where these may be diverting from whatever curl is doing right. Unfortunately, I haven't found any clues after lots of trials and errors.

My next move is to switch to the pure_ruby of eventmachine, but the initial resistance I found with that suggested to me to maybe reach out for help first.
@aburnheimer
Copy link
Author

Doing as voloko suggested, and restoring the full url as bgreenlee suggested did not result in anymore success here:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-form-ur
lencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :params=>{}, :user
_agent=>"TwitterStream", :host=>"proxy_ip removed", :proxy=>"http://...proxy_ho
stname removed...:3128", :auth=>"123467890abcdef123467890abcdef123456789:X", :s
sl=>true, :oauth=>{}, :path=>"/room/123456/live.json", :filters=>[]}
send_request: ["GET https://proxy_ip removed:443/room/123456/live.json HTTP/1.1
", "Host: streaming.campfirenow.com", "Accept: */*", "User-Agent: TwitterStream
", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZAB
CD", "\r\n"]

nor here - note the port number is present in the Host header for this attempt:

[aburnheimer@hostname ~]$ ./Listen_in_on_Campfire_room.rb
options: {:port=>443, :method=>"GET", :content_type=>"application/x-www-form-ur
lencoded", :auto_reconnect=>true, :content=>"", :timeout=>0, :params=>{}, :user
_agent=>"TwitterStream", :host=>"proxy_ip removed", :proxy=>"http://...proxy_ho
stname removed...:3128", :auth=>"123467890abcdef123467890abcdef123456789:X", :s
sl=>true, :oauth=>{}, :path=>"/room/123456/live.json", :filters=>[]}
send_request: ["GET https://proxy_ip removed:443/room/123456/live.json HTTP/1.1
", "Host: streaming.campfirenow.com:443", "Accept: */*", "User-Agent: TwitterSt
ream", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWX
YZABCD", "\r\n"]

@aburnheimer
Copy link
Author

Found it... Have to have the options look as such, just as voloko suggested way-back:

options: {:auth=>"123467890abcdef123467890abcdef123456789:X", :method=>"GET",
:auto_reconnect=>true, :content=>"", :content_type=>"application/x-www-form-url
encoded", :user_agent=>"TwitterStream", :host=>"proxy_ip removed", :params=>{}, :t
imeout=>0, :proxy=>nil, :port=>3128, :oauth=>{}, :path=>"https://streaming.camp
firenow.com:443/room/123456/live.json", :ssl=>false, :filters=>[]}
send_request: ["GET https://streaming.campfirenow.com:443/room/123456/live.json
 HTTP/1.1", "Host: streaming.campfirenow.com", "Accept: */*", "User-Agent: Twit
m", "Authorization: Basic ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZA
BCD", "\r\n"]

I'll wrap this into a pull request, or something similar later this evening.

@aburnheimer aburnheimer mentioned this issue Dec 2, 2012
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Connect through a web proxy more directly aburnheimer/twitter-stream
1 participant
@aburnheimer