Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy server blocks API requests ending in ".com" #1745

Closed
Morhoo opened this issue Dec 6, 2023 · 14 comments
Closed

Proxy server blocks API requests ending in ".com" #1745

Morhoo opened this issue Dec 6, 2023 · 14 comments
Assignees
@phw198
Labels
enhancement New feature question resolved Working hotfix available
Milestone
v2.10.3

Comments

@Morhoo
Copy link

Morhoo commented Dec 6, 2023 •
edited
Loading

OGCS v2.10.0.0

Hi, today I encountered an issue with corporate proxy server blocking requests to Google API to:

www.googleapis.com/calendar/v3/users/me/calendarList/{id}@group.calendar.google.com

Reason is it stupidly evaulates the request as a download of file with ".com" extension. Solution is to add a slash at the end of request URL like this:

www.googleapis.com/calendar/v3/users/me/calendarList/{id}@group.calendar.google.com/

It still works on Google side and is enough to trick the proxy to allow it through. Only problem is, that if I change the settings.xml file ("Id" tag), OGCS internaly url-encode the value and slash is translated to %2f, which is not working.

So my question is, is there a way how to achieve this change without any third-party tools? Can appending of slash be added as an option to OGCS in future versions?

Thank you.
@Morhoo Morhoo added the question label Dec 6, 2023
@phw198
Copy link
Owner

phw198 commented Dec 8, 2023

I think the issue is specifically related to the calendarList API call which ends with the calendar ID and .com in the URI. If you can provide your log file around that error, I could look to catch it and retry with the trailing slash just for that call...

@phw198 phw198 added the more info needed Not enough detail provided to fully troubleshoot label Dec 8, 2023
@phw198 phw198 changed the title Proxy server blocks requests Proxy server blocks API requests ending in ".com" Dec 8, 2023
@Morhoo
Copy link
Author

Morhoo commented Dec 8, 2023

Hi, sending log file with maximum verbosity.

It is as you say. During call to calendar API there is a 403 response from our corporate proxy server, denying the access because of downloading unrecognized media type file. I just masked "id" part of the URL from the response with asterisks.

I tried to intercept the call and tamper it manually using trailing slash and it does work. Just don't know how to force it in the app.

Thank you.

OGcalsync.log

@phw198 phw198 removed the more info needed Not enough detail provided to fully troubleshoot label Dec 8, 2023
@phw198 phw198 self-assigned this Dec 8, 2023
phw198 added a commit that referenced this issue Dec 8, 2023
@phw198
Add trailing slash to calendarList API call for confused proxies.
9a3bcc0 
#1745
@phw198 phw198 added enhancement New feature waiting fix confirmation Hotfix available for testing labels Dec 8, 2023
@phw198
Copy link
Owner

phw198 commented Dec 8, 2023

📦 Please have a go with hotfix v2.10.2.7.zip (see instructions for applying) and let me know how it goes?

@Morhoo
Copy link
Author

Morhoo commented Dec 8, 2023 •
edited
Loading

Thank you for such a fast response, however the hotfix doesn't seem to change anything in my case. The request to Google API sill goes without slash at the end resulting in 403:

Intercepted communication between OGCS and Google:

obrazek

obrazek

@phw198
Copy link
Owner

phw198 commented Dec 8, 2023

The first call should fail and then a second call should have the / appended - could you provide your log file again, please?

@Morhoo
Copy link
Author

Morhoo commented Dec 8, 2023

Sure, here you go.

OGcalsync.log

But I am pretty sure no other request follows after first failure. :(

@phw198
Copy link
Owner

phw198 commented Dec 8, 2023 •
edited
Loading

The error response you are receiving has completely changed 😕

In your previous log we had the proxy HTML response logged in the FAIL line:

2023-12-08 12:05:40,376 FINE  13 OutlookGoogleCalendarSync.Settings [463] -  Using profile Sync.Engine.Calendar.Instance.Profile
2023-12-08 12:05:40,615 ULTRA-FINE  1 OutlookGoogleCalendarSync.Sync.Engine+Job+Queue [54] -  Sync queue size: 0
2023-12-08 12:05:40,660 FAIL  13 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [1911] -  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

Now you are just getting:

2023-12-08 17:23:21,408 FINE  19 OutlookGoogleCalendarSync.Settings [467] -  Using profile Sync.Engine.Calendar.Instance.Profile
2023-12-08 17:23:21,686 ULTRA-FINE  1 OutlookGoogleCalendarSync.Sync.Engine+Job+Queue [54] -  Sync queue size: 0
2023-12-08 17:23:21,688 FAIL  19 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [2105] -  The service calendar has thrown an exception. HttpStatusCode is Forbidden. No error message was specified.

The hotfix I don't believe would have changed this. Have you made a change with your interception? Were/are you using something like Fiddler or BurpSuite?

@Morhoo
Copy link
Author

Morhoo commented Dec 8, 2023

Burp Suite for request analysis, but I don't use it when generating logfiles. I had to upgrade the application version from 2.10.0.0 beta to 2.10.2.0 alpha prior to applying the hotfix, so maybe this was the change?

@phw198
Copy link
Owner

phw198 commented Dec 8, 2023

The Google API libraries were updated in 2.10.1, so I guess they've changed what is being returned - seems a retrograde step to go from the verbose proxy HTTP response to "No error message was specified".

Ah well, this is why I always ask for issues to be raised only on the latest alpha....!

phw198 added a commit that referenced this issue Dec 8, 2023
@phw198
Alter matching string on error message returned.
79be271 
#1745
@phw198
Copy link
Owner

phw198 commented Dec 8, 2023

📦 Please give it another go with hotfix v2.10.2.8.zip (see instructions for applying).

@Morhoo
Copy link
Author

Morhoo commented Dec 8, 2023

I'm sorry for not using alpha from the beginning. I tried both and decided beta is safer option.

OGcalsync.log

Now I can confirm it sends two requests, both without trailing slash though. :(

phw198 added a commit that referenced this issue Dec 9, 2023
@phw198
Fully drop down to HttpClient to call API endpoint.
c810844 
#1745
@phw198
Copy link
Owner

phw198 commented Dec 9, 2023 •
edited
Loading

📦 Third time's a charm for hotfix v2.10.2.9.zip... 🤞

@Morhoo
Copy link
Author

Morhoo commented Dec 9, 2023 •
edited
Loading

Yep, that did it!

2023-12-09 23:10:37,134 WARN  10 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [1741] -  Call to CalendarList API endpoint failed. Retrying with trailing '/' in case of poorly configured proxy.
2023-12-09 23:10:37,473 INFO  10 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [1752] -  Google calendar timezone: Europe/Prague
2023-12-09 23:10:37,474 FINE  10 OutlookGoogleCalendarSync.Settings [467] -  Using profile Sync.Engine.Calendar.Instance.Profile
2023-12-09 23:10:37,478 FINE  10 OutlookGoogleCalendarSync.Settings [467] -  Using profile Sync.Engine.Calendar.Instance.Profile
2023-12-09 23:10:37,478 DEBUG  10 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [286] -  Retrieving all events from Google: 02.12.2023 -> 08.02.2024
2023-12-09 23:10:37,738 DEBUG  10 OutlookGoogleCalendarSync.GoogleOgcs.Calendar [300] -  Page 1 received.
2023-12-09 23:10:37,740 DEBUG  10 OutlookGoogleCalendarSync.Recurrence [418] -  Identifying exceptions in recurring Google events.
2023-12-09 23:10:37,741 DEBUG  10 OutlookGoogleCalendarSync.Recurrence [426] -  Found 0 exceptions.
2023-12-09 23:10:37,741 INFO  10 OutlookGoogleCalendarSync.Console [296] -  12 Google calendar entries found.

Thank you very much, I appreciate the effort in such specific case. If you need perhaps someone for Czech translation, I can be of help.

@phw198 phw198 added resolved Working hotfix available and removed waiting fix confirmation Hotfix available for testing labels Dec 16, 2023
@phw198 phw198 added this to the v2.10.3 milestone Dec 16, 2023
@phw198 phw198 mentioned this issue Dec 16, 2023
Open
@phw198
Copy link
Owner

phw198 commented Dec 16, 2023

Thanks for confirming and the offer for Czech translations 👍

@phw198 phw198 closed this as completed in 56efcfa Jan 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@phw198 phw198

Labels
enhancement New feature question resolved Working hotfix available
Projects
None yet
Milestone
v2.10.3
Development

No branches or pull requests
2 participants
@phw198 @Morhoo