@sosnovsky Please confirm if you are working/worked on this task.

@ioanmo226 I tried to re-enable live test in https://github.com/FlowCrypt/flowcrypt-ios/tree/bugfix/issue-1850-semaphore-live-test, but it fails because Google asks for additional verification on login.

Probably 1 successful login will be enough and then live test should work without issues, but I wasn't able to pass verification on remote machine - I tried to make screenshots of remote Semaphore machine to choose right number for verification, but it still failed to sign in.

this will need my help, probably

i was thinking to maybe use a test domain with our own idp instead of google. then we can control it

easier option may be to attempt to disable these other verifications. previously not possible, but i saw some new options i think

Like browser extension which uses flowcrypt.dev domain with auth0?

it doesn't.. or does it? i'm forgetting.

the intention was to do something like that, yes. i forgot we actually did that before. if we did, does it still work?

It does. we are currently using [email protected] account with auth0 (IDP) for browser extension live test
Live tests are working fine in browser extension :-)

we could use the same exact account here. we'd just need to add missing sample emails, if any, needed by ios tests

before that though, i'll go through google admin settings. that would be less work, if i can get it to work that way

There is a way to turn off "challenges" for 10 minutes at a time. They say the challenge is only needed on new devices (which in the cloud, every device is a new device, almost).

If we were to back up the login state at the end of a successful local run. Wherever the ios device saves the google login information. Is that in the web browser cookies or elsewhere? If we could export that, add that to semaphore secrets, and have the semaphore code load that into the iOS emulator when starting, that would mean it would appear to be the same device, meaning same session, and there should be no challenges, or they should not be that frequent.

Previously we didn't need to pass these challenges on each live test run. I think this problem appeared when we switched from macos-xcode13 to macos-xcode14 machine on Semaphore, so maybe some cookies are persisted between CI test runs.

We can try to disable this login challenge for 10 minutes and run live test on Semaphore, to check if sign in works well in this case. Then we should check if re-running live test still shows login challenge or login passes without it.

From my experience on browser extension, if you do the 10 minute thing enough times, you may cover most of the machines they allocate to you, and you'll mostly be ok. But changes in tooling suddenly bring those challenges back, like it did here.

Also can use https://accounts.google.com/b/0/DisplayUnlockCaptcha (where the 0 is the account number you are logged into, else you are unlocking another account. You can find out by logging into gmail with the target account and seeing what number is in URL). It does the same thing - remove challenge for 10 minutes.

If you try that a few times before running live tests, you may find that it may stop giving us trouble.

@tomholub, I've fixed the live tests. They were even failing locally due to some keys no longer existing, environment support issues, and so on.
However, they are still failing on SemaphoreCI because of the 2-step verification.

Perhaps we could create a new account, [email protected], and remove the [email protected] account? Alternatively, we could use the existing [email protected] account, which I believe is currently being used for the browser test.

If we continue to use this [email protected], same problem would appear again in the future I think.
Or maybe we could try disabling the login challenge, then let SemaphoreCI run, and hope the issue doesn't reappear. (disabling login challenge is only 10 mins and running live test takes about 40 mins though. first build xcode project, run mock other test and then live test)

DisplayUnlockCaptcha is not available though.

@tomholub In case you missed it

This will basically be a never ending problem unless we completely sidestep Google for authentication, and configure our domain to use a 3rd party IdP instead.

I'll try it on the flowcrypt.dev domain first. The way it works, a Google auth prompt gets popped up (where you maybe click on your account to choose it) and after it understands which account you want to log in for, Google forwards the user to an auth window hosted by another IdP, like Auth0. Now, Auth0 can be reliably configured to just use email + password without further hassles. The only thing on the client app tests, I think we'll have to update the selectors of which fields to fill based on whatever the other IdP presents.

I've been hesitating with this, because I don't properly understand it, and I worry if I misconfigure it, we'll never be able to log in again (if I break it and even the account of the admin cannot log into Google Admin Console to fix it). But starting out on the flowcrypt.dev domain would make it less risky.

Unfortunately still didn't get to this