Unable to interact with native password type text field in webview

[AlexisDeslandes]

Hello, i have an issue with login.
I use the package flutter_appauth which work with a webview to authenticate.
I have to fill a web form to log, enterText works with the username but not for the password, even after hiding the keyboard.
There is a placeholder with the word: Password

I tried many ways to do:
await $.native.enterText(Selector(text: 'Password'), text: '');
await $.native.enterTextByIndex('', index: 1);
Selector(field with contains ...)

Got: Patrol action failed: enterText() failed with code NOT_FOUND
And the second one says that index 1 doesn't exists.

After view hierarchy dump I got:

I did the dump and got:

[...
{
"AXFrame": "{{74, 372}, {227, 40}}",
"AXUniqueId": null,
"frame": {"y": 372, "x": 74, "width": 227, "height": 40},
"role_description": "text field",
"AXLabel": null,
"content_required": false,
"type": "TextField",
"title": null,
"help": "",
"custom_actions": [],
"AXValue": "Username",
"enabled": true,
"role": "AXTextField",
"subrole": null
},
{
"AXFrame": "{{74, 427}, {57, 15}}",
"AXUniqueId": null,
"frame": {"y": 427, "x": 74, "width": 57, "height": 15},
"role_description": "text",
"AXLabel": "Password",
"content_required": false,
"type": "StaticText",
"title": null,
"help": "",
"custom_actions": [],
"AXValue": "",
"enabled": true,
"role": "AXStaticText",
"subrole": null
},
{
"AXFrame": "{{74, 448}, {227, 41}}",
"AXUniqueId": null,
"frame": {"y": 448, "x": 74, "width": 227, "height": 41},
"role_description": "secure text field",
"AXLabel": null,
"content_required": false,
"type": "TextField",
"title": null,
"help": "",
"custom_actions": [],
"AXValue": "Password",
"enabled": true,
"role": "AXTextField",
"subrole": "AXSecureTextField"
},
...]

My tests were done using macos & iPhone simulator.
If needed I could try to take the original html from the login page and try to do something to remove confidential data.
Would you have a solution?

[bartekpacia]

Thanks for creating an issue. This is a bug that we'll have to fix.

I think that the cause is that the password is a secure text field, not a normal text field. This will require changing the enterText() method in the iOS automation server.

Source:

• https://developer.apple.com/forums/thread/5910

[AlexisDeslandes]

I would like to try fix this issue @bartekpacia.
I don't succeed to print something from ios automation server enterText() method.
Do you have a command to execute to rebuild ios code?

[bartekpacia]

Xcode project of the native iOS automation is downloaded by default to ~/.cache/patrol/ios-0.6.15 (for the current newest version of patrol_cli).

It's built on every patrol drive (unfortunately, see #278), so you can just edit Swift code and have it applied.

That said, while the user-facing API is quite stable and we try not to make many breaking changes, the implementation is not – I work on it almost every day and change lots of code. You can submit a PR if you want, but expect a lot of merge conflicts.

I'll try to fix this issue by the end of this week though, so you can just keep waiting :)

[AlexisDeslandes]

Ok perfect, thanks you very much !

[bartekpacia]

Hi @AlexisDeslandes, today is Friday, and this should be fixed :)

[AlexisDeslandes]

Thanks @bartekpacia, I tried tu run patrol drive command but got this error:

Testing failed:
'CGVectorMake' is unavailable in Swift
Testing cancelled because the build failed.

** TEST FAILED **
Unhandled exception:
Exception: Test failed. See logs above. Also, consider running with --verbose.
#0      IOSDriver._runServer.<anonymous closure> (package:patrol_cli/src/features/drive/platform/ios_driver.dart:174:9)
#1      List.forEach (dart:core-patch/growable_array.dart:416:8)
#2      ProcessListeners.listenStdErr.<anonymous closure> (package:patrol_cli/src/common/extensions/process.dart:50:14)
#3      _RootZone.runUnaryGuarded (dart:async/zone.dart:1586:10)
#4      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#5      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#6      _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#7      _StreamController._add (dart:async/stream_controller.dart:648:7)
#8      _StreamController.add (dart:async/stream_controller.dart:596:5)
#9      _Socket._onData (dart:io-patch/socket_patch.dart:2324:41)
#10     _RootZone.runUnaryGuarded (dart:async/zone.dart:1586:10)
#11     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#12     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#13     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#14     _StreamController._add (dart:async/stream_controller.dart:648:7)
#15     _StreamController.add (dart:async/stream_controller.dart:596:5)
#16     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1849:33)
#17     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1322:14)
#18     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#19     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
#20     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:122:13)
#21     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:193:5)

I saw you added:

// Adapted from https://samwize.com/2016/02/28/everything-about-xcode-ui-testing-snapshot/
extension XCUIElement {
  func forceTap() {
    if self.isHittable {
      self.tap()
    } else {
      let coordinate = self.coordinate(withNormalizedOffset: CGVectorMake(0.0, 0.0))
      coordinate.tap()
    }
  }
}

CGVectorMake has to be replaced with CGVector I think.

[bartekpacia]

Oh, that's strange. What version of Xcode and patrol_cli do you have installed?

[AlexisDeslandes]

I have xcode 13.4.1 and patrol_cli v0.7.0

[bartekpacia]

Could you update to Xcode 14?

Looks OK on my machine.

[AlexisDeslandes]

It was a problem with xCode version, it seems. Thanks
My attempt with enterText() failed, the text was not typed but the logs said successful.
It succeed with enterTextByIndex() :)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.