iOS assist shortcuts/intent not working with Google GenAI Conversation

[danielfrg]

I have integrated the Google Generative AI Conversation into "Assist" and it all works great on the web and iOS app.

Except when trying to run it from a shortcut when I always get "Sorry, I couldn't understand that".

I believe this is because of the language selection.

I only have one Voice Assistant. When look at the "Debug Assistant" on the HA app/web I see the requests that I send from the web but not the ones i send from shortcuts.

Those requests have language set as "*". But thats not an option from the shortcuts app.

I should also mention that other shortcuts work just fine. It's only the "Assist" one that is failing.

Model Name:
Software Version: 2024.3.1
App version: latests

[paull27]

Can confirm this problem also happens with Open AI Conversation agent. Not possible to use it with iOS shortcuts

[earlyXbirdNG]

I installed the Assist shortcut and set up multiple voice assistant pipelines with ChatGPT and Google AI. However, I can’t change the conversation agent on the shortcut; it always defaults to the limited Home Assistant agent.

[henry4711lp]

Same issue here.
Can't change conversation agent
More or less comparable: home-assistant/core#97214

[jfromaniello]

I am having the same issue. It does what expected from within the ios application but not with the Assist shortcut.

[sabaatworld]

Any solution for this?

[The-Croz]

This is still an issue with the iOS HA app. If you select "Assist in app" in Shortcuts you can select a pipeline to use but if you select "Assist" so it can be used/triggered through Siri or HomePods then it uses the default limited Home Assistant agent and you cannot change this.

Please resolve by adding this functionality into the iOS app with Shortcuts. Thanks!

[The-Croz]

Also, I just added this to Month of WTH on the Home Assistant Community. Go comment and vote!

https://community.home-assistant.io/t/wth-cant-we-choose-a-voice-pipeline-on-ios-siri-shortcuts/805035

[bgoncal]

#3242