[BUG] Request failed with ', 401, #122

new-it · 2023-07-07T11:25:22Z

Describe the bug
It's our initial setup of IntuneCD in with Pipelines.

To Reproduce
Steps to reproduce the behavior:
Run the Pipeline

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.
As we only want to start with a backup and a documentation we choosed these AzureAD App rights:

Run type (please complete the following information):

Mode: 1
Client Pipeline
Version 1.4.9

Additional context

.yml file:

trigger: none

pool:
  vmImage: ubuntu-latest

variables:
  REPO_DIR: $(Build.SourcesDirectory)
  TENANT_NAME: xxx.onmicrosoft.com
  CLIENT_ID: xxx

steps:

- checkout: self
  persistCredentials: true

- script: pip3 install IntuneCD
  displayName: Install IntuneCD

- script: |
      git config --global user.name "xxx"
      git config --global user.email "xxx"
  displayName: Configure Git

- script: IntuneCD-startbackup -m 1 -o yaml
  env:
    REPO_DIR: $(REPO_DIR)
    TENANT_NAME: $(TENANT_NAME)
    CLIENT_ID: $(CLIENT_ID)
    CLIENT_SECRET: $(CLIENT_SECRET)
  displayName: Run IntuneCD backup

- script: |
    export branch_name=configs-`date +'%Y-%m-%d-%H-%M'`
    cd $(REPO_DIR)
    git checkout -b $branch_name
    git add --all
    git commit -m "Updated configurations"
    git push --set-upstream origin $branch_name
  displayName: Commit changes

- script: IntuneCD-startdocumentation -t $(TENANT_NAME) -i 'This is a demo introduction'
  env:
    REPO_DIR: $(REPO_DIR)
  displayName: Run IntuneCD documentation

Output:

Starting: Run IntuneCD backup

Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.212.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line

Generating script.
Script contents:
IntuneCD-startbackup -m 1 -o yaml

/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/737016a5-23e4-41c5-a4a2-bbc354224046.sh
Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 370, in start
    run_backup(args.path, args.output, exclude, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 165, in run_backup
    results.append(savebackup(path, output, exclude, token))
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/backup_appConfiguration.py", line 34, in savebackup
    data = makeapirequest(ENDPOINT, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/graph_request.py", line 84, in makeapirequest
    raise Exception(
Exception: ('Request failed with ', 401, ' - ', '{"error":{"code":"UnknownError","message":"{\\"ErrorCode\\":\\"Forbidden\\",\\"Message\\":\\"{\\\\r\\\\n  \\\\\\"_version\\\\\\": 3,\\\\r\\\\n  \\\\\\"Message\\\\\\": \\\\\\"An error has occurred - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: e80ad478-e080-4167-ab5d-6615f7ce881e - Url: [https://fef.msub03.manage.microsoft.com/AppLifecycle_2306/StatelessAppMetadataFEService/deviceAppManagement/mobileAppConfigurations?api-version=5023-05-30\\\\\\",\\\\r\\\\n](https://fef.msub03.manage.microsoft.com/AppLifecycle_2306/StatelessAppMetadataFEService/deviceAppManagement/mobileAppConfigurations?api-version=5023-05-30\\\\\\%22,\\\\r\\\\n)  \\\\\\"CustomApiErrorPhrase\\\\\\": \\\\\\"\\\\\\",\\\\r\\\\n  \\\\\\"RetryAfter\\\\\\": null,\\\\r\\\\n  \\\\\\"ErrorSourceService\\\\\\": \\\\\\"\\\\\\",\\\\r\\\\n  \\\\\\"HttpHeaders\\\\\\": \\\\\\"{\\\\\\\\\\\\\\"WWW-Authenticate\\\\\\\\\\\\\\":\\\\\\\\\\\\\\"Bearer realm=\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"urn:intune:service,9225b241-44e1-44a8-8bfe-c10e39177505,f0f3c450-59bf-4f0d-b1b2-0ef84ddfe3c7,3e9c57b9-808d-4aa0-9500-4b2d369279e7\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\"}\\\\\\"\\\\r\\\\n}\\",\\"Target\\":null,\\"Details\\":null,\\"InnerError\\":null,\\"InstanceAnnotations\\":[]}","innerError":{"date":"2023-07-07T11:06:12","request-id":"e80ad478-e080-4167-ab5d-6615f7ce881e","client-request-id":"e80ad478-e080-4167-ab5d-6615f7ce881e"}}}')
##[error]Bash exited with code '1'.
Finishing: Run IntuneCD backup

The text was updated successfully, but these errors were encountered:

almenscorner · 2023-07-07T12:02:40Z

Can you share the permissions granted on the app registration you are using in the pipeline?

new-it · 2023-07-07T12:24:10Z

DeviceManagementApps.Read.All
DeviceManagementConfiguration.ReadWrite.All
DeviceManagementManagedDevices.Read.All
DeviceManagementServiceConfig.Read.All
Group.Read.All
Policy.Read.All
Policy.Read.ConditionalAccess
User.Read

Adminconsent is granted. There is also a screenshot in my initial post.

almenscorner · 2023-07-07T12:27:17Z

Sorry missed the screen shot 😬 the permissions are granted as delegated but they have to be "application" instead

new-it · 2023-07-07T13:03:11Z

Oh I missed that completely. Now I'm getting further, but also an error 403:

Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 370, in start
    run_backup(args.path, args.output, exclude, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 270, in run_backup
    results.append(savebackup(path, output, token))
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/backup_remoteAssistancePartner.py", line 28, in savebackup
    data = makeapirequest(ENDPOINT, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/graph_request.py", line 84, in makeapirequest
    raise Exception(
Exception: ('Request failed with ', 403, ' - ', '{"error":{"code":"Forbidden","message":"{\\r\\n  \\"_version\\": 3,\\r\\n  \\"Message\\": \\"An error has occurred - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: f6c95f9c-ee09-458a-95c3-cb1db532d474 - Url: https://fef.msub03.manage.microsoft.com/RemoteAssistService/StatelessRemoteAssistService/deviceManagement/remoteAssistancePartners?api-version=5022-08-15 - CustomApiErrorPhrase: Forbidden\\",\\r\\n  \\"CustomApiErrorPhrase\\": \\"Forbidden\\",\\r\\n  \\"RetryAfter\\": null,\\r\\n  \\"ErrorSourceService\\": \\"\\",\\r\\n  \\"HttpHeaders\\": \\"{}\\"\\r\\n}","innerError":{"date":"2023-07-07T13:01:36","request-id":"f6c95f9c-ee09-458a-95c3-cb1db532d474","client-request-id":"f6c95f9c-ee09-458a-95c3-cb1db532d474"}}}')
##[error]Bash exited with code '1'.
Finishing: Run IntuneCD backup

almenscorner · 2023-07-07T15:30:45Z

Looks like the same issue as in #120, do you have a partner configured?

new-it · 2023-07-08T18:57:37Z

Yes, these three are configured:

almenscorner · 2023-07-09T07:39:20Z

For whatever reason it seems like the permissions is not enough when the teamviewer connector is present. Can you for now try to exclude it from the backup?

I will have to take a deeper look at this when I'm back from vacation

new-it · 2023-07-09T11:05:26Z

I've changed the script entry to - script: IntuneCD-startbackup -m 1 -o yaml -e RemoteAssistancePartner

Now I'm getting this:

Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 370, in start
    run_backup(args.path, args.output, exclude, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 304, in run_backup
    get_group_report(path, output)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 95, in get_group_report
    groups = collect_groups(path)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 87, in collect_groups
    process_file(
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 30, in process_file
    data = load_file(name, f)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/load_file.py", line 24, in load_file
    repo_data = json.load(file)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
##[error]Bash exited with code '1'.
Finishing: Run IntuneCD backup

almenscorner · 2023-07-11T08:22:27Z

Is it possible for you to try a run locally and not in a pipeline to see if there's any difference?

new-it · 2023-07-12T13:51:09Z

I can't get it to run locally in general. I'm absolutely no python pro either. I'll wait until after your vacation, hoping that it will work via pipelines then.

almenscorner · 2023-08-15T19:30:02Z

Bug has been verified in #120, the backup works with interactive auth but not when running with application permissions which it should. I need to bring this to Microsoft.

almenscorner · 2023-09-01T05:27:35Z

Hi @new-it, can you do a new run including the remote management partner and provide me with the ActivityID of the failed run?

new-it · 2023-09-15T12:53:18Z

Sorry for the late response, I was on holiday.

Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 370, in start
    run_backup(args.path, args.output, exclude, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 270, in run_backup
    results.append(savebackup(path, output, token))
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/backup_remoteAssistancePartner.py", line 28, in savebackup
    data = makeapirequest(ENDPOINT, token)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/graph_request.py", line 68, in makeapirequest
    raise Exception("Request failed with ", response.status_code, " - ", response.text)
Exception: ('Request failed with ', 403, ' - ', '{"error":{"code":"Forbidden","message":"{\\r\\n  \\"_version\\": 3,\\r\\n  \\"Message\\": \\"An error has occurred - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: c9e18294-d8f2-499d-a521-81c5bb3adf42 - Url: https://fef.msub03.manage.microsoft.com/RemoteAssistService/StatelessRemoteAssistService/deviceManagement/remoteAssistancePartners?api-version=5022-08-15 - CustomApiErrorPhrase: Forbidden\\",\\r\\n  \\"CustomApiErrorPhrase\\": \\"Forbidden\\",\\r\\n  \\"RetryAfter\\": null,\\r\\n  \\"ErrorSourceService\\": \\"\\",\\r\\n  \\"HttpHeaders\\": \\"{}\\"\\r\\n}","innerError":{"date":"2023-09-15T12:48:44","request-id":"c9e18294-d8f2-499d-a521-81c5bb3adf42","client-request-id":"c9e18294-d8f2-499d-a521-81c5bb3adf42"}}}')
##[error]Bash exited with code '1'.
Finishing: Run IntuneCD backup

new-it · 2023-09-26T07:31:59Z

@almenscorner Any update on this?

almenscorner · 2023-09-26T07:41:12Z

Hi - I am still waiting on a reply from Microsoft on this issue. I'll ping them and ask for an update.

new-it · 2023-09-26T07:49:53Z

~~Is there any option to exclude the teamviewer connector from the backup?~~

almenscorner · 2023-09-26T07:51:20Z

Yes, you can use the -e RemoteAssistancePartner argument in the backup command

new-it · 2023-09-27T07:21:19Z

Sadly, we are now facing an issue with another profile.

Backing up Quality Update profile: Quality updates enforcement - last month
Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 379, in start
    run_backup(args.path, args.output, exclude, token, args.prefix)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 332, in run_backup
    get_group_report(path, output)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 93, in get_group_report
    groups = collect_groups(path)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 85, in collect_groups
    process_file(
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 30, in process_file
    data = load_file(name, f)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/load_file.py", line 24, in load_file
    repo_data = json.load(file)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
##[error]Bash exited with code '1'.
Finishing: Run IntuneCD backup

almenscorner · 2023-09-27T07:42:20Z

It looks like you have an empty JSON file somewhere causing the assignment report module to fail. Can you check if there is any JSON files in your repo with no content?

new-it · 2023-09-29T11:57:11Z

We've cloned this repo without making any changes to it. We've also tried to run the flow with an empty repo.

new-it · 2023-10-12T12:58:48Z

We're running into the problem even if we exclude the specific object.

- script: IntuneCD-startbackup -m 1 -o yaml -e windowsQualityUpdates RemoteAssistancePartner
  env:
    REPO_DIR: $(REPO_DIR)
    TENANT_NAME: $(TENANT_NAME)
    CLIENT_ID: $(CLIENT_ID)
    CLIENT_SECRET: $(CLIENT_SECRET)
  displayName: Run IntuneCD backup

Backing up Quality Update profile: Quality updates enforcement - last month
Traceback (most recent call last):
  File "/home/vsts/.local/bin/IntuneCD-startbackup", line 8, in <module>
    sys.exit(start())
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 386, in start
    run_backup(
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/run_backup.py", line 337, in run_backup
    get_group_report(path, output)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 93, in get_group_report
    groups = collect_groups(path)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 85, in collect_groups
    process_file(
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/assignment_report.py", line 30, in process_file
    data = load_file(name, f)
  File "/home/vsts/.local/lib/python3.10/site-packages/IntuneCD/load_file.py", line 24, in load_file
    repo_data = json.load(file)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
##[error]Bash exited with code '1'.

almenscorner · 2023-10-12T13:59:21Z

The error above is not necessarily tied to Quality update profiles as there might be an empty file somewhere else. It is the assignment report that fails. Are you getting the same error if you run a backup locally to a folder instead of a repo targeting the same environment?

new-it · 2023-10-13T07:54:29Z

Running locally in pwsh with Py 3.11 via .\IntuneCD-startbackup.exe -m 1 -a .\auth.json -p .\out\ -e RemoteAssistancePartner:

Backing up Proactive Remediation: [...]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\lf\AppData\Roaming\Python\Python311\Scripts\IntuneCD-startbackup.exe\__main__.py", line 7, in <module>
  File "C:\Users\lf\AppData\Roaming\Python\Python311\site-packages\IntuneCD\run_backup.py", line 386, in start
    run_backup(
  File "C:\Users\lf\AppData\Roaming\Python\Python311\site-packages\IntuneCD\run_backup.py", line 293, in run_backup
    results.append(savebackup(path, output, exclude, token, prefix, append_id))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lf\AppData\Roaming\Python\Python311\site-packages\IntuneCD\backup_proactiveRemediation.py", line 90, in savebackup
    f.write(decoded)
  File "C:\Program Files\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\ufeff' in position 0: character maps to <undefined>

almenscorner · 2023-10-13T07:55:45Z

Please follow this step and then run the backup: https://github.com/almenscorner/IntuneCD/wiki/FAQ#q-i-get-a-codec-cant-encode-character--when-running-the-backup

new-it · 2023-10-23T09:27:57Z

I've managed to get it to work locally, and for some reason it just started working as a pipeline without any further changes. Thanks a lot for your help!

new-it added the bug Something isn't working label Jul 7, 2023

new-it closed this as completed Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Request failed with ', 401, #122

[BUG] Request failed with ', 401, #122

new-it commented Jul 7, 2023 •

edited

Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 7, 2023 •

edited

Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 7, 2023 •

edited

Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 8, 2023

almenscorner commented Jul 9, 2023

new-it commented Jul 9, 2023 •

edited

Loading

almenscorner commented Jul 11, 2023

new-it commented Jul 12, 2023

almenscorner commented Aug 15, 2023 •

edited

Loading

almenscorner commented Sep 1, 2023

new-it commented Sep 15, 2023 •

edited

Loading

new-it commented Sep 26, 2023 •

edited

Loading

almenscorner commented Sep 26, 2023

new-it commented Sep 26, 2023 •

edited

Loading

almenscorner commented Sep 26, 2023

new-it commented Sep 27, 2023

almenscorner commented Sep 27, 2023

new-it commented Sep 29, 2023

new-it commented Oct 12, 2023

almenscorner commented Oct 12, 2023

new-it commented Oct 13, 2023

almenscorner commented Oct 13, 2023 •

edited

Loading

new-it commented Oct 23, 2023

[BUG] Request failed with ', 401, #122

[BUG] Request failed with ', 401, #122

Comments

new-it commented Jul 7, 2023 • edited Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 7, 2023 • edited Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 7, 2023 • edited Loading

almenscorner commented Jul 7, 2023

new-it commented Jul 8, 2023

almenscorner commented Jul 9, 2023

new-it commented Jul 9, 2023 • edited Loading

almenscorner commented Jul 11, 2023

new-it commented Jul 12, 2023

almenscorner commented Aug 15, 2023 • edited Loading

almenscorner commented Sep 1, 2023

new-it commented Sep 15, 2023 • edited Loading

new-it commented Sep 26, 2023 • edited Loading

almenscorner commented Sep 26, 2023

new-it commented Sep 26, 2023 • edited Loading

almenscorner commented Sep 26, 2023

new-it commented Sep 27, 2023

almenscorner commented Sep 27, 2023

new-it commented Sep 29, 2023

new-it commented Oct 12, 2023

almenscorner commented Oct 12, 2023

new-it commented Oct 13, 2023

almenscorner commented Oct 13, 2023 • edited Loading

new-it commented Oct 23, 2023

new-it commented Jul 7, 2023 •

edited

Loading

new-it commented Jul 7, 2023 •

edited

Loading

new-it commented Jul 7, 2023 •

edited

Loading

new-it commented Jul 9, 2023 •

edited

Loading

almenscorner commented Aug 15, 2023 •

edited

Loading

new-it commented Sep 15, 2023 •

edited

Loading

new-it commented Sep 26, 2023 •

edited

Loading

new-it commented Sep 26, 2023 •

edited

Loading

almenscorner commented Oct 13, 2023 •

edited

Loading