macOS 11 (Big Sur): Request to access Microphone

[chjan]

Tested from 11 of 20.11.20 to latest build DMG of 11.12.20. Same severe result:

Summary

Java apps on Big Sur are currently stuck IF system resources are used that are NOT defined as system default resource. One simple example is the built in mic of a MacBook Pro. If the mic is set to default in the system, Java can access it, else not.

The reason for this behaviour is that Big Sur requires the user to manually acknowledge "permission requests" sent by the apps. One example on an app that does this correctly is the Reaper DAW (WWW.REAPER.FM). One see the requests are lined up automatically as this app is installed. Look to Apple system preferences / security to observe this.

Java needs to do the same, or there must be an API that enables the app programmer to make such permission requests. This is really super high priority and a remedy should come in a short term release. At the moment, java apps on Big Sur are stuck (for the case that non-default resources of any kind is used).

Steps to reproduce

Just observe with mic as example that Java does not request permission, while Reaper (free to install) do request permission.

Expected results

Actual results

Triaging info

Java version:

What is your operating system and platform?

How did you install Java?

Did it work before?

Did you test with other Java versions?

[aahlenst]

Thanks for getting in touch. Your description is rather vague. It is unclear whether it is a problem of missing entitlements and it's also unclear whether some app has bundled Java or whether we're talking here about something that uses Java. If it's the latter, you need to raise this with OpenJDK, especially if new APIs need to be implemented.

Ideally, provide a small sample that reproduces the problem and describe the expected behviour.

[aahlenst]

Related: adoptium/temurin-build#1720

[chjan]

That's ok. I am not either, but this "lack of a mandatory feature" seems to me like nothing else if not a major showstopper, and it is kind of hard to find a place to report the problem. Your site focusing on AdoptOpenJDK... is one option, but if you look to Oracle with their commercial variants, where is the bug report feature? I am using their Facebook message feature and got some attention therein, however, it is an unsatisfactory situation.

So, I can attach a picture if you like. The picture tells you that the following applications, used as examples, do the right thing, because they request permission for a system resource, a microphone in this case. You can observe that more apps are lining up in compliance, only java has not grasped the new game.

What is not so obvious, but still true, is that when you as a user execute a java application as a .jar executable, by double-clicking it, neither the app (my own) nor the JDK/JRE is posing the mandatory request for (in this example) permission to use the microphone of the laptop. And this is mandatory due to the brand new ands built-in security mechanisms of the Apple OS, as I said: The user of the machine must make the choice to allow system resources to be used or exploited by this or that app that the user him/herself probably has started. This is not to be confused with the traditional setup of a mic inside a java application.

So then I repeat: The applications TERMINAL and REAPER are compliant to these new security mechanisms, and you can observe that these two apps already has provoked the user to "CHECK" the said allowance for microphones. And as I discuss this here, I in parallel install new stuff like PARALLEL and some EQ app. You see also that this is the "security" part of the "system preferences" of the Big Sur version of the Apple OS (i.e. latest version).

So the question you and we could ask, supposedly as entities that would like Java to work flawlessly also on an Apple computer, is this: Why does the JDK/JRE not pose that question for mic allowance?

To our knowledge, there is no new "api" that would make it possible to initiate such permission request from the applications, so it must be done by those working with the R&D of the JDK/JRE, and sorry for capital letters: WHEREVER ANYONE WORKS WITH JAVA ON APPLE OPSYS, including but not limited to you and your team.

[chjan]

I must add one more info: When the allowance is not yet granted by the user, there is no error report, no error notification, no alarm, nothing. It just stops working, thus creating havoc. Mic does not work unless it is currently the default audio input device. This is for java. For Reaper, due to allowance granted, it works. It is quite normal for some applications that the mic is not currently default (due to something called audio routing, which its a bit advanced).

And even more: One could imagine that the operating system would have been so kind as to ask the user this vital question for allowance, but as you can observe in the screenshot above, no such option is available to manually choose additional apps. Therefore, the whole scenario is a showstopper at the moment.

[aahlenst]

You write that you're having problems with your own app, so do I assume correctly that you are a Java developer? If yes, can you write a main() program that accesses the microphone and does not work? Can you paste that here?

When you start your application in Terminal, do you get any output there from the system or JDK? Can you have a look into macOS's system console whether you can get any info there? Can you paste it here? Does it look like https://bugs.openjdk.java.net/browse/JDK-8244951?

Can you try with the macOS build from http://jdk.java.net/17/? Does it fail, too?

Do you have a link to the Apple documentation that describes exactly what kind of API needs to be implemented?

[chjan]

JDK17? Still not working, but thanks for the link with clues regarding how to file bugs. I was at oracle Java pages and no link to that JDK site there.

Yes, it kind of resembles that bug linked, but not bulls eye as I understand it. My report is talking about a general problem in how the whole Java community handles the new security mechanisms in Apple Op-sys.

I do not have link to Apple documentation. Although our Java product SoundPimp has been on the market for a decade, customers on Apple are not that many, so we are not experts on Apple.

I attach two more screenshots. One showing that there is a "developer mood" identified in the Apple security setup, however, it does not work as remedy for this problem. Secondly, you also see that SOME of the lines in the system resource selector has the + - option to add applications that shall be allowed. And then you see I have added Java JDK from the /bin folder. Not working. But you have also seen that some system resources, like mic, does not have that + - option. I have no idea why.

[tresf]

It is unclear whether it is a problem of missing entitlements

The entitlement appears to be called NSMicrophoneUsageDescription. This was previously an iOS-only feature which is why it's hard to find the exact documentation link for macOS (at the time of writing this, most search results land on the wrong docs). OpenJDK seems to already have this entitlement in JDK16 and JDK11, so I don't think there's any changes needed. Please correct me if this is wrong.

if new APIs need to be implemented.

It appears to be called AVCaptureDevice authorizationStatus(for:) [api mirror]. Since this is new with 10.14, openjdk will have to be enhanced with this functionality (assuming .plist changes won't make a difference ).

switch AVCaptureDevice.authorizationStatus(for: .video) {
    case .authorized: // The user has previously granted access to the camera.
        self.setupCaptureSession()
    
    case .notDetermined: // The user has not yet been asked for camera access.
        AVCaptureDevice.requestAccess(for: .video) { granted in  // change to audio, I assume?
            if granted {
                self.setupCaptureSession()
            }
        }
    
    case .denied: // The user has previously denied access.
        return

    case .restricted: // The user can't grant access due to restrictions.
        return
}

Unlike other security features in macOS, it appears there's no manual workaround. Testing on Big Sur, the Microphone area has no option to manually add an application, so I would make an educated guess that the missing API is a hard-stop.

Note, testing on an M1, I can't get access to the Microphone at all, which contradicts the OP's "Default device" observations. This may be a difference in default behavior between Intel and M1 to ease the transition into a stricter security model, but that's speculative.

Because I can't find a way to obtain any input from the Microphone at all, I have no viable code to share. On the M1, the OS seems to simply block access to the device. I can only list details about playback devices, not Microphones or input devices.

[tresf]

@chjan can you please change the title to be more descriptive, e.g. "Request to access Microphone"? Oddly, I have code which reads the screen and the prompt raises just fine, I assume because the Security manager has an option to raise this request so blindly calling this "system resources" is too vague and should be more specific. Also, words like MAJOR should be removed (subjective) and instead the maintainers can set priority through their own triaging efforts (which is volunteer, mind you).

[aahlenst]

So, AVCaptureDevice isn't known to OpenJDK: https://bugs.openjdk.java.net/issues/?jql=text%20~%20%22AVCaptureDevice%22. I also read through a ton of issues with the term "Microphone". https://bugs.openjdk.java.net/browse/JDK-8212558 is the closest but predates Catalina and Big Sur. So someone needs to report that upstream. Everybody can do that on https://bugreport.java.com/bugreport/, but please read https://docs.oracle.com/en/java/javase/15/troubleshoot/submit-bug-report.html beforehand. Sample code to reproduce the problem is mandatory. Otherwise it will be closed right away.

@chjan Maybe a crazy idea, but worth a try: Depending on your program flow, you could try calling into native macOS APIs with JNA to trigger the permission dialog.

[kovdb]

We had a similar issue with our java software.

We were not requesting access to a microphone, but to another application. This also needs to be granted first in the Automation section of the Privacy tab.

The access was not requested by macOS and it consequently did not work. We could fix this by packaging our application in a proper Application Bundle with a proper plist, application icon, correct folder structure...

Once we did this, macOS was requesting proper access on first use and our application was added automatically to the list in the Automation section (with the correct icon and application name).

[chjan]

Have you tried to run your test mic class as an executable jar

I've created an Application called SoundRecorder.app which is basically a stub to java -jar SoundRecorder.jar and no dialog is raised, a sound file is created where the WAV should be, but it contains no sound.

I haven't added entitlements (my experience says they shouldn't be needed unless distributing through the App Store -- since they're generally reserved only for sandboxed applications) but I'm not afraid to if needed.

I've found a lot of people complaining about this exact problem, but no concrete answers:

• https://stackoverflow.com/questions/55727488/ "Application doesn't ask for permission to access microphone in MacOS 10.14 Mojave"
• https://stackoverflow.com/questions/58728150"How to give Microphone permission to Java application (macOS Mojave and above)?"
• https://stackoverflow.com/questions/56084303"OpenCV command line app can't access camera under macOS Mojave"
• https://stackoverflow.com/a/66210338/3196753"Cannot Access Microphone on Mac Mojave using pyaudio"
• https://stackoverflow.com/questions/61989711"How to record microphone on macos with pyaudio?"

... and a long thread about this exact same problem for VSCode users: microsoft/vscode#95062 (still no solution). I believe PyCharm fixed it by requesting access from the macOS libraries directly, something that can be done with a 3rd-party Java tool, JNA.

I'll keep digging.

Tres, please understand, I have given the correct and mandatory things that needs to be done with a stub as you call it (an javaapp.app folder containing a lot of subfolders). In short: Entitlements and signing. See above.

I dont understand what you mean when you say that entitlements are included in the jdk? Not at all. Entitlements are something you define ONLY WHEN you create your stub, through java packager or (hopefully) some much better alternative.

I say again: Try your test.jar executable by double clicking it in its folder. Predicted result: It will not get anything else but silence out of the input stream. Now try to run it in Apple's terminal app: Predicted result: MacOs BigSur will ask you "Do you want Terminal to be allowed to access mic?". You confirm, and then your app should work. In other words, when you start a .jar executable from Terminal, then Terminal is allowed to access mic. It is not Java that is granted this allowance then, it is the Terminal app. See?

So now, to create a commercial product for MacOs, we of course need to package the jar executable according to rules on MacOs. And this is to create an app "stub" as you call it.

Now we are heading for trouble because jpackager (java 14 and onwards) is a completely rotten piece of sw that will require its user to really know the details about what to write and not write in order to make it work. Then you need a Apple developer user to sign it, and there you go. This is the current situation, that this is EXTREMELY COMPLEX.

And therefore, the "java community" must liten to me when I state that it will be mandatory to refurbish jpackager into something much better, and with that I mean that jpackager should not be a terminal app, but a gui app that with a teaspoon describes the options the user has or must use. For example in my case, the audio input entitlement.

Am I getting through? I will kindly request that projectleaders of this thread step forward and make an intellgent remark about what to do. Because, clearly, someone must to something to improve this fatal situation. (Not that it is impossible to succeed, but you need to read both this and that blog to be successful. But "package for store" should NOT be difficult, it should be easy)

[tresf]

The entitlements are baked into the binary file itself. This post shows the binary file viewed in a hex editor:

#222 (comment) (click the link to expand the XML).

The apps that work seems to have this in common. More investigation to come.

[tresf]

aren't we all engaged in ensuring that Java survives on Mac computers?

This is much larger than Java. VSCode hasn't figured this out either and they do NOT use Java.

[tresf]

Repeating what was already mentioned in case it's gotten lost:

I've created a small, stub launcher which reproduces the original bug (a Java app won't ask for Microphone permissions).

I'll keep working from this example moving forward until I have a workable Application bundle which prompts for access. This will be a lot of trial and error until it works, some of which may involve invoking the Microphone APIs directly (which I believe is how PyCharm got around this issue).

I still see no evidence this is a Java-specific bug, but I do thanks @aahlenst for reopening as I think the finding will be valuable for macOS Desktop developers at large.

[github-actions]

We are marking this issue as stale because it has not been updated for a while. This is just a way to keep the support issues queue manageable.
It will be closed soon unless the stale label is removed by a committer, or a new comment is made.

[karianna]

@chjan Is this still an issue for you?

[karianna]

There is apparently a related fix in 17.0.2 which may help.

[github-actions]

We are marking this issue as stale because it has not been updated for a while. This is just a way to keep the support issues queue manageable.
It will be closed soon unless the stale label is removed by a committer, or a new comment is made.