[makeotf] macOS 10.15 issue

[frankrolf]

I am currently poking around with the macOS 10.15 beta – in it, it no longer seems possible to run makeotf in its current configuration:

➜  Desktop makeotf -f test.ufo
tx: --- /Users/fg/Desktop/test.ufo
tx: (ufr) Warning: No version specified in source UFO font.
makeotf [Warning] Could not find default features file. Font will be built without any layout features.
makeotfexe [FATAL] file error <No such file or directory> [../../../var/folders/yl/cz8lx5qn6ps_4pw00l8gtm9c0000gp/T/tmpm9kopaya]
makeotf [Error] Failed to build output font file 'test-test.otf'.

however, this works:

➜  Desktop sudo makeotf -f test.ufo
Password:
tx: --- /Users/fg/Desktop/test.ufo
tx: (ufr) Warning: No version specified in source UFO font.
makeotf [Warning] Could not find default features file. Font will be built without any layout features.
makeotfexe [NOTE] Adding glyph .notdef, SID 0.
Built development mode font 'test-test.otf'.

I don’t know what mechanism expects a file in the var folder, I also don’t know which steps need to be taken to make makeotf behave as it used to, but I assume it’s good to know about this change.

[readroberts]

What's happening is that makeotfexe still requires a Type 1 font as input, so makeotf.py converts any other kind of input using 'tx'. The temporary Type1 font is written to a location under '/var', determined by the call to tempfile.mkstemp() inside fdkutils.py::get_temp_path(). I have read that Catalina, 10.15, has upped the security requirements, and that the system root directory is now in a separate partition and is read-only to regular users. I have some trouble with this, as if true, it would break all the python tempfile methods under 10.15, and I can't see any complaints on the web. Can you create a file in the directory '/var/folders', without root access?

[benkiel]

Also, which python are you using? User installed Py3?

[frankrolf]

@readroberts

➜  ~ touch /var/folders/test
touch: /var/folders/test: Permission denied

@benkiel
Yes, I am using Python 3 installed via Homebrew. Our team has dropped the built-in Python a while ago, and I’ve also been teaching students to not mess with that and install their own Python3 instead.

[frankrolf]

@readroberts However, those both work:

➜  ~ touch /tmp/test
➜  ~ touch /var/tmp/test

[cjchapman]

@readroberts From what I can tell, it is not a permissions issue, it's a path formation issue.

The temporary file seems to be getting created with correct permissions, the problem is that something in the Python code is prepending all those "../../.." to the path incorrectly, so the C code is seeing an incorrect relative path rather than the correct absolute path.

[cjchapman]

@readroberts & @frankrolf I think something's going awry with how makeRelativePaths() makes relative paths to the temporary area. However, it's not clear to me that they even need to be relative paths.

I'm away from the macOS 10.15 trash can Mac Pro that I borrowed, so I can't fully test this at the moment, but Frank if you have a chance, try making this change in makeRelativePaths() in makeotf.py:

    # fmndbFilePath = makeRelativePath(
    #     fontDir, getattr(makeOTFParams, kFileOptPrefix + kFMB))
    fmndbFilePath = getattr(makeOTFParams, kFileOptPrefix + kFMB)

and just comment out these lines in that same function:

    # if makeOTFParams.tempFontPath:
    #     makeOTFParams.tempFontPath = makeRelativePath(
    #         fontDir, makeOTFParams.tempFontPath)

Read, can you confirm if a change along these lines is OK in general, or do they really need to be relative paths?

[readroberts]

No they don't need to be relative paths. The reason for the relative path is that the current paths get recorded in the project file as a default, and the relative paths are usually shorter.

However, I am still not convinced that the relative path is the problem. I run the same command under 10.14, and it works, but also generates the same relative path.
The original is:
/var/folders/d3/yxwv5yn5715fk31lq36lnk780000gn/T/tmpt5t579li
The current directory is: /Users/rroberts/Desktop
the relative path is:
../../../var/folders/d3/yxwv5yn5715fk31lq36lnk780000gn/T/tmptuhf45tp

Note that /Users/rroberts/Desktop + ../../../var/folders/d3/yxwv5yn5715fk31lq36lnk780000gn/T/tmptuhf45tp is correct.

I'm all for Frank making the change you suggest. I suspect that the absolute path is the problem.

[cjchapman]

@readroberts OK, I see your point about the relative path. I thought I got an incorrect relative path when I did an experiment on a macOS 10.15 machine, but perhaps I was confused about which directory it should be relative to (and I was surprised that it was a relative path in the first place).

@frankrolf One thing you could try even before (and also after) the code change experiment I suggested above is to run makeotf -V -f test.ufo where the -V verbose option will show you the paths that are being passed to makeotfexe. Then use something like ls -l <path> to confirm if the paths exist and have the appropriate permissions.

I plan on digging into this further when I get back to the office and can work on the macOS 10.15 machine.

[cjchapman]

@frankrolf You could also use the makeotf -V output as a starting point for calling makeotfexe directly from the command line, e.g. you could try relative vs. absolute paths yourself directly.

[cjchapman]

@readroberts OK, it looks like we're both sort of right -- the relative path is correct in theory, however it doesn't work in practice on OS 10.15:

cchapman@ariza-osx input % makeotf -V -f trademark.ufo                                                           
makeotf [Warning] Could not find default features file. Font will be built without any layout features.
makeotf [Note] Running makeotfexe with commands:
   cd "/Users/cchapman/GitHub/afdko/tests/tx_data/input"
   makeotfexe "-f" "../../../../../../../var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmpj80el380" "-o" "SourceSansPro-Regular.otf" -V -mf "../../../../../../../var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmp4atz_bbl" -shw
makeotfexe [FATAL] file error <No such file or directory> [../../../../../../../var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmp4atz_bbl]
makeotf [Error] Failed to build output font file 'SourceSansPro-Regular.otf'.
cchapman@ariza-osx input % pwd   
/Users/cchapman/GitHub/afdko/tests/tx_data/input
cchapman@ariza-osx input % ls -l ../../../../../../../var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmpj80el380
ls: ../../../../../../../var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmpj80el380: No such file or directory
cchapman@ariza-osx input % ls -l /var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmpj80el380 
-rw-------  1 cchapman  staff  3679 Jul 11 16:13 /var/folders/37/fssc5g6911x32c4l18pjjyhc0000gr/T/tmpj80el380

I also tried this:

cchapman@ariza-osx ~ % pwd    
/Users/cchapman
cchapman@ariza-osx ~ % ls ../../var
ls: ../../var: No such file or directory
cchapman@ariza-osx ~ % ls ../..
Applications	Users		home		private		usr
Library		Volumes		mnt		sw
System		cores		opt		tmp
cchapman@ariza-osx ~ % ls /var
McAfee		backups		jabberd		mail		rpc		tmp
_secmgt		db		lib		msgs		run		vm
agentx		empty		localadmin	netboot		rwho		yp
at		folders		log		networkd	select
audit		install		ma		root		spool
cchapman@ariza-osx ~ % 

I get the same results with bash as with zsh.

[cjchapman]

So... it seems we have to use an absolute path to get to anything under /var on macOS 10.15.

[readroberts]

Actually, it makes sense to use an absolute path for the temp files. Makeotf originally wrote the temp files locally to the current directory, and then relative paths made the string a lot shorter, as a tiem when we used to have problems with path length. Now that the temp files are deep under /var, with no part of the path in common with the current directory, there isn't any real advantage to using relative paths.

[frankrolf]

Confirmed working on my end.
Thank you!

[cjchapman]

@frankrolf You're welcome. Thanks for the confirmation!

@blueshade7 Thanks for loaning me your Mac Pro with the macOS 10.15 beta!