zig runtime configuration options: zig config and how it relates to zig-cache location

[mikdusan]

long term proposal

Once Zig is installed, a user will need various configuration options. Think git config-like ability. This will define consistent and flexible behaviour for zig-cache directory location.

Currently zig-cache is located in one of:

• shell current working directory
• user-account application-support location for Zig that is platform-specific

Propose that we steer Zig in direction of familiar configuration mechanism of git config. Location options for zig-cache can be:

• dir → example: /tank01/zig-cache
• pwd → example: $PWD/zig-cache
• tmp → example: /var/folders/xp/s2xy5w692315r6rmm8_30v3m0000gp/T/zig-cache
• local → example: /User/mike/project/.zig/cache
• global → example: /User/mike/Library/Application Support/Zig/cache

here is a sample of what kinds of things could be configured:

zig config cache.dir local
zig config color.ui on
zig config link.verbose true
zig config cc.verbose true
zig config fmt.style canonical
zig config fmt.silence_error true

note: any child zig executions launched in a different pwd wishing to use the same work-tree would be launched with --work-tree

The following are configuration-points selected with zig config command line options. Each represents a distinct config file from which Zig draws runtime behaviour where settings are resolved in a find-first search-order { local, global, global-app, system }.

--local → work-tree settings (DEFAULT for zig config)

The work-tree is found by walking up the filesystem tree and checking if .zig directory exists. Search beings with shell current working directory. If none is found then shell current working directory is used. If a file is found instead of a directory, it is completely ignored. example config file pathnames:

• /User/mike/project/hello_world/.zig/config

--global → home settings

Settings are stored in locations private to a single user account. These locations are based on platform convention for "dot files" in home directories. For platforms where dot files are not applicable then this convention vanishes, and --global-app takes its place. examples config file pathnames:

• /users/mike/.zigconfig linux
• /Users/mike/.zigconfig macOS

note: for platforms where dot files are not applicable then --global is replaced with --global-app semantics --global-app command-line option vanishes

--global-app → home app area settings

If a user wants to keep settings adjacent to other things like zig-cache. example config file pathnames:

• /users/mike/.local/share/zig/config → linux
• /Users/mike/Library/Application Support/Zig/config → macOS

--system → zig install settings

Zig knows how to determine its install location. From install location a settings location is derived. examples config file pathnames:

• /etc/zig/config → core bundling with an OS
• /usr/local/etc/zig/config → add to an OS
• /Applications/Zig.app/Contents/Zig/etc/config → macOS

[hryx]

Interesting idea! It looks like there are actually multiple proposals here:

• what to do about zig-cache
• more configuration options which could be added in the future
• a tool/system for statefully setting these options

One note about the example:

zig config fmt.style canonical
zig config fmt.silence_error true

zig fmt won't be configurable (needs to be added to a FAQ). And I'm not sure what silencing errors would do in this case; zig fmt should either succeed or fail.

I have a couple issues with the idea in general:

• The spirit of zig is to minimize configurability so that the system comes closer to "one way to do things (the right way)". So in the ideal world, there is no configuration at all.
• When compiler options are needed, there are command line flags. These are not only explicit and visible, but they do not depend on the system's current state. I imagine that @andrewrk, who uses NixOS, would lean this direction.

Your analogy is git config, but is there an example of a programming language compiler that does this in general?

[mikdusan]

• what to do about zig-cache
• more configuration options which could be added in the future
• a tool/system for statefully setting these options

yes! Zig is fairly sophisticated and I agree that over-configuration is something to be avoided. That said, zig-cache is a perfect example of something that is tough to decide where it goes but I know it doesn't go where I want it to go today. One of the last places I wanted it to go was actually hidden away like Xcode.app does in some application support folder. And yet there it was. I almost always want my build output, including caches, to go in a build directory even if it means copies of cache entries. And putting it in $PWD/zig-cache is convenient but probably a bit extreme for my preference. Very subjective stuff.

A very common course of evolution in software is to keep adding ENV options for configuration. ENV has its place but can get unwieldy very quickly. Is Zig going to have a zig-pkgs.conf? or a Zig server/agent , so zig-server.conf? I'm pretty sure more things will come up.

zig config fmt.style canonical
zig config fmt.silence_error true

zig fmt won't be configurable (needs to be added to a FAQ). And I'm not sure what silencing errors would do in this case; zig fmt should either succeed or fail.

not suggesting we add those features. but they did come to mind because right now (or it did a week ago) zig fmt fails on top-level fields so I have to take zig out of my path for a vi-session to work around it. again not suggesting it's worth a change, but my pref would be "ok if fmt can't fmt, then be quiet and don't mess with my vi-session. I get there are other work arounds, like fixing vi to tolerate zig fmt failure. and I'm not aware of any features to mark source ranges off-limits to fmt. I have some bit fiddling code fragments that are so much more readable manually formatted, and would never expect fmt to get it right. but do expect there to be a way for format to be hands-off.
I acknowledge there is a good medium ground here. it's possible to have off-limits ability AND possible to have one-style only which is what I gather from most of your meaning. I'm on board with that. maybe fmt.style was a bad choice for example but really just wanted to show zig config usage and these things popped into mind.

I have a couple issues with the idea in general:

• The spirit of zig is to minimize configurability so that the system comes closer to "one way to do things (the right way)". So in the ideal world, there is no configuration at all.

I like this too. If it can be made to work like that excellent. I only want config options when they have enough need-pressure and are not over-engineered. Sorry if some of my examples appeared over-engineered; but I don't want that to detract from the main prediction that there are going to be a number of configuration options and we should deeply consider git config approach before an over abundance of ENV options and many feature-specific .conf files emerge.

Your analogy is git config, but is there an example of a programming language compiler that does this in general?

I cannot think of any programming language compiler that take a git config approach. But look to ccache as an example of hashing compiler output, tracking it, several levels of configuration including ENV and .conf files at multiple layers (similar to --global and --system). but if memory serves, the .conf files are hand-edited without cli convenience.

related to zig-cache, the future might bring some automagical management for cache sizes, and such. again, these settings are so subjective they cannot be hard-coded. if/when we get there, where would such settings reside?

[andrewrk]

Some notes about zig-cache:

There are 2 zig-cache directories. One is project-local. One is global. The global one makes sense to be global because it has builtin.a, compiler-rt.a, musl libc.a, builtin.zig, etc, built from source lazily for the target, and will be shared by the same zig compiler version for all projects. It saves a lot of accumulated time.

The project-local one is by default in the same directory as build.zig, in the root of the project directory. It uses the same caching system but has project-local artifacts in it. The project-local directory is also available for general use by users as a sort of "scratch directory" as long as the caching system files are not disturbed.

Zig does not use any system temporary directories, and I don't think there is any reason to.

zig-cache is a perfect example of something that is tough to decide where it goes but I know it doesn't go where I want it to go today.
Very subjective stuff.

I don't agree with either of these statements. Can you point to any specific file on your file system (inside one of the zig-cache folders) and tell me why it doesn't belong there? Has the caching system caused a specific problem for you?

[hryx]

@mikdusan Awesome, thanks for the thoughtful reply. I think what would be helpful for me is if this were split into two separate proposals: a) a zig cache config proposal, and b) a config system/tool proposal for the long term, each with its own justification / use case / problem demonstration. I think they're truly unrelated issues.

before an over abundance of ENV options and many feature-specific .conf files emerge.

Agreed that should be avoided! But the thing is that there currently aren't any environment variables or conf files, so I have a hard time worrying about an overabundance. :>

[tiehuis]

I'm not aware of any features to mark source ranges off-limits to fmt.

This works:

// zig fmt: off
test     "this is left alone"  {   }
// zig fmt: on

This is undocumented though, so I'm not surprised you missed this. See #1523.

[mikdusan]

The global one makes sense to be global because it has builtin.a, compiler-rt.a, musl libc.a, builtin.zig, etc, built from source lazily for the target, and will be shared by the same zig compiler version for all projects. It saves a lot of accumulated time.

that is really cool. I'll have to think hard on how that impacts what I'm proposing for zig config.

The project-local one is by default in the same directory as build.zig, in the root of the project directory. It uses the same caching system but has project-local artifacts in it. The project-local directory is also available for general use by users as a sort of "scratch directory" as long as the caching system files are not disturbed.

What about non-build.zig related commands? This is where zig-cache has a tendency to get created in pwd.

zig-cache is a perfect example of something that is tough to decide where it goes but I know it doesn't go where I want it to go today.
Very subjective stuff.

I don't agree with either of these statements. Can you point to any specific file on your file system (inside one of the zig-cache folders) and tell me why it doesn't belong there? Has the caching system caused a specific problem for you?

I would like for zig to infer work-tree and use $WORKTREE/zig-cache as the location for all zig exec where pwd is inside $WORKTREE.

today these are a few cases showing $PWD/zig-cache popping up while I'm cd'ing and vi'ing all over the place in a project.

creates $WORKTREE/zig-cache with fmt and fmt.o entries:

cd $WORKTREE
vi main.zig

creates $WORKTREE/sub/zig-cache with fmt and fmt.o entries:

cd $WORKTREE/sub
vi ../main.zig

creates $WORKTREE/sub/zig-cache with native_libc.txt:

cd $WORKTREE/sub
zig build-obj foo.zig

[mikdusan]

closing this issue as it seems like a solution for a non-existing problem. thank you everyone for taking the time to reply.

[timotheecour]

@mikdusan, I really like nim's way to handle configuration via nims files (eg in ~/.config/nim/config.nims) which allows customizing compiler. So what's the conclusion here, how can I customize where zig-cache goes?

[Tetralux]

@timotheecour You can pass --cache-dir to zig.exe.

[timotheecour]

that's what I was afraid of, can we re-open this feature request?
the point of a user config file is that you don't have to modify any script/program invoking zig for it to apply. There's no way to pass a cmdline argument to all zig invocations in a complex program (eg it could be a compiled binary etc).

like I said, nim has ~/.config/nim/config.nims, and a ton of other programs have their config files as well, eg:
git, tig, tmux etc. Zero-config is a great idea in theory but in practice it implies suboptimal one-size-fits-none