SwiftTemplate fails to generate sometimes when swift/xcode print warnings to stdout

[liamnichols]

Background

While working on Xcode 13.3/StaticInternalSwiftSyntaxParser support, I noticed that the .swifttemplate generation started to become a lot more flakey than before and the errors weren't always obvious.. For example:

Using configuration file at '.sourcery.yml'
Scanning sources...
Found 438 types in 136 files, 57 changed from last run.
Loading templates...
Loaded 1 templates.
Generating code...
error: [1/1] Planning build
Building for debugging...
[1/40] Compiling SourceryRuntime Enum.swift
[2/40] Compiling SourceryRuntime Equality.generated.swift
[3/40] Compiling SourceryRuntime Extensions.swift
[4/40] Compiling SourceryRuntime FileParserResult.swift
[5/40] Compiling SourceryRuntime Generic.swift
[6/40] Compiling SourceryRuntime GenericRequirement.swift
[7/40] Compiling SourceryRuntime Import.swift
[8/40] Compiling SourceryRuntime JSExport.generated.swift
[9/40] Emitting module SourceryRuntime
[10/40] Compiling SourceryRuntime Log.swift
[11/40] Compiling SourceryRuntime Method.swift
[12/40] Compiling SourceryRuntime Modifier.swift
[13/40] Compiling SourceryRuntime PhantomProtocols.swift
[14/40] Compiling SourceryRuntime Coding.generated.swift
[15/40] Compiling SourceryRuntime Composer.swift
[16/40] Compiling SourceryRuntime Definition.swift
[17/40] Compiling SourceryRuntime Description.generated.swift
[18/40] Compiling SourceryRuntime Dictionary.swift
[19/40] Compiling SourceryRuntime Diffable.generated.swift
[20/40] Compiling SourceryRuntime Diffable.swift
[21/40] Compiling SourceryRuntime Documentation.swift
[22/40] Compiling SourceryRuntime AccessLevel.swift
[23/40] Compiling SourceryRuntime Annotations.swift
[24/40] Compiling SourceryRuntime Array+Parallel.swift
[25/40] Compiling SourceryRuntime Array.swift
[26/40] Compiling SourceryRuntime AssociatedType.swift
[27/40] Compiling SourceryRuntime Attribute.swift
[28/40] Compiling SourceryRuntime AutoHashable.generated.swift
[29/40] Compiling SourceryRuntime BytesRange.swift
[30/40] Compiling SourceryRuntime Class.swift
[31/40] Compiling SourceryRuntime Closure.swift
[32/40] Compiling SourceryRuntime Protocol.swift
[33/40] Compiling SourceryRuntime ProtocolComposition.swift
[34/40] Compiling SourceryRuntime Struct.swift
[35/40] Compiling SourceryRuntime Subscript.swift
[36/40] Compiling SourceryRuntime TemplateContext.swift
[37/40] Compiling SourceryRuntime Tuple.swift
[38/40] Compiling SourceryRuntime Type.swift
[39/40] Compiling SourceryRuntime TypeName.swift
[40/44] Compiling SourceryRuntime Typealias.swift
[41/44] Compiling SourceryRuntime Typed.generated.swift
[42/44] Compiling SourceryRuntime Typed.swift
[43/44] Compiling SourceryRuntime Variable.swift
[44/46] Emitting module SwiftTemplate
[45/46] Compiling SwiftTemplate main.swift
Build complete! (1.22s)

warning: Usage of /Users/<redacted>/Library/org.swift.swiftpm/collections.json has been deprecated. Please delete it and use the new /Users/<redacted>/Library/org.swift.swiftpm/configuration/collections.json instead.

This is confusing, because it's reported "Build complete! (1.22s)", but for some reason it failed and it wasn't really clear until I realised that the warning about collections.json is something that swift build is printing out to stderr and SwiftTemplate will see that as a failure based on this logic:

Sourcery/SourcerySwift/Sources/SwiftTemplate.swift

Lines 243 to 247 in 75e91b5

	if compilationResult.exitCode != 0 || !compilationResult.error.isEmpty {
	throw [compilationResult.output, compilationResult.error]
	.filter { !$0.isEmpty }
	.joined(separator: "\n")
	}

Now that I've also switched CI to use 13.3, I'm seeing some tests fail there too. For example: https://app.circleci.com/pipelines/github/krzysztofzablocki/Sourcery/467/workflows/8ac497af-d2fa-4963-bf8d-c9f5ecf865ce/jobs/2123

This time it appears to be different messages in stderr:

2022-03-23 13:58:54.153 xcodebuild[27208:59773] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionSentinelHostApplications for extension Xcode.DebuggerFoundation.AppExtensionHosts.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2022-03-23 13:58:54.153 xcodebuild[27208:59773] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionPointIdentifierToBundleIdentifier for extension Xcode.DebuggerFoundation.AppExtensionToBundleIdentifierMap.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2022-03-23 13:58:55.276 xcodebuild[27209:59780] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionSentinelHostApplications for extension Xcode.DebuggerFoundation.AppExtensionHosts.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2022-03-23 13:58:55.277 xcodebuild[27209:59780] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionPointIdentifierToBundleIdentifier for extension Xcode.DebuggerFoundation.AppExtensionToBundleIdentifierMap.watchOS of plug-in com.apple.dt.IDEWatchSupportCore

Since SwiftTemplate uses xcrun swift build, this means that anybody who has Xcode 13.3 set via Xcode-select will start seeing these issues more frequently.

Fixing the Issue

I guess there are two ways we can go about fixing this:

1. Only fail if the exit code is something other than 0 (ignore stderr in the check)
2. Whitelist some messages

I imagine that option 1 would be the easiest solution, but I'm not sure if there was a specific scenario where the exit code might be 0 but a genuine error was printed?

[krzysztofzablocki]

I think option 1 makes most sense to me 👍

@liamnichols would you like to get extra access / join as one of the core team for Sourcery? you contribution are very appreciated

[liamnichols]

Thanks for confirming!

@liamnichols would you like to get extra access / join as one of the core team for Sourcery? you contribution are very appreciated

Sure! We've only just started using Sourcery, so I have to learn about it in a bit more detail but I'd love to help out! Thanks!

[krzysztofzablocki]

@liamnichols can we close this now that #1037 is merged?

[liamnichols]

Yep, thanks for merging!