Float16 Not Available Error

[fdchiu]

when trying to compile and run through bazel according to your instruction on MacBook Pro M1 with Xcode 13.4 and swift 5.4, I was getting Float16 not available for macOS error.

There are a bunch of error related to Float16:

---------- other log's ignored before this line ----------
ERROR: /Users/david/Project/DALL-E/swift-diffusion/examples/BUILD.bazel:102:13: Compiling Swift module //examples:txt2img failed: (Exit 1): worker failed: error executing command bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/darwin-opt/bin/examples/examples_txt2img.swiftmodule-0.params
error: emit-module command failed with exit code 1 (use -v to see invocation)
examples/txt2img/main.swift:6:37: error: 'Float16' is unavailable in macOS
public typealias UseFloatingPoint = Float16
^~~~~~~
Swift.Float16:4:23: note: 'Float16' has been explicitly marked unavailable here
@Frozen public struct Float16 {
^
examples/txt2img/main.swift:121:17: error: global function 'CLIPTextModel(:vocabularySize:maxLength:embeddingSize:numLayers:numHeads:batchSize:intermediateSize:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
let textModel = CLIPTextModel(
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/src/CLIPTextModel.swift:71:13: note: where 'T' = 'UseFloatingPoint' (aka 'Float16')
public func CLIPTextModel<T: TensorNumeric>(
^
examples/txt2img/main.swift:135:42: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
let casualAttentionMask = graph.variable(Tensor(.CPU, .NHWC(1, 1, 77, 77)))
^
examples/txt2img/main.swift:139:57: error: type 'UseFloatingPoint' (aka 'Float16') has no member 'greatestFiniteMagnitude'
casualAttentionMask[0, 0, i, j] = -UseFloatingPoint.greatestFiniteMagnitude
~~~~~~~~~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~
examples/txt2img/main.swift:161:17: error: cannot infer contextual base in reference to member 'CHW'
).reshaped(.CHW(2, 77, 768))
~^~~
examples/txt2img/main.swift:162:13: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
let x_T = graph.variable(.GPU(0), .NCHW(1, 4, startHeight, startWidth), of: UseFloatingPoint.self)
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:165:13: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var xIn = graph.variable(.GPU(0), .NCHW(2, 4, startHeight, startWidth), of: UseFloatingPoint.self)
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:167:44: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
unet.compile(inputs: xIn, graph.variable(Tensor(from: ts)), c)
^
examples/txt2img/main.swift:173:33: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
var oldDenoised: DynamicGraph.Tensor? = nil
^
examples/txt2img/main.swift:183:28: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
let t = graph.variable(Tensor(from: ts))
^
examples/txt2img/main.swift:189:20: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var etUncond = graph.variable(
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:191:18: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var etCond = graph.variable(
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:6:37: error: 'Float16' is unavailable in macOS
public typealias UseFloatingPoint = Float16
^~~~~~~
Swift.Float16:4:23: note: 'Float16' has been explicitly marked unavailable here
@Frozen public struct Float16 {
^
examples/txt2img/main.swift:121:17: error: global function 'CLIPTextModel(:vocabularySize:maxLength:embeddingSize:numLayers:numHeads:batchSize:intermediateSize:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
let textModel = CLIPTextModel(
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/src/CLIPTextModel.swift:71:13: note: where 'T' = 'UseFloatingPoint' (aka 'Float16')
public func CLIPTextModel<T: TensorNumeric>(
^
examples/txt2img/main.swift:135:42: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
let casualAttentionMask = graph.variable(Tensor(.CPU, .NHWC(1, 1, 77, 77)))
^
examples/txt2img/main.swift:139:57: error: type 'UseFloatingPoint' (aka 'Float16') has no member 'greatestFiniteMagnitude'
casualAttentionMask[0, 0, i, j] = -UseFloatingPoint.greatestFiniteMagnitude
~~~~~~~~~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~
examples/txt2img/main.swift:161:17: error: cannot infer contextual base in reference to member 'CHW'
).reshaped(.CHW(2, 77, 768))
~^~~
examples/txt2img/main.swift:162:13: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
let x_T = graph.variable(.GPU(0), .NCHW(1, 4, startHeight, startWidth), of: UseFloatingPoint.self)
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:165:13: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var xIn = graph.variable(.GPU(0), .NCHW(2, 4, startHeight, startWidth), of: UseFloatingPoint.self)
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:167:44: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
unet.compile(inputs: xIn, graph.variable(Tensor(from: ts)), c)
^
examples/txt2img/main.swift:173:33: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
var oldDenoised: DynamicGraph.Tensor? = nil
^
examples/txt2img/main.swift:183:28: error: type 'UseFloatingPoint' (aka 'Float16') does not conform to protocol 'TensorNumeric'
let t = graph.variable(Tensor(from: ts))
^
examples/txt2img/main.swift:189:20: error: instance method 'variable(::of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var etUncond = graph.variable(
^
/private/var/tmp/bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
examples/txt2img/main.swift:191:18: error: instance method 'variable(:_:of:)' requires that 'UseFloatingPoint' (aka 'Float16') conform to 'TensorNumeric'
var etCond = graph.variable(
^
/private/var/tmp/_bazel_david/dbc2f5fdbf3e5db2dfb4ac1d16060e30/execroot/main/external/s4nnc/nnc/DynamicGraph.swift:611:15: note: where 'Element' = 'UseFloatingPoint' (aka 'Float16')
public func variable<Element: TensorNumeric>(
^
swift_worker: Could not copy bazel-out/darwin-opt/bin/_swift_incremental/examples/txt2img_objs/txt2img/main.swift.o to bazel-out/darwin-opt/bin/examples/txt2img_objs/txt2img/main.swift.o (No such file or directory)
Target //examples:txt2img failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 164.415s, Critical Path: 34.07s
INFO: 258 processes: 54 internal, 198 darwin-sandbox, 6 worker.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

[liuliu]

You are on Intel based macOS which doesn't support Float16. It is likely this machine if compiled successfully won't be able to run the model on GPU neither though. It will compile for M1 / arm64 macOS.

[fdchiu]

I was actually compiling and trying to run it on a Macbook Pro M1 machine. Do you have any idea what's missing here? Also if the package would work for iOS (assuming build and compile for iOS target is possible)? BTW, I followed your steps in Readme and it finished compiling all the dependencies (I can see the lib*.a files) but the problem happened with text2img compiling. Thanks for your help! David

…

On Sun, Dec 4, 2022 at 6:34 PM Liu Liu ***@***.***> wrote: You are on Intel based macOS which doesn't support Float16. It is likely this machine if compiled successfully won't be able to run the model on GPU neither though. It will compile for M1 / arm64 macOS. — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKEYUJ5VXVS5YRPAZU2DMLWLVICPANCNFSM6AAAAAASTW5QNM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rodrigo79rph]

I was able to compile and run the examples on a Mackbook Air M1, machine. I would also like to compile them and run them on a Macbook Pro Intel Core i7, @liuliu can you please share how I can replace the Float16 to make it work as I know you were workin on an Intel Mac before? Thank you. Also is there any documentation you recommend I can follow to compile and deploy to iOS? Thanks for your help!
Rodrigo

[liuliu]

@fdchiu sorry somehow missed this in the pyramid of emails. The most likely cause of your problem is because either clang or your Xcode is running on Rosetta mode, therefore, even if it is in M1, it compiles to x86_64.

I was able to compile and run the examples on a Mackbook Air M1, machine. I would also like to compile them and run them on a Macbook Pro Intel Core i7, @liuliu can you please share how I can replace the Float16 to make it work as I know you were workin on an Intel Mac before?

There is a UseFloatingPoint typealias on top of examples/txt2img/main.swift, you can switch that to Float32 to use on Intel. But it is slow (I remember it is around 15 minutes per image).

[fdchiu]

It's working now! I think the issue is more about how the bazel version is installed. I removed the bazel installed by following their manual process for a macbook install. Instead, I used homebrew install which installs the right flavor for Mac M1, while the manual process provided by bazel team on their web site suited for x86 based macbook. Maybe adding one line in readme about the bazel installation using homebrew instead of the manual process can help others to avoid the pitfall. By the way, will you provide an iOS framework version in the future? I would be highly interested and can also help. Thanks for your help and reply!

…

On Fri, Dec 9, 2022 at 11:19 AM Liu Liu ***@***.***> wrote: @fdchiu <https://github.com/fdchiu> sorry somehow missed this in the pyramid of emails. The most likely cause of your problem is because either clang or your Xcode is running on Rosetta mode, therefore, even if it is in M1, it compiles to x86_64. I was able to compile and run the examples on a Mackbook Air M1, machine. I would also like to compile them and run them on a Macbook Pro Intel Core i7, @liuliu <https://github.com/liuliu> can you please share how I can replace the Float16 to make it work as I know you were workin on an Intel Mac before? There is a UseFloatingPoint typealias on top of examples/txt2img/main.swift, you can switch that to Float32 to use on Intel. But it is slow (I remember it is around 15 minutes per image). — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKEYUNHIDHW2WQA223HUTDWMOA5PANCNFSM6AAAAAASTW5QNM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[weiyanlin117]

I experienced exact same issue. My macbook is M1 Pro and my Xcode is not in Rosetta.
After a little bit investigation, I notice somehow swift_binary always generate params for bazel on Macbook as

-target
x86_64-apple-macosx13.3
-sdk
__BAZEL_XCODE_SDKROOT__
-F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks
-I__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/usr/lib
-emit-object
-output-file-map
bazel-out/darwin-opt/bin/examples/txt2img.output_file_map.json
-Xfrontend
-no-clang-module-breadcrumbs
-emit-module-path
bazel-out/darwin-opt/bin/examples/examples_txt2img.swiftmodule
...

with -target x86_64-apple-macosx13.3 , although I am in arm64, swiftc is still compiling with x86_64 config. which will make float16 failed.

xcrun swift --version
swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-5.8.0.124.2 clang-1403.0.22.11.100)
Target: arm64-apple-macosx13.0

looks good

my bazel is install throw homebrew as

bazel --version                                                                                                                                             
bazel 6.1.1-homebrew

I suspect some issues from swift_binary , as I didn't see anything place from swift-diffusion explicitly force target to x86

I tried to fix through BUILD.bazel file but not successful.
My current workaround is to force arm64 in bazel build command as

bazel run examples:txt2img --cpu=darwin_arm64 --compilation_mode=opt -- /Users/wlin1/Downloads/sd-v1-4.ckpt   "a photograph of an astronaut riding a horse" 

additional --cpu=darwin_arm64 could fix the issue.
Hope this could help other people experienced similar issue.