Support for OpenSSL 3.2.0

[femto]

Bug Report

the commit cb333fe when make std_spec

Invalid memory access (signal 11) at address 0x(???)
[0x1064ef57b] *Exception::CallStack::print_backtrace:Nil +107 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x105a6889d] ~procProc(Int32, Pointer(LibC::SiginfoT), Pointer(Void), Nil)@src/crystal/system/unix/signal.cr:131 +285 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x7ff80f17adfd] _sigtramp +29 in /usr/lib/system/libsystem_platform.dylib
[0x7ff816939a7b] xmlFindCharEncodingHandler +175 in /usr/lib/libxml2.2.dylib
[0x7ff8169a22b1] xmlNewSaveCtxt +103 in /usr/lib/libxml2.2.dylib
[0x7ff8169a2490] xmlSaveToIO +31 in /usr/lib/libxml2.2.dylib
[0x1070201e1] *XML::Node#to_xml<String::Builder>:String::Builder +513 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x10701ffd9] *XML::Node#to_s<String::Builder>:Nil +9 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x10715cade] *XML::NodeSet@Enumerable(T)#join<String::Builder, Char>:Nil +158 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x10715ca3e] *XML::NodeSet#to_s<String::Builder>:Nil +14 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x10715ca12] *XML::NodeSet@Object#to_s:String +66 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106458ffd] ~procProc(Nil)@spec/std/xml/xpath_spec.cr:210 +141 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106a285a0] *Spec::Example#internal_run<Time::Span, Proc(Nil)>:(Array(Spec::Result) | Nil) +224 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106a28492] *Spec::Example#run:Nil +1234 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106a2ba4a] *Spec::ExampleGroup@Spec::Context#internal_run:Nil +122 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106a2b8f7] *Spec::ExampleGroup#run:Nil +359 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x1068fe204] *Spec::RootContext+@Spec::Context#internal_run:Nil +164 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x1068fe15a] *Spec::RootContext+@Spec::RootContext#run:Nil +26 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x1068b7140] *Spec::execute_examples:Nil +32 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x105aadd30] ~procProc(Int32, (Exception | Nil), Nil)@src/spec/dsl.cr:208 +64 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106838c7e] *Crystal::AtExitHandlers::run<Int32, (Exception+ | Nil)>:Int32 +142 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106ae17e1] *Crystal::exit<Int32, (Exception+ | Nil)>:Int32 +33 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x106ae1761] *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 +97 in /Users/femtozheng/c-project/crystal/.build/std_spec
[0x1059ea049] main +9 in /Users/femtozheng/c-project/crystal/.build/std_spec
make: *** [std_spec] Error 11

I've built crystal several commits earlier, so probably this one is not using lastest crystal?
so I tried : make && make std_spec
now:

Unhandled exception in spawn: SSL_read: I/O error (OpenSSL::SSL::Error)
  from src/openssl/ssl/socket.cr:137:9 in 'unbuffered_read'
  from src/io/buffered.cr:82:16 in 'read'
  from src/io.cr:1176:12 in 'copy'
  from src/io.cr:566:9 in 'gets_to_end'
  from spec/std/openssl/ssl/socket_spec.cr:167:20 in '->'
  from src/fiber.cr:146:11 in 'run'
  from src/fiber.cr:98:34 in '->'
Caused by: SSL_read: Undefined error: 0 (RuntimeError)

and it hangs

one question, would make std_spec automatically call make? or do I need to make && make std_spec?

[straight-shoota]

These errors doent seem very related, except that they're both involving some external libraries.
My guess would be some dependency incompatibility perhaps?
What versions of libxml and libssl are you using?

would make std_spec automatically call make? or do I need to make && make std_spec?

No, make std_spec does not build the compiler. It calls the wrapper script bin/crystal which uses a local compiler build if available (.build/crystal) or whatever compiler executable you have in $PATH. The script should print a notifcation (Using compiled compiler at .build/crystal) to stderr if it uses a local build.
If you want to make sure to have a local build, you need to run make. To get rid of it, run make clean.

[femto]

my make std_spec doesn't print out using local compiler
xml2-config --version
reports 2.9.4
openssl version
reports
OpenSSL 3.2.0 23 Nov 2023 (Library: OpenSSL 3.2.0 23 Nov 2023)

[straight-shoota]

Thanks. I can reproduce the error with OpenSSL 3.2.0.

[straight-shoota]

This seems to be an issue with our error handling code for SSL_read. Unfortunately, it is quite convoluted.

crystal/src/openssl/ssl/socket.cr

Lines 129 to 139 in e3200d9

	LibSSL.ssl_read(@ssl, slice.to_unsafe, count).tap do |bytes|
	if bytes <= 0 && !LibSSL.ssl_get_error(@ssl, bytes).zero_return?
	ex = OpenSSL::SSL::Error.new(@ssl, bytes, "SSL_read")
	if ex.underlying_eof?
	# underlying BIO terminated gracefully, without terminating SSL aspect gracefully first
	# some misbehaving servers "do this" so treat as EOF even though it's a protocol error
	return 0
	end
	raise ex
	end
	end

The error constructor tries to determine the reason for the error, particularly if it's an EOF (lines 92-98):

crystal/src/openssl.cr

Lines 83 to 110 in e3200d9

	def initialize(ssl : LibSSL::SSL, return_code : LibSSL::Int, func = nil)
	@error = LibSSL.ssl_get_error(ssl, return_code)
	case @error
	when .none?
	message = "Raised erroneously"
	when .syscall?
	@code, message = fetch_error_details
	if @code == 0
	case return_code
	when 0
	message = "Unexpected EOF"
	@underlying_eof = true
	when -1
	cause = RuntimeError.from_errno(func || "OpenSSL")
	message = "I/O error"
	else
	message = "Unknown error"
	end
	end
	when .ssl?
	@code, message = fetch_error_details
	else
	message = @error.to_s
	end
	super(func ? "#{func}: #{message}" : message, true, cause: cause)
	end

Apparently the error return value of SSL_read changed in OpenSSL 3.2 from 0 to -1. But our codes usese this differenceto tell EOF from other errors.
According to the current OpenSSL docs for SSL_ERROR_SYSCALL, you should rather consult errno in this case. And it seems that a success value there means EOF.
So I'll try that.

[Blacksmoke16]

Resolved via #14169

[straight-shoota]

#14169 was eventually reverted. A new approach is growing in #14219