-
Notifications
You must be signed in to change notification settings - Fork 737
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
curl_easy_perform() failed: Out of memory error when uploading blobs #308
Comments
This may be covered by a previous issue; however, that previous issue was for a different code sample and different OS. |
Hi @drhalftone , Also it is going to be much easier to debug if we have verbose logging enabled. size_t log = 1;
(void)IoTHubClient_LL_SetOption(iotHubClientHandle, OPTION_CURL_VERBOSE, &log); Make sure you have this include definition on the top of each sample: #include "azure_c_shared_utility/shared_util_options.h" |
This sample starts off well but fails quickly there after:
DrHalftone:iothub_client_sample_http dllau$ ./iothub_client_sample_http
Starting the IoTHub client sample HTTP...
Info: IoT Hub SDK for C, version 1.1.27
IoTHubClient_LL_SetMessageCallback...successful.
IoTHubClient_LL_SendEventAsync accepted message [0] for transmission to IoT Hub.
* Trying 40.114.53.146...
* TCP_NODELAY set
* Connected to DrHalftonesFirstIOTHub.azure-devices.net (40.114.53.146) port 443 (#0)
* SSL: couldn't create a context: error:140A90F2:lib(20):func(169):reason(242)
* Closing connection 0
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:556 curl_easy_perform() failed: Out of memory
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:558 (result = HTTPAPI_OPEN_REQUEST_FAILED)
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/src/httpapiex.c Func:HTTPAPIEX_ExecuteRequest Line:475 unable to recover sending to a working state
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/iothub_client/src/iothubtransporthttp.c Func:DoEvent Line:1680 unable to HTTPAPIEX_SAS_ExecuteRequest
* Trying 40.114.53.146...
* TCP_NODELAY set
* Connected to DrHalftonesFirstIOTHub.azure-devices.net (40.114.53.146) port 443 (#0)
* SSL: couldn't create a context: error:140A90F2:lib(20):func(169):reason(242)
* Closing connection 0
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:556 curl_easy_perform() failed: Out of memory
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:558 (result = HTTPAPI_OPEN_REQUEST_FAILED)
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/src/httpapiex.c Func:HTTPAPIEX_ExecuteRequest Line:475 unable to recover sending to a working state
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/iothub_client/src/iothubtransporthttp.c Func:DoMessages Line:2073 unable to HTTPAPIEX_SAS_ExecuteRequest
IoTHubClient_LL_SendEventAsync accepted message [1] for transmission to IoT Hub.
* Trying 40.114.53.146...
* TCP_NODELAY set
* Connected to DrHalftonesFirstIOTHub.azure-devices.net (40.114.53.146) port 443 (#0)
* SSL: couldn't create a context: error:140A90F2:lib(20):func(169):reason(242)
* Closing connection 0
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:556 curl_easy_perform() failed: Out of memory
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:558 (result = HTTPAPI_OPEN_REQUEST_FAILED)
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/src/httpapiex.c Func:HTTPAPIEX_ExecuteRequest Line:475 unable to recover sending to a working state
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/iothub_client/src/iothubtransporthttp.c Func:DoEvent Line:1680 unable to HTTPAPIEX_SAS_ExecuteRequest
* Trying 40.114.53.146...
* TCP_NODELAY set
* Connected to DrHalftonesFirstIOTHub.azure-devices.net (40.114.53.146) port 443 (#0)
* SSL: couldn't create a context: error:140A90F2:lib(20):func(169):reason(242)
* Closing connection 0
Error: Time:Sat Nov 25 20:49:51 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:556 curl_easy_perform() failed: Out of memory
On Nov 22, 2017, at 4:48 PM, ewertons <[email protected]> wrote:
Hi @drhalftone <https://github.com/drhalftone> ,
does the issue occur right away when the sample is run?
Could you run a little experiment on your machine please? Could you run this sample <https://github.com/Azure/azure-iot-sdk-c/blob/master/iothub_client/samples/iothub_client_sample_http/iothub_client_sample_http.c> as well?
Also it is going to be much easier to debug if we have verbose logging enabled.
On each sample, after "IoTHubClient_LL_CreateFromConnectionString" could you add this line please?
size_t log = 1;
(void)IoTHubClient_LL_SetOption(iotHubClientHandle, OPTION_CURL_VERBOSE, &log);
Make sure you have this include definition on the top of each sample:
#include "azure_c_shared_utility/shared_util_options.h"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1ooM09GLjReElpC6PqfAkITPjFT2pks5s5JaTgaJpZM4QkYAb>.
|
This is a valid workaround that will remove openssl in favor of gnutls.
|
Is there any progress on this for Mac OS X? |
@drhalftone, @tameraw @jebrando are working on a permanent fix. ETA this month. I've asked them to ensure they test on Mac as well. |
That would be awesome! |
@drhalftone - I researched a workaround with Mac, but I wasn't able to find one. The fix should be rolled out soon. @zolvarga @jebrando |
@drhalftone for data sake are you working in the c-sdk or python-sdk? Also how many devices are being returned in your query. This bug has been very elusive, but I think I'm on to something. |
I’m using the c-sdk.
What do you mean by, "how many devices are being returned in your query?"
On Jan 11, 2018, at 12:42 PM, Jelani Brandon <[email protected]> wrote:
@drhalftone <https://github.com/drhalftone> for data sake are you working in the c-sdk or python-sdk? Also how many devices are being returned in your query. This bug has been very elusive, but I think I'm on to something.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1ovkNgTlz-v5VZrsocAFbx7F9uA1Cks5tJkf5gaJpZM4QkYAb>.
|
@jebrando - He is getting this error when trying to upload to blob. Get devices is also an issue, but any call that uses the curl lib has this error. |
Hi @drhalftone,
So the last resort was trying to upgrade curl to the latest version available, and that actually resulted in no errors.
|
So i'm not sure exactly what I'm supposed to do. I assume you want me to install curl through brew. I also think you want me to use the --with-openssl option. So I did use brew; however, brew says that the curl installation is keg only. I also see some instructions about how to get a project to link to this brew version of curl, superseding the Mac OS X version. Should I follow those instructions and link to the brew version of curl? |
Hi @drhalftone , And correct, you should use --with-openssl if building the SDK on a platform that doesn't have OpenSSL as its default library for SSL encryption. |
How exactly do I configure cmake to use the brew version of curl? The github page includes very specific instructions on configuring cmake to use openssl with the command:
cmake -DOPENSSL_ROOT_DIR:PATH=/usr/local/opt/openssl ..
It would be great if you could post new instructions that include the changes for curl.
On Jan 12, 2018, at 12:06 PM, ewertons <[email protected]> wrote:
Hi @drhalftone <https://github.com/drhalftone> ,
yes, the solution is to upgrade the curl libraries in your system to the latest version.
The details of upgrading a specific package on the Mac OS usually are made available by the provider of the platform.
And correct, you should use --with-openssl if building the SDK on a platform that doesn't have OpenSSL as its default library for SSL encryption.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1oo_2gna7oCCC0vC-GACqONbCAVGhks5tJ5EbgaJpZM4QkYAb>.
|
Hi @drhalftone , |
How do I do that? I can only do what brew tells me, and here is what it says when I try to install curl: DrHalftone:~ dllau$ brew install curl If you need to have this software first in your PATH run: For compilers to find this software you may need to set: ==> Summary |
It's working! Okay, I did have to link against the brew version of curl by setting the following flags in the CMake GUI utility: LDFLAGS: -L/usr/local/opt/curl/lib I then had the certificates issue. So I was in the process of setting the verbose flag to 1, but that option fails in the upload_blob sample. But at the same time, even though I didn't successfully set the verbose flag, the code is reporting that it is now working: DrHalftone:iothub_client_sample_upload_to_blob dllau$ ./iothub_client_sample_upload_to_blob At the moment, I don't know exactly what combination of things I've tried did the trick or if I am going to be able to repeat the process. But at least I know some combination fixed it. |
Indeed. [details to come in a moment] Thanks for pushing through, I believe your system is good to go, we just want to be sure. |
Hi @drhalftone , Feel free to leave any comments regarding the content or language. Thanks, |
It was very unclear to me how @drhalftone did achieve to include the brew installed curl in the compile. So it is a simple as adding to the cmake command in azure-iot-sdk-python/build_all/mac/build.sh these additional arguments: -DCURL_INCLUDE_DIR:PATH=/usr/local/opt/curl/include -DCURL_LIBRARY:FILEPATH=/usr/local/opt/curl/lib/libcurl.4.dylib. I can confirm that then the HTTP_API error is solved on mac. You can find out about these two compile arguments by starting the cmake-gui and choosing as BUILD_DIR = azure-iot-sdk-python/c/cmake/iotsdk_mac. And the press the advanced flag option, to see all advanced flags. |
First, I just cloned a clean and new copy of the azure iot GitHub account, but there is no "Mac" directory inside the build_all directory. Here is what is in the build_all directory: drwxr-xr-x 11 dllau staff 352 Feb 14 13:01 . So I tried running cmake according to: cmake -DOPENSSL_ROOT_DIR:PATH=/usr/local/opt/openssl -DCURL_INCLUDE_DIR:PATH=/usr/local/opt/curl/include -DCURL_LIBRARY:FILEPATH=/usr/local/opt/curl/lib/libcurl.4.dylib .. and got the following response: CMake Warning:
Can you please post detailed instructions on how you confirmed that the errors went away on Mac? Since I am now using curl 7.58.0, I can say that it is no longer working. And I haven't been able to re-compile my code successfully. So I am back of the opinion that Azure IOT does not run on Mac. |
Ok, let's sort things out. As per the exact command line options, we will post the syntax here and in the documentation soon. |
@drhalftone,
At the end of the installation you will get this info:
Save the path "/usr/local/Cellar/curl/7.58.0", this is where the library and headers were actually installed.
Do not forget to modify the file bellow to include your device connection string:
NOTE: BE CAREFUL NOT TO OVERRIDE THE VARIABLE IF YOU ALREADY HAVE IT SET ON YOUR SYSTEM.
|
Here is the result of the cmake command. Note I was inside, "/Users/dllau/azure-iot-sdk-c/cmake" DrHalftone:cmake dllau$ cmake -DOPENSSL_ROOT_DIR:PATH=/usr/local/opt/openssl -DCURL_INCLUDE_DIR:PATH=/usr/local/Cellar/curl/7.58.0/include -DCURL_LIBRARY:FILEPATH=/usr/local/Cellar/curl/7.58.0/lib/libcurl.dylib ~/azure-iot-sdk-c
-- Build files have been written to: /Users/dllau/azure-iot-sdk-c/cmake |
Could you confirm:
|
cmake version 3.10.2
DrHalftone:cmake dllau$ ls /usr/local/Cellar/curl/
7.58.0
On Feb 16, 2018, at 8:58 PM, ewertons <[email protected]> wrote:
Could you confirm:
If /usr/local/Cellar/curl/7.58.0 exists in your machine
your version of cmake
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1on2_zW61u0z8zWEKlZVJ2ZzDc4nBks5tVjJdgaJpZM4QkYAb>.
|
This could be a cmake version issue. We have cmake version 3.7 Could you add these two options to your cmake command line and try again? |
Thanks for helping running the verifications, as you know Mac OS is quite strict when it comes to updates, so changing the version of cmake in our environment is not so straightforward. |
Your instructions aren’t quite clear. Are you saying add those two options in addition to the previous options. So I’m using:
cmake -DOPENSSL_ROOT_DIR:PATH=/usr/local/opt/openssl -DNO_CMAKE_PATH=TRUE -DCURL_DIR:PATH=/usr/local/Cellar/curl/7.58.0 ~/azure-iot-sdk-c
and getting the following:
CMake Warning:
Manually-specified variables were not used by the project:
CURL_DIR
On Feb 16, 2018, at 9:52 PM, ewertons <[email protected]> wrote:
This could be a cmake version issue. We have cmake version 3.7
https://cmake.org/cmake/help/v3.10/command/find_package.html?highlight=find_package <https://cmake.org/cmake/help/v3.10/command/find_package.html?highlight=find_package>
Could you add these two options to your cmake command line and try again?
-DNO_CMAKE_PATH=TRUE -DCURL_DIR:PATH=/usr/local/Cellar/curl/7.58.0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1oj9hqnIZuE0M7Qz0Dmg0-joRdXm-ks5tVj77gaJpZM4QkYAb>.
|
@drhalftone, |
Great!
On Feb 21, 2018, at 4:05 PM, ewertons <[email protected]> wrote:
@drhalftone <https://github.com/drhalftone>,
at this point we will need to have a environment that fully match yours to attempt a repro and get instructions to suffice for you to successfully build and run the azure-iot-sdk-c samples.
We will get a new MacOSX machine available within the next days, and we then will proceed with this investigation.
If we cannot get a repro, we will contact you for perhaps a live session so we can work directly on your environment, if possible.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1ohkf8YyamfR-ivj5pz9G3UY7fsFsks5tXIUKgaJpZM4QkYAb>.
|
Starting on OS X El Capitan, Mac OS comes with SIP (System Integrity Protection) which blocks brew from installing/creating links over the existing paths on /usr/* (https://support.apple.com/en-us/HT204899) |
Could you try these updated steps please?
At the end of the installation you will get this info:
Save the path "/usr/local/Cellar/curl/7.58.0", this is where the library and headers were actually installed.
Do not forget to modify the file bellow to include your device connection string:
|
@drhalftone - please confirm if the instructions above from @ewertons resolve your issue and if you have any additional feedback. Thanks. |
Hi @drhalftone, |
It seems to be working. The file is created in my azure storage account; however, I do see these two lines on the output: Starting the IoTHub client sample upload to blob... Is this a concern? |
Is it working? |
I thought I posted on github? It is uploading blobs, but I did see a couple of error messages as well. Here are the output lines of interest:
Starting the IoTHub client sample upload to blob...
Error: Time:Mon Mar 12 23:11:05 2018 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_SetOption Line:831 failure in curl_easy_setopt - CURLOPT_SSL_CTX_FUNCTION
Error: Time:Mon Mar 12 23:11:05 2018 File:/Users/dllau/azure-iot-sdk-c/c-utility/src/httpapiex.c Func:HTTPAPIEX_ExecuteRequest Line:400 HTTPAPI_SetOption failed when called for option TrustedCerts
On Mar 13, 2018, at 11:43 AM, ewertons <[email protected]> wrote:
Is it working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#308 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKL1opBEPDI56qEFUpzYQbQ6jTWpPlV_ks5td-kGgaJpZM4QkYAb>.
|
Then no, it's not a concern. |
Then I am happy to say that it works. I think you should point out that the magic sauce is the line: export DYLD_LIBRARY_PATH="/usr/local/Cellar/curl/7.58.0/lib:$DYLD_LIBRARY_PATH" This has to be run anytime I want to run the compiled executable not when I want to compile. So this is the line that tells the executable which curl library to use. It is not needed to be run before compiling. |
Alright, glad to hear that. |
Mac OS X 10.13.1
Mac Book Pro, mid-2014
Info: IoT Hub SDK for C, version 1.1.27
Description of the issue:
I'm trying to run the sample, iothub_client_sample_upload_to_blob.
Code sample exhibiting the issue:
iothub_client_sample_upload_to_blob.c
Console log of the issue:
Starting the IoTHub client sample upload to blob...
Info: IoT Hub SDK for C, version 1.1.27
Error: Time:Mon Nov 20 09:48:13 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:556 curl_easy_perform() failed: Out of memory
Error: Time:Mon Nov 20 09:48:13 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/adapters/httpapi_curl.c Func:HTTPAPI_ExecuteRequest Line:558 (result = HTTPAPI_OPEN_REQUEST_FAILED)
Error: Time:Mon Nov 20 09:48:13 2017 File:/Users/dllau/azure-iot-sdk-c/c-utility/src/httpapiex.c Func:HTTPAPIEX_ExecuteRequest Line:475 unable to recover sending to a working state
Error: Time:Mon Nov 20 09:48:13 2017 File:/Users/dllau/azure-iot-sdk-c/iothub_client/src/iothub_client_ll_uploadtoblob.c Func:IoTHubClient_LL_UploadToBlob_step1and2 Line:396 unable to HTTPAPIEX_SAS_ExecuteRequest
Error: Time:Mon Nov 20 09:48:13 2017 File:/Users/dllau/azure-iot-sdk-c/iothub_client/src/iothub_client_ll_uploadtoblob.c Func:IoTHubClient_LL_UploadToBlob_Impl Line:873 error in IoTHubClient_LL_UploadToBlob_step1
hello world failed to upload
The text was updated successfully, but these errors were encountered: