System.DllNotFoundException: Unable to load DLL 'e_sqlite3' with .Net Core 2 project on Raspberry PI

[longo-net]

We have an application developed with .NET Core 2.0 Preview 1 that use entity framework with a sqlite db.
The app works fine on pc with windows and linux with x64 cpu, but if we publish it to "ubuntu.16.04-arm" runtime and copy the output on Raspberry Pi 2, when we try to run the app we have this error:

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'e_sqlite3': The specified module could not be found.
(Exception from HRESULT: 0x8007007E)
at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number()
at SQLitePCL.SQLite3Provider_e_sqlite3.SQLitePCL.ISQLite3Provider.sqlite3_libversion_number()
at SQLitePCL.raw.SetProvider(ISQLite3Provider imp)
at SQLitePCL.Batteries_V2.Init()

In the folder root there are two subfolders called "x86" and "x64" with inside "e_sqlite3.dll".

Could it be that the arm version of e_sqlite3.dll is missing?

[ericsink]

Oh my. I apparently didn't see this issue when it came in "18 days ago". My apologies.

Yes, it is true that my build process currently does not build e_sqlite3 for Linux arm. I would have to investigate what kind of cross compiler setup is needed for that case.

[ms579984]

I'm currenty hitting the same issue with .NET Core 2.0 Preview 2. Any progress or schedule on this ?

[ericsink]

This is nearly completed. I will plan to have a build for testing sometime this week.

[longo-net]

Hi, as workaround, i solved copying manually the following file in the root of my .net core project:
https://github.com/ericsink/SQLitePCL.raw/tree/master/android/sqlite3/libs/armeabi-v7a

[ericsink]

Try this:

https://www.nuget.org/packages/SQLitePCLRaw.lib.e_sqlite3.linux/1.1.8-pre20170717084758

This package is usually brought in as a dependency. Just add a PackageReference for SQLitePCLRaw.lib.e_sqlite3.linux at the version given above.

[los93sol]

@ericsink What package should we use for the Pi3 running Windows iOT Core?

[ericsink]

@los93sol I don't have that configuration for testing, but I believe it works fine in the current release. If you're using bundle_e_sqlite3, NuGet should do the right thing. I think.

[los93sol]

I'm actually using .net core 2.0 preview, it works fine on x64, but on arm (on the pi3), i get the same DllNotFoundException for e-sqlite3.dll. My stacktrace is identical to the one in this issue

[ericsink]

Ah, I see.

This issue was about Linux, and the Windows arm problem is somewhat different.

Anyway, I thought this config was working, but perhaps not. Looking at it now, I see a likely problem. I'll dig into it further.

[los93sol]

Good news, let me know if youd like me to test anything

[los93sol]

Some additional information....

I'm deploying to the pi as a self contained app and have specified the runtime as win10-arm.

As a workaround temporarily would it work if i simply build a UWP app and copy the assemblies from that?

[los93sol]

Like the other poster above, this is also .net core preview 2, as you said, it appears there isn't a build target defined

[ericsink]

"As a workaround temporarily would it work if i simply build a UWP app and copy the assemblies from that?"

I don't know.

My build system does contain some other Windows/ARM builds, but I don't know yet if any of them are the right configuration.

[ericsink]

My investigation this evening does make me curious about whether the UWP/ARM binary might work. The build settings seem basically the same.

The corresponding sqlite DLLs are in SQLitePCLRaw.lib.e_sqlite3.v140. An ARM dll is included.

[los93sol]

Thanks for looking into this, I just tried with that package copied from a UWP app into my .NET core directly and got the following....

System.TypeInitializationException: The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an
exception. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.DllNotFoundException: Unable to load DLL 'e_sqlite3': The specified module could not be found. (Exception
from HRESULT: 0x8007007E)
at SQLitePCL.SQLite3Provider_e_sqlite3.NativeMethods.sqlite3_libversion_number()
at SQLitePCL.SQLite3Provider_e_sqlite3.SQLitePCL.ISQLite3Provider.sqlite3_libversion_number()
at SQLitePCL.raw.SetProvider(ISQLite3Provider imp)
at SQLitePCL.Batteries_V2.Init()
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[]
parameters, CultureInfo culture)
at Microsoft.Data.Sqlite.Utilities.BundleInitializer.Initialize()
at Microsoft.Data.Sqlite.SqliteConnection..cctor()
--- End of inner exception stack trace ---

[ericsink]

Did any e_sqlite3 dll make it into the build output directory?

If not, what happens if you put the ARM (from the v140 nupkg I mentioned) dll in there manually?

There are two separate questions here: How to get the native dll to end up in the right place, and whether that dll was built with the right configuration.

[los93sol]

I actually tried both, I dropped the DLL in from a UWP app directly and after that I tried adding the nuget package to my project. Adding the nuget package did result in the e_sqlite dll in the output directory without me having to play games with it, but I got the same stack trace both ways.

It might be worth noting that I also reference Microsoft.EntityFrameworkCore.SQLite and in it's dependencies it appears to be using SQLitePCLRaw.bundle_green (1.1.7). I'm wondering if that's where the issue is coming from, conflicting assemblies and EF might be going down the wrong path?

[los93sol]

Adding the package to the project it ended up in the root of the publish dir, could it be hunting for it in a subfolder? I'm not sure how to get a debugger to see where its searching

[ericsink]

Yes, bundle_green could be causing a problem. Any way we can eliminate that variable for diagnostic purposes?

Re: "hunting in a subfolder" -- that's probably not the issue. On some platforms, we do look in a subfolder, but we fallback to the main folder if it's not found.

However, if the DLL is found, but it cannot be loaded because its own dependencies cannot be, the error message is the same. Usually this happens because of the lack of a C runtime library. Not sure if this is happening in your case or not. Just mentioning it.

[los93sol]

I'm not sure how to invoke it directly to test if that works or not

[los93sol]

Okay, I figured out how to exclude bundle_green....

dotnet/efcore#7667

Based on that I switched from using Microsoft.EntityFrameworkCore.SQLite to Microsoft.EntityFrameworkCore.SQLite.Core which does not have the dependency on bundle_green.

The result.....

Worked on desktop (x64)
Same DLLNotFoundException on the pi

[ericsink]

Thanks for the update. I'm still working on this.

[los93sol]

Anything I can do to help?

[ericsink]

Not yet. I hope to soon have something for you to test.

[xpfr]

I tried using the different files mentioned in the above workarounds (such as libe_sqlite3.so or through the package https://www.nuget.org/packages/SQLitePCLRaw.lib.e_sqlite3.linux/1.1.8-pre20170717084758). But I'm still getting the message on my Raspberry Pi. Should libe_sqlite3.so be located in a given subfolder? (I tried 'armsf')

[ericsink]

@xfpr To be clear, you're talking about Linux, not Windows IOT. This issue's thread is getting mixed up.

No, there should be no need for the file to be in a subfolder. That said, I don't actually have any confirmation that the test package actually works.

[xpfr]

You're right. I'm talking about Linux as for the original error being reported. I still didn't find ways to avoid the exception.
Here are the files in the project directory that I'm getting through the Publish action:

As mentioned, I also tried what worked for @longo-net, i.e. to put the file "libe_sqlite3.so" in the same folder (and even in a 'armsf' sub-folder).
Without success. I'm still getting the exception.
Do you plan to issue a fix applicable to Linux as well?

[ericsink]

"Do you plan to issue a fix applicable to Linux as well?"

Yes, although I currently don't know what fix is necessary.

I think I need to just get a couple of Raspberry Pi 3's, one for Linux and one for Windows IOT, and just sort this out.

[ericsink]

@xpfr With a simple console project, if I add the SQLitePCLRaw.lib.e_sqlite3.linux version 1.1.8-pre20170717084758 package, and then dotnet publish -r linux-arm, I do get the libe_sqlite3.so file in the publish output directory.

My project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="1.1.7" />
    <PackageReference Include="SQLitePCLRaw.ugly" Version="1.1.7" />
    <PackageReference Include="SQLitePCLRaw.lib.e_sqlite3.linux" Version="1.1.8-pre20170717084758" />
  </ItemGroup>
</Project>

Simple Program.cs:

using System;
using SQLitePCL;
using SQLitePCL.Ugly;

namespace p3
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            SQLitePCL.Batteries_V2.Init();
            using (var db = ugly.open(":memory:"))
            {
                var s = db.query_scalar<string>("SELECT sqlite_version()");
                Console.WriteLine("{0}", s);
            }
        }
    }
}

The file in the publish output:

-rwxrwxrwx 1 root root 581932 Jul 24 21:26 libe_sqlite3.so

Do these steps result in that .so file in the publish output for your case?

[longo-net]

@xpfr I published my project to "ubuntu.16.04-arm" and after i copied https://github.com/ericsink/SQLitePCL.raw/blob/master/android/sqlite3/libs/armeabi-v7a/libe_sqlite3.so in the project root folder (i used Raspberry Pi 2 with ubuntu 16.04)

[xpfr]

Sorry. In my previous attempts, I didn't copy the right .so file to the root directory on the Pi.
I did the "dotnet publish" again using the package you shared (SQLitePCLRaw.lib.e_sqlite3.linux version 1.1.8-pre20170717084758) and checked that I got the file .so in the publish output. That's ok.
I copied this file to the Pi and I didn't get the exception any more.
Everything works fine now. Many thanks.

[ericsink]

@xpfr Excellent. Thanks for the followup.

This confirms the fix for the linux-arm issue.

The Windows-arm issue is still pending. My Pi 3 is on its way.

[Lenael]

Good to know, that there is workaround for linux users.
I'm using the same configuration as @los93sol and I get the same error - System.DllNotFoundException: Unable to load DLL 'e_sqlite3'. I think I've tried everyting but the problem persists.
Thank you @ericsink for your efforts.
If there will be something to test, I'd be glad to help.

[ericsink]

@los93sol @Lenael

For testing with .NET Core on Win IOT on Raspberry Pi:

https://www.nuget.org/packages/SQLitePCLRaw.lib.e_sqlite3.v110_xp/1.1.8-pre20170726145242

[los93sol]

@ericsink Thank you, that worked, do you have plans to roll the fix into bundle_green so it will work out of the box with EFCore?

[ericsink]

"do you have plans to roll the fix into"

Yep. Hopefully this coming week.

[Lenael]

Sorry for late response.

Confirmed. Everything works fine with dotnet 2.0.0-preview2-006497, great!
Thank you @ericsink!

[los93sol]

@ericsink Just a quick note on 051c5b2 I actually build for Windows iot core with win10-arm RID, I believe with 2.0 that is the standard. Not sure if that makes a difference or not, but wanted to point it out just in case.

[ericsink]

That would make more sense to me. Can you by chance point me at something that says which RID is more correct?

[los93sol]

The closest I could find was where it was recommended to me https://github.com/dotnet/corefx/issues/18023#issuecomment-312347161 but looking at it again he actually said win-arm, though when I did that it actually did not work as expected, but win10-arm did

[ericsink]

Not sure what is considered the most correct today, but I found the following when I was trying to figure out what RID to use:

aspnet/KestrelHttpServer#340

aspnet/libuv-build#15

[ericsink]

Both the linux and Windows 10 IoT builds of e_sqlite3 for ARM are included in version 1.1.8, which has been pushed to nuget.

[siebigteroth]

Getting this error on Android x86 and version 1.1.8

[ericsink]

If you are on Android, you are not getting "this error". Please post a separate issue with complete details.

[ms-141630]

Iam getting the following error:

Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HL8JCGCMIUF7", Request id "0HL8JCGCMIUF7:00000003": An unhandled exception was thrown by the application.
System.TypeInitializationException: The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'e_sqlite3': The specified module or one of its dependencies could not be found.

Application is hosted on a Raspberry 3 with ubuntu.16.04-arm

I installed the package https://www.nuget.org/packages/SQLitePCLRaw.lib.e_sqlite3.v110_xp/1.1.8-pre20170726145242 but no change.

Iam using entity framework core 2.0

[ms-141630]

@longo-net how did you fix your problem?

[ms-141630]

Alright Iam stupid, should have red the documentation first. Works like a charm now. Thanks for the great work on this!