Load map resources from local assets

[ljbade]

Currently takes the app a second or two to load.

This is due to extracting the asset folder from APK.

Need to make this faster, get rid of it (by reading directly from APK), or only do it once at install time (can hook this), or only if the files are not there.

TODO: more detail on this so you guys understand what I am talking about

[ljbade]

Found Context.getPackageCodePath that may allow the C++ code to read styles directly from the APK zip file if we include C++ zip library.

Also I am going to restrict Assetbridge to extract only the files we need. This should make things faster. See #585

[kkaefer]

Cache shaders: #588

[ljbade]

Also getCodeCacheDir sounds useful:

Returns the absolute path to the application specific cache directory on the filesystem designed for storing cached code. The system will delete any files stored in this location both when your specific application is upgraded, and when the entire platform is upgraded.

But it is only on Android 5+

[ljbade]

OK de1f37f includes the stuff to limit which files are extracted.

[ljbade]

@kkaefer what would be required to get the URL styles to load directly from a zip file? This would save extacting all the styles even if you are not using all of them

Also need to get curl to read the ca-bundle file from the zip too.

[ljbade]

Okay it looks like libzip is the way to go.

[ljbade]

I am proposing a URL scheme like: zip://path/to/zipfile#path/in/zipfile

@kkaefer is that OK?

[ljbade]

OK that will not work with relative files, instead I will just use asset://path/to/file which means we will load from the APK's assets dir.

[ljbade]

Implemented asset:// URLs in f0e4e27

Now I need to get curl to load ca-bundle from memory. See http://curl.haxx.se/libcurl/c/cacertinmem.html and http://blog.steffenl.com/2013/02/load-certificate-from-memory-using-curl.html

[ljbade]

Implemented loading CA bundle from memory: 80be6c4

I removed Assetbridge now that we no longer need it.

Done.

[kkaefer]

+1 on asset:// URLs. Why make it Android-specific though? This will make stylesheets that use that URL scheme platform specific. Instead, we could use this scheme to provide a couple of default styles/sprites/fonts that always ship with all applications.

I refrained from implementing the certificate loading from memory since it requires OpenSSL headers, which we don't necessarily have available on all platform, and I didn't want to add a direct dependency on it, as compiling OpenSSL can get quite hairy.

[ljbade]

Just discovered that Android actually has a native version of the AssetManager in <android/asset_manager.h> so I might be able to get rid of libzip. Reopening.

@kkaefer no specific reason for it to be Android specific. I can refactor it to allow other platforms to implement it.

Interesting about OpenSSL headers. Perhaps we can just make it an #ifdef that is set on platforms that have OpenSSL headers and want to load certs from asset://?

@springmeyer was looking at switching from OpenSSL to Google's BoringSSL fork at least for Android. This is needed as OpenSSL has a compile bug on ARM Android that Google patched in their fork.

[ljbade]

Looks like NDK AssetManager has some flaws.

Will keep using libzip.

[ljbade]

I will get android-mason branch merged, then I can come back and expand asset:// for other platforms.

@incanus do you think asset:// will work for iOS too?

[incanus]

To make sure I understand, the idea is to use asset:// for ease of reference cross-platform?

This is easy enough to do as a layer on top of NSBundle, which searches by filename + extension through the bundled resources. For example we do this already here:

mapbox-gl-native/macosx/main.mm

Line 103 in 51c3224

const std::string path([[[NSBundle mainBundle] pathForResource:@"bright-v6" ofType:@"json" inDirectory:@"styles/"] UTF8String]);

[ljbade]

Yeah that is what I understand from @kkaefer

Ensures same style JSON can be used on both Android and iOS

[incanus]

Working on asset:// on iOS but it's held up by cutting-room-floor/mapbox-gl-cocoa#72 since this needs to work for both the library (which has a MapboxGL.bundle for resources, including styles) and the iOS test app, which includes the styles directly in its own bundle.

[ljbade]

@incanus I have refactored asset_request to use the cross platform pattern.

Currently Linux, OSX and iOS use a noop implementation that just returns a 500 error.

So you will need to copy this and add as necessary to load from iOS.

[kkaefer]

Note that we'll also have to figure out a way to handle asset:// URLs on JS.

[ljbade]

Hmm, yes.

Perhaps just remap asset:// to a HTTP request with prefix to some static URL, e.g. person might have static HTTP server/S3 for serving assets like images, scripts etc for their website

[incanus]

Ok @ljbade what else are we working on here to "make the app faster"? We should rename the ticket if nothing else or else split off into separate, actionable, assignable tickets. Right now for example we're discussing asset:// across a couple tickets.

[ljbade]

@incanus I renamed the issue, is that better?

Anyway this issue was more about the Android implementation of asset://, but that has now grown to cross-platform concept.

Now Android is done with asset URL we should close this and have seperate tickets for asset support on other platforms?

[incanus]

Nah let's do this since it's a single issue thing now:

• iOS
• OS X
• Linux
• JS (add support for asset:// mapbox-gl-js#857)

[kkaefer]

I think we should merge FileRequest and AssetRequest, and move the actual implementation to a platform-specific file, like HTTPRequest already does (it uses Cocoa stuff on OS X/iOS, and CURL on all other platforms).

FileRequest and AssetRequest are already very similar: relative paths in a file:// URLS are relative to the executable or bundle (iOS/OS X), which means they are already used to access "bundled" files like the stylesheet or image sprites.

[jfirebaugh]

👍

[mikemorris]

👍 to merging into FileRequest with platform-specific implementations

[ljbade]

Okay will work on this then.

[kkaefer]

@ljbade merged this. You can now move the Android implementation to the new scheme.

[ljbade]

@kkaefer awesome

[ljbade]

@kkaefer So where did this get to? I think Android was done right?

[kkaefer]

This is implemented in other systems as well. OS X and iPhone bundles load data from the resources folder, Linux builds well load assets relative to the binary's location