[Breaking Change Request] [vm] Deprecate and remove dart native extensions #45451

mraleph · 2021-03-24T17:51:35Z

Summary

Dart Native Extensions is a mechanism for loading native code into a running Dart program and binding Dart methods to native implementatations.

It consists of the following parts:

dart-ext:... import scheme which instructs VM to load a native library;
native keyword which specifies that a function has a native implementation;
Dart VM C API (as defined in runtime/include/dart_api.h) which can be used to reflectively inspect and mutate the state of the Dart program;

Currently native extensions only really work in a standalone Dart VM - Flutter embedder hides VM C API symbols and does not allow developers to interact with Dart through Dart VM APIs.

Relience on C API means that:

Native extensions are not very efficient for fine-grained binding.
Reflective access through Dart VM C API is also incompatible with AOT (requires careful usage of @pragma('vm:entry-point') which in turn disables treeshaking).

dart:ffi was designed to not have these problems - be both more effecient and more AOT friendly. dart:ffi works in standalone VM as well as Flutter.

From our perspective it does not make sense to maintain two completely different approaches to binding Dart code to native especially given that one of them is obviously superior. We don't want more packages using native extensions to appear in the ecosystem and we want to push all new Dart users to use dart:ffi.

We propose the following:

In Dart 2.14 issue a deprecatin warning whenever analyzer or CFE see dart-ext: import.
In Dart 2.15 remove support for dart-ext: imports and hide Dart VM C API symbols from standalone dart binary shipped with SDK

Impact

All packages that use dart-ext: will be broken by the release. We will work to estimate the impact, but we consider it to be very small because none of the Flutter compatible packages will be affected, as Flutter does not support this mechanism in the first place.

TODO

end support for dart-ext import scheme in the analyzer #47407

The text was updated successfully, but these errors were encountered:

mraleph · 2021-03-24T17:52:00Z

/cc @a-siva @mkustermann

a-siva · 2021-03-24T22:27:21Z

Adding the March milestone for the deprecation warning in 2.14 from analyzer and CFE when dart-ext import is seen.

In 2.15 we should also remove samples/sample_extension

a-siva · 2021-03-24T22:28:20Z

Also added area-analyzer and area-front-end labels for the warning work in 2.13

devoncarew · 2021-03-24T22:30:27Z

A deprecation warning wouldn't be much work, but we probably want to discuss adding new work to a release a few working days before a branch cut.

mraleph · 2021-03-24T22:36:30Z

@devoncarew I have updated versions to 2.14 and 2.15 and moved the milestone. No intention to fall over ourselves with this.

blagoev · 2021-03-31T15:50:51Z

Does Dart SDK follow semver versioning?

mraleph · 2021-03-31T16:09:20Z

@blagoev Not really.

franklinyow · 2021-04-09T20:54:06Z

Ping @vsmenon @matanlurey @Hixie for approval

Hixie · 2021-04-09T22:01:34Z

Fine by me since this doesn't affect Flutter anyway. :-)

vsmenon · 2021-04-09T22:38:18Z

lgtm

franklinyow · 2021-04-16T23:34:52Z

Approved

franklinyow · 2021-04-16T23:37:55Z

@mraleph Please add a post to https://groups.google.com/a/dartlang.org/g/announce

mraleph · 2021-04-20T06:56:43Z

@devoncarew @johnniwinther does it sound reasonable to have analyzer produce deprecation warnings in 2.14? I want to get versions right in the announcement.

johnniwinther · 2021-04-20T07:25:16Z

SGTM

Do you have wording to use in the deprecation warning?

mraleph · 2021-04-20T07:45:14Z

@johnniwinther I suggest "Dart native extensions are deprecated and will be removed in 2.15 release of Dart SDK. Migrate to using FFI instead (https://dart.dev/guides/libraries/c-interop)". @mit-mit any thoughts?

srawlins · 2021-04-20T15:26:45Z

What is the plan for moving internal code off of dart-ext? I see a few cases when I search lang:dart import\ ['"]dart-ext in code search.

mraleph · 2021-04-20T15:33:19Z

I have reached to some of the internal owners and working on planning migration

vsmenon · 2021-08-04T13:36:35Z

@mraleph - does this still need to be open?

a-siva · 2021-08-04T15:44:05Z

We still have samples/sample_extension in the repo, are we good to go on deleting it. I can submit a PR deleting the directory

devoncarew · 2021-08-04T15:54:40Z

I believe we were going to deprecate in 2.14 (shipping soon) and remove in 2.15.

a-siva · 2021-08-04T15:57:48Z

Yes the removal will be in the first beta of 2.15

mraleph · 2021-08-31T10:34:29Z

@johnniwinther @srawlins could you make changes to CFE and analyzer to start rejecting dart-ext: imports? (promote the current deprecation warning to a compilation error).

lrhn · 2021-09-01T11:43:51Z

At the time we remove this feature, we should also disallow using native as a function body.
Code like

int systemRand() native "SystemRand";

should become a parsing error, not a run-time "native function 'SystemRand' (0 arguments) cannot be found" error.
(We can be gentle with the compile-time error if we recognize the intent, but it should be a compile-time error).

mraleph · 2021-09-01T11:50:30Z

@lrhn yeah, I will do few separate changes to replace native with an annotation.

Per breaking change #45451 we are removing support for dart-ext: style native extensions from the Dart VM. This CL removes the associated VM code, tests and samples. It also ports a single test which used dart-ext: import to use FFI instead. TEST=ci Bug: #45451 Change-Id: Iae984bce32baf29a950b5de1323939006a217b94 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212050 Commit-Queue: Slava Egorov <[email protected]> Reviewed-by: Daco Harkes <[email protected]> Reviewed-by: Martin Kustermann <[email protected]>

#45451 Change-Id: Ieed7c4a200b0480975715a149596b1f5ea733814 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212002 Reviewed-by: Slava Egorov <[email protected]> Commit-Queue: Johnni Winther <[email protected]>

mraleph · 2021-09-07T18:28:39Z

VM and CFE has removed support for dart-ext. Analyzer work pending.

Removal of native syntax is tracked in #28791

devoncarew · 2021-10-05T15:55:43Z

@mraleph - do you know what the remaining work here is for the stable release? If there is work remaining, perhaps track using sub-issues and update the first comment in this one to be a checklist / meta issue.

mraleph · 2021-10-08T08:49:09Z

The only missing piece is promotion of warning / info to error in the analyzer for dart-ext: imports. Will file an issue.

mraleph · 2021-10-13T12:55:21Z

This is done.

The sample_extension has been deleted a while ago. Bug: #45451 Change-Id: I69400018345d47f3b078d563246f36412eb3472b Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264840 Commit-Queue: Slava Egorov <[email protected]> Auto-Submit: Alexander Thomas <[email protected]> Reviewed-by: Slava Egorov <[email protected]>

mraleph added the breaking-change-request This tracks requests for feedback on breaking changes label Mar 24, 2021

mraleph mentioned this issue Mar 24, 2021

Flutter 2.0 released with bad Dart runtime. flutter/flutter#77812

Closed

devoncarew added the area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. label Mar 24, 2021

a-siva added this to the March Beta Release milestone Mar 24, 2021

a-siva added area-front-end Use area-front-end for front end / CFE / kernel format related issues. area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. labels Mar 24, 2021

mraleph modified the milestones: March Beta Release, April Beta Release Mar 24, 2021

devoncarew added the P2 A bug or feature request we're likely to work on label Mar 29, 2021

blagoev mentioned this issue Mar 31, 2021

Dart VM tries to load native extensions with an incorrect path on Windows #45370

Closed

franklinyow assigned mraleph Apr 16, 2021

This was referenced Apr 20, 2021

Add deprecation warning whenever CFE see dart-ext: import #45758

Closed

Add deprecation warning whenever analyzer see dart-ext: import #45759

Closed

This comment has been minimized.

Sign in to view

devoncarew modified the milestones: July release (beta), September release (beta) Aug 24, 2021

lrhn mentioned this issue Aug 30, 2021

Feature Request: Grouping the ~68 Keywords into context #43072

Open

parlough mentioned this issue Sep 2, 2021

[PAGE ISSUE]: 'Native extensions for the standalone Dart VM' dart-lang/site-www#3542

Closed

devoncarew modified the milestones: September release (Dart 2.15 beta 2), October Release (Dart 2.15 beta 3 + stable candidate) Oct 5, 2021

mraleph mentioned this issue Oct 8, 2021

end support for dart-ext import scheme in the analyzer #47407

Closed

mraleph closed this as completed Oct 13, 2021

hhstore mentioned this issue Nov 10, 2021

Dart FFI: Binding C/C++/Go/Rust Libs For Dart hhstore/blog#242

Open

renlite mentioned this issue Mar 28, 2023

Missing documentation: '@Native' and 'external' (Question) #51879

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Breaking Change Request] [vm] Deprecate and remove dart native extensions #45451

[Breaking Change Request] [vm] Deprecate and remove dart native extensions #45451

mraleph commented Mar 24, 2021 •

edited

Loading

mraleph commented Mar 24, 2021

a-siva commented Mar 24, 2021 •

edited

Loading

a-siva commented Mar 24, 2021

devoncarew commented Mar 24, 2021

mraleph commented Mar 24, 2021

blagoev commented Mar 31, 2021

mraleph commented Mar 31, 2021

franklinyow commented Apr 9, 2021

Hixie commented Apr 9, 2021

vsmenon commented Apr 9, 2021

franklinyow commented Apr 16, 2021

franklinyow commented Apr 16, 2021

mraleph commented Apr 20, 2021

johnniwinther commented Apr 20, 2021

mraleph commented Apr 20, 2021

srawlins commented Apr 20, 2021

mraleph commented Apr 20, 2021

vsmenon commented Aug 4, 2021

a-siva commented Aug 4, 2021

devoncarew commented Aug 4, 2021

a-siva commented Aug 4, 2021

This comment has been minimized.

This comment has been minimized.

mraleph commented Aug 31, 2021 •

edited

Loading

lrhn commented Sep 1, 2021 •

edited

Loading

mraleph commented Sep 1, 2021

mraleph commented Sep 7, 2021

devoncarew commented Oct 5, 2021

mraleph commented Oct 8, 2021

mraleph commented Oct 13, 2021

[Breaking Change Request] [vm] Deprecate and remove dart native extensions #45451

[Breaking Change Request] [vm] Deprecate and remove dart native extensions #45451

Comments

mraleph commented Mar 24, 2021 • edited Loading

Summary

Impact

TODO

mraleph commented Mar 24, 2021

a-siva commented Mar 24, 2021 • edited Loading

a-siva commented Mar 24, 2021

devoncarew commented Mar 24, 2021

mraleph commented Mar 24, 2021

blagoev commented Mar 31, 2021

mraleph commented Mar 31, 2021

franklinyow commented Apr 9, 2021

Hixie commented Apr 9, 2021

vsmenon commented Apr 9, 2021

franklinyow commented Apr 16, 2021

franklinyow commented Apr 16, 2021

mraleph commented Apr 20, 2021

johnniwinther commented Apr 20, 2021

mraleph commented Apr 20, 2021

srawlins commented Apr 20, 2021

mraleph commented Apr 20, 2021

vsmenon commented Aug 4, 2021

a-siva commented Aug 4, 2021

devoncarew commented Aug 4, 2021

a-siva commented Aug 4, 2021

This comment has been minimized.

This comment has been minimized.

mraleph commented Aug 31, 2021 • edited Loading

lrhn commented Sep 1, 2021 • edited Loading

mraleph commented Sep 1, 2021

mraleph commented Sep 7, 2021

devoncarew commented Oct 5, 2021

mraleph commented Oct 8, 2021

mraleph commented Oct 13, 2021

mraleph commented Mar 24, 2021 •

edited

Loading

a-siva commented Mar 24, 2021 •

edited

Loading

mraleph commented Aug 31, 2021 •

edited

Loading

lrhn commented Sep 1, 2021 •

edited

Loading