Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Julia support to flatbuffers #5088

Closed
wants to merge 26 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e54e05e
add Julia support to flatbuffers
rjkat Aug 31, 2018
73181da
add Julia support to flatbuffers
rjkat Aug 31, 2018
35db6c5
Merge branch 'master' of ssh://github.com/rjkat/flatbuffers-julia
rjkat Dec 11, 2018
06c6249
VS2010 fixes
rjkat Dec 11, 2018
6b87534
VS2010 fixes
rjkat Dec 11, 2018
9056551
add FlatBuffers package
rjkat Dec 13, 2018
79d8326
merge with upstream
rjkat Dec 14, 2018
33b20f6
Apply suggestions from code review
rjkat Dec 19, 2018
f2726ec
code review changes
rjkat Dec 20, 2018
630addd
fix namespace test
rjkat Dec 22, 2018
258fd55
put defaults back the way they were
rjkat Jan 4, 2019
b246260
Merge branch 'master' of ssh://github.com/rjkat/flatbuffers-julia
rjkat Jan 4, 2019
a466f5d
formatting
rjkat Jan 4, 2019
fed18f8
merge with upstream
rjkat Jan 5, 2019
bb70036
fix last commit
rjkat Jan 5, 2019
b7b78ed
work in progress on stateless compilation
rjkat Jan 9, 2019
bfd1ab4
work in progress
rjkat Jan 9, 2019
f6cd1a4
stateless flatbuffers
rjkat Jan 9, 2019
c3103a3
clang format
rjkat Jan 9, 2019
d4e9df3
stateless julia code generation
rjkat Jan 9, 2019
40e0b78
clang format
rjkat Jan 9, 2019
1484974
disable precompilation for generated modules
rjkat Jan 22, 2019
5f94647
Merge branch 'master' of https://github.com/google/flatbuffers
rjkat Nov 17, 2019
6a69c14
get flatc building again, bring in latest FlatBuffers.jl
rjkat Nov 17, 2019
6ae0bf4
bring in julia tests
rjkat Nov 18, 2019
95e54d8
work on circular type dependencies
rjkat Nov 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ set(FlatBuffers_Compiler_SRCS
src/idl_gen_general.cpp
src/idl_gen_go.cpp
src/idl_gen_js_ts.cpp
src/idl_gen_julia.cpp
src/idl_gen_php.cpp
src/idl_gen_python.cpp
src/idl_gen_lobster.cpp
Expand Down
2 changes: 2 additions & 0 deletions docs/source/Compiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ For any schema input files, one or more generators can be specified:

- `--rust`, `-r` : Generate Rust code.

- `--julia`: Generate Julia code.

For any data input files:

- `--binary`, `-b` : If data is contained in this file, generate a
Expand Down
2 changes: 2 additions & 0 deletions docs/source/FlatBuffers.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ sections provide a more in-depth usage guide.
own programs.
- How to [use the generated Rust code](@ref flatbuffers_guide_use_rust) in your
own programs.
- How to [use the generated Julia code](@ref flatbuffers_guide_use_julia) in your
own programs.
- [Support matrix](@ref flatbuffers_support) for platforms/languages/features.
- Some [benchmarks](@ref flatbuffers_benchmarks) showing the advantage of
using FlatBuffers.
Expand Down
57 changes: 57 additions & 0 deletions docs/source/JuliaUsage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Use in Julia {#flatbuffers_guide_use_julia}
==============
## Before you get started
Before diving into the FlatBuffers usage in Julia, it should be noted that the
[Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide to general
FlatBuffers usage in all of the supported languages (including Julia). This
page is designed to cover the nuances of FlatBuffers usage, specific to
Julia.

You should also have read the [Building](@ref flatbuffers_guide_building)
documentation to build `flatc` and should be familiar with
[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler) and
[Writing a schema](@ref flatbuffers_guide_writing_schema).

## FlatBuffers Julia package
The `flatc` backend for Julia makes heavy use of the [FlatBuffers.jl package](https://github.com/JuliaData/FlatBuffers.jl).
Documentation for this package may be found [here](http://juliadata.github.io/FlatBuffers.jl/stable).
You can also browse the source code on the [FlatBuffers.jl GitHub page](https://github.com/JuliaData/FlatBuffers.jl).

## Testing the FlatBuffers Julia library
The code to test the Julia library can be found at [FlatBuffers.jl/test/flatc.jl](https://github.com/JuliaData/FlatBuffers.jl/tree/master/test/flatc.jl). To run the tests, run `julia test/flatc.jl` in the top-level directory.
To obtain Julia itself, go to the [Julia homepage](http://julialang.org)
for binaries or source code for your platform.

## Using the FlatBuffers Julia library
*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth
example of how to use FlatBuffers in Julia.*

There is support for both reading and writing FlatBuffers in Julia.
To use FlatBuffers in your own code, first generate Julia modules from your
schema with the `--julia` option to `flatc`. Then you can include both
FlatBuffers and the generated code to read or write a FlatBuffer.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.julia}
import FlatBuffers

include("MyGame/MyGame.jl")
import .MyGame.Example.Monster

monster = open("monsterdata_test.mon", "r") do f Monster(f) end
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now you can access values like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.julia}
hp = monster.hp
pos = monster.pos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
More detailed documentation for using the FlatBuffers Julia package
may be found [here](http://juliadata.github.io/FlatBuffers.jl/stable).

## Speed
The FlatBuffers Julia package has not been thoroughly optimized for speed.
For example, instead of referring to memory in-place, values are instead
loaded with `unsafe_load`. It does however support deduplication of vtables, which
saves roughly 30% of time spent writing, when compared to writing objects
with duplicate vtables.

<br>
36 changes: 19 additions & 17 deletions docs/source/Support.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,29 +18,31 @@ In general:

NOTE: this table is a start, it needs to be extended.

Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ----
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ?
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Julia |
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ---- | ---- |
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes |
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No |
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | No |
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No |
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No |
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | Yes |
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | ? |
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | Yes |
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | ? |
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | ? |
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | ? |
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | jq*/rk* |

* aard = aardappel (previously: gwvo)
* ev = evolutional
* js = jonsimantov
* mik = mikkelfj
* ch = chobie
* kr = krojew
* jq = quinnj
* rk = rkat
rjkat marked this conversation as resolved.
Show resolved Hide resolved

<br>
Loading