Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: google-ai-edge/mediapipe
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.7.11
Choose a base ref
...
head repository: google-ai-edge/mediapipe
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 0.7.12
Choose a head ref
  • 2 commits
  • 86 files changed
  • 1 contributor

Commits on Sep 28, 2020

  1. Project import generated by Copybara.

    GitOrigin-RevId: a67c2c19ade0ba122816a250ecee3aab490ef16b
    MediaPipe Team authored and chuoling committed Sep 28, 2020
    Copy the full SHA
    cccf624 View commit details

Commits on Oct 19, 2020

  1. Project import generated by Copybara.

    GitOrigin-RevId: f9a66589eaf652bb93f8e37ed9e4da26e59ef214
    MediaPipe Team authored and chuoling committed Oct 19, 2020
    Copy the full SHA
    c828392 View commit details
Showing with 1,226 additions and 952 deletions.
  1. +1 −1 Dockerfile
  2. +5 −6 README.md
  3. +5 −3 WORKSPACE
  4. +2 −2 docs/getting_started/hello_world_android.md
  5. +15 −17 docs/getting_started/install.md
  6. +5 −6 docs/index.md
  7. +1 −1 docs/solutions/face_mesh.md
  8. +6 −3 docs/tools/tracing_and_profiling.md
  9. +56 −108 mediapipe/calculators/core/BUILD
  10. +29 −18 mediapipe/calculators/core/gate_calculator.cc
  11. +6 −1 mediapipe/calculators/core/gate_calculator.proto
  12. +48 −1 mediapipe/calculators/core/gate_calculator_test.cc
  13. +36 −115 mediapipe/calculators/image/BUILD
  14. +0 −1 mediapipe/calculators/tensorflow/BUILD
  15. +30 −6 mediapipe/calculators/tensorflow/pack_media_sequence_calculator.cc
  16. +54 −0 mediapipe/calculators/tensorflow/pack_media_sequence_calculator_test.cc
  17. +41 −81 mediapipe/calculators/tflite/BUILD
  18. +18 −1 mediapipe/calculators/tflite/tflite_inference_calculator.cc
  19. +14 −0 mediapipe/calculators/tflite/tflite_inference_calculator.proto
  20. +7 −3 mediapipe/calculators/tflite/tflite_tensors_to_landmarks_calculator.cc
  21. +52 −267 mediapipe/calculators/util/BUILD
  22. +73 −75 mediapipe/calculators/util/annotation_overlay_calculator.cc
  23. +2 −1 mediapipe/calculators/util/detection_label_id_to_text_calculator.cc
  24. +2 −0 mediapipe/calculators/util/landmark_letterbox_removal_calculator.cc
  25. +2 −0 mediapipe/calculators/util/landmark_projection_calculator.cc
  26. +2 −0 mediapipe/calculators/util/landmarks_smoothing_calculator.cc
  27. +44 −17 mediapipe/calculators/util/landmarks_to_render_data_calculator.cc
  28. +15 −0 mediapipe/calculators/util/landmarks_to_render_data_calculator.proto
  29. +1 −0 mediapipe/calculators/video/BUILD
  30. +2 −1 mediapipe/calculators/video/box_tracker_calculator.cc
  31. +0 −2 mediapipe/calculators/video/tool/BUILD
  32. +3 −0 mediapipe/examples/android/src/java/com/google/mediapipe/apps/basic/MainActivity.java
  33. +12 −6 mediapipe/examples/android/src/java/com/google/mediapipe/apps/iristrackinggpu/MainActivity.java
  34. +1 −1 mediapipe/examples/coral/Dockerfile
  35. +11 −0 mediapipe/examples/desktop/autoflip/calculators/content_zooming_calculator.cc
  36. +19 −0 mediapipe/examples/desktop/autoflip/calculators/content_zooming_calculator_test.cc
  37. +13 −0 mediapipe/framework/BUILD
  38. +28 −0 mediapipe/framework/basic_types_registration.cc
  39. +23 −20 mediapipe/framework/collection_test.cc
  40. +1 −0 mediapipe/framework/deps/file_helpers.cc
  41. +39 −2 mediapipe/framework/formats/BUILD
  42. +0 −2 mediapipe/framework/formats/annotation/BUILD
  43. +10 −1 mediapipe/framework/formats/landmark.proto
  44. +0 −2 mediapipe/framework/formats/motion/BUILD
  45. +4 −3 mediapipe/framework/input_stream_manager.cc
  46. +2 −0 mediapipe/framework/legacy_calculator_support.h
  47. +6 −1 mediapipe/framework/packet.h
  48. +0 −1 mediapipe/framework/profiler/BUILD
  49. +4 −2 mediapipe/framework/profiler/graph_profiler.cc
  50. +8 −0 mediapipe/framework/subgraph.h
  51. +0 −2 mediapipe/framework/testdata/BUILD
  52. +10 −47 mediapipe/framework/tool/BUILD
  53. +22 −0 mediapipe/framework/tool/gate_subgraph.proto
  54. +20 −0 mediapipe/framework/tool/subgraph_expansion.cc
  55. +5 −0 mediapipe/framework/tool/subgraph_expansion.h
  56. +1 −1 mediapipe/gpu/BUILD
  57. +1 −0 mediapipe/gpu/gl_simple_shaders.cc
  58. +7 −0 mediapipe/graphs/hand_tracking/BUILD
  59. +3 −0 mediapipe/graphs/hand_tracking/multi_hand_tracking_desktop_live.pbtxt
  60. +0 −2 mediapipe/graphs/instant_motion_tracking/calculators/BUILD
  61. +13 −4 mediapipe/graphs/object_detection_3d/calculators/gl_animation_overlay_calculator.cc
  62. +5 −1 mediapipe/java/com/google/mediapipe/components/BUILD
  63. +205 −49 mediapipe/java/com/google/mediapipe/components/CameraXPreviewHelper.java
  64. +1 −1 mediapipe/modules/face_landmark/face_landmark_landmarks_to_roi.pbtxt
  65. +16 −5 mediapipe/objc/BUILD
  66. +43 −1 mediapipe/objc/MPPPlayerInputSource.m
  67. +1 −1 mediapipe/objc/MPPTimestampConverter.mm
  68. +2 −2 mediapipe/python/image_frame_test.py
  69. +1 −0 mediapipe/python/pybind/BUILD
  70. +8 −6 mediapipe/python/pybind/calculator_graph.cc
  71. +4 −0 mediapipe/python/pybind/image_frame.cc
  72. +8 −0 mediapipe/python/pybind/validated_graph_config.cc
  73. +5 −28 mediapipe/util/BUILD
  74. +11 −0 mediapipe/util/annotation_renderer.cc
  75. +1 −0 mediapipe/util/annotation_renderer.h
  76. +6 −0 mediapipe/util/render_data.proto
  77. +11 −4 mediapipe/util/sequence/media_sequence.py
  78. +2 −1 mediapipe/util/sequence/media_sequence_test.py
  79. +7 −0 mediapipe/util/tflite/BUILD
  80. +10 −6 mediapipe/util/tflite/cpu_op_resolver.cc
  81. +6 −2 mediapipe/util/tflite/cpu_op_resolver.h
  82. +12 −5 mediapipe/util/tflite/tflite_gpu_runner.cc
  83. +7 −0 mediapipe/util/tflite/tflite_gpu_runner.h
  84. +4 −3 mediapipe/util/tracking/tracking.cc
  85. +1 −1 setup.py
  86. +9 −2 third_party/BUILD
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -54,7 +54,7 @@ RUN pip3 install tf_slim
RUN ln -s /usr/bin/python3 /usr/bin/python

# Install bazel
ARG BAZEL_VERSION=3.0.0
ARG BAZEL_VERSION=3.4.1
RUN mkdir /bazel && \
wget --no-check-certificate -O /bazel/installer.sh "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/b\
azel-${BAZEL_VERSION}-installer-linux-x86_64.sh" && \
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -8,15 +8,14 @@ nav_order: 1

--------------------------------------------------------------------------------

## Cross-platform ML solutions made simple
## Live ML anywhere

[MediaPipe](https://google.github.io/mediapipe/) is the simplest way for researchers
and developers to build world-class ML solutions and applications for mobile,
desktop/cloud, web and IoT devices.
[MediaPipe](https://google.github.io/mediapipe/) offers cross-platform, customizable
ML solutions for live and streaming media.

![accelerated.png](docs/images/accelerated_small.png) | ![cross_platform.png](docs/images/cross_platform_small.png)
:------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------:
***End-to-End acceleration***: *built-in fast ML inference and processing accelerated even on common hardware* | ***Build one, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT*
***End-to-End acceleration***: *Built-in fast ML inference and processing accelerated even on common hardware* | ***Build once, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT*
![ready_to_use.png](docs/images/ready_to_use_small.png) | ![open_source.png](docs/images/open_source_small.png)
***Ready-to-use solutions***: *Cutting-edge ML solutions demonstrating full power of the framework* | ***Free and open source***: *Framework and solutions both under Apache 2.0, fully extensible and customizable*

@@ -93,7 +92,7 @@ run code search using

## Publications

* [MediaPipe 3D Face Transform](https://mediapipe.page.link/face-geometry-blog)
* [MediaPipe 3D Face Transform](https://developers.googleblog.com/2020/09/mediapipe-3d-face-transform.html)
in Google Developers Blog
* [Instant Motion Tracking With MediaPipe](https://developers.googleblog.com/2020/08/instant-motion-tracking-with-mediapipe.html)
in Google Developers Blog
8 changes: 5 additions & 3 deletions WORKSPACE
Original file line number Diff line number Diff line change
@@ -10,7 +10,7 @@ http_archive(
sha256 = "1dde365491125a3db70731e25658dfdd3bc5dbdfd11b840b3e987ecf043c7ca0",
)
load("@bazel_skylib//lib:versions.bzl", "versions")
versions.check(minimum_bazel_version = "2.0.0")
versions.check(minimum_bazel_version = "3.4.0")


# ABSL cpp library lts_2020_02_25
@@ -324,8 +324,9 @@ maven_install(
"androidx.lifecycle:lifecycle-common:2.2.0",
"androidx.annotation:annotation:aar:1.1.0",
"androidx.appcompat:appcompat:aar:1.1.0-rc01",
"androidx.camera:camera-core:aar:1.0.0-alpha06",
"androidx.camera:camera-camera2:aar:1.0.0-alpha06",
"androidx.camera:camera-core:1.0.0-beta10",
"androidx.camera:camera-camera2:1.0.0-beta10",
"androidx.camera:camera-lifecycle:1.0.0-beta10",
"androidx.constraintlayout:constraintlayout:aar:1.1.3",
"androidx.core:core:aar:1.1.0-rc03",
"androidx.legacy:legacy-support-v4:aar:1.0.0",
@@ -337,6 +338,7 @@ maven_install(
"com.google.flogger:flogger-system-backend:0.3.1",
"com.google.flogger:flogger:0.3.1",
"com.google.guava:guava:27.0.1-android",
"com.google.guava:listenablefuture:1.0",
"junit:junit:4.12",
"org.hamcrest:hamcrest-library:1.3",
],
4 changes: 2 additions & 2 deletions docs/getting_started/hello_world_android.md
Original file line number Diff line number Diff line change
@@ -446,8 +446,8 @@ visible so that we can start seeing frames from the `previewFrameTexture`.
However, before starting the camera, we need to decide which camera we want to
use. [`CameraXPreviewHelper`] inherits from [`CameraHelper`] which provides two
options, `FRONT` and `BACK`. We can pass in the decision from the `BUILD` file
as metadata such that no code change is required to build a another version of
the app using a different camera.
as metadata such that no code change is required to build another version of the
app using a different camera.

Assuming we want to use `BACK` camera to perform edge detection on a live scene
that we view from the camera, add the metadata into `AndroidManifest.xml`:
32 changes: 15 additions & 17 deletions docs/getting_started/install.md
Original file line number Diff line number Diff line change
@@ -42,16 +42,16 @@ apps, see these [instructions](./building_examples.md#ios).

Follow the official
[Bazel documentation](https://docs.bazel.build/versions/master/install-ubuntu.html)
to install Bazel 2.0 or higher.
to install Bazel 3.4 or higher.

For Nvidia Jetson and Raspberry Pi devices with ARM Ubuntu, Bazel needs to
be built from source.

```bash
# For Bazel 3.0.0
wget https://github.com/bazelbuild/bazel/releases/download/3.0.0/bazel-3.0.0-dist.zip
# For Bazel 3.4.0
wget https://github.com/bazelbuild/bazel/releases/download/3.4.0/bazel-3.4.0-dist.zip
sudo apt-get install build-essential openjdk-8-jdk python zip unzip
unzip bazel-3.0.0-dist.zip
unzip bazel-3.4.0-dist.zip
env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
sudo cp output/bazel /usr/local/bin/
```
@@ -221,7 +221,7 @@ build issues.
Follow the official
[Bazel documentation](https://docs.bazel.build/versions/master/install-redhat.html)
to install Bazel 2.0 or higher.
to install Bazel 3.4 or higher.
3. Install OpenCV.
@@ -356,7 +356,7 @@ build issues.
Option 2. Follow the official
[Bazel documentation](https://docs.bazel.build/versions/master/install-os-x.html#install-with-installer-mac-os-x)
to install Bazel 2.0 or higher.
to install Bazel 3.4 or higher.
4. Install OpenCV and FFmpeg.
@@ -427,7 +427,6 @@ build issues.
linkstatic = 1,
visibility = ["//visibility:public"],
)
```
5. Make sure that Python 3 and the Python "six" library are installed.
@@ -506,7 +505,7 @@ next section.
Follow the official
[Bazel documentation](https://docs.bazel.build/versions/master/install-windows.html)
to install Bazel 2.0 or higher.
to install Bazel 3.4 or higher.
6. Set Bazel variables.
@@ -567,7 +566,6 @@ next section.
# I20200514 20:43:12.279618 1200 hello_world.cc:56] Hello World!
# I20200514 20:43:12.279618 1200 hello_world.cc:56] Hello World!
# I20200514 20:43:12.280613 1200 hello_world.cc:56] Hello World!
```
If you run into a build error, please read
@@ -607,14 +605,14 @@ cameras. Alternatively, you use a video file as input.

```bash
username@DESKTOP-TMVLBJ1:~$ curl -sLO --retry 5 --retry-max-time 10 \
https://storage.googleapis.com/bazel/3.0.0/release/bazel-3.0.0-installer-linux-x86_64.sh && \
sudo mkdir -p /usr/local/bazel/3.0.0 && \
chmod 755 bazel-3.0.0-installer-linux-x86_64.sh && \
sudo ./bazel-3.0.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/3.0.0 && \
source /usr/local/bazel/3.0.0/lib/bazel/bin/bazel-complete.bash
username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/3.0.0/lib/bazel/bin/bazel version && \
alias bazel='/usr/local/bazel/3.0.0/lib/bazel/bin/bazel'
https://storage.googleapis.com/bazel/3.4.0/release/bazel-3.4.0-installer-linux-x86_64.sh && \
sudo mkdir -p /usr/local/bazel/3.4.0 && \
chmod 755 bazel-3.4.0-installer-linux-x86_64.sh && \
sudo ./bazel-3.4.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/3.4.0 && \
source /usr/local/bazel/3.4.0/lib/bazel/bin/bazel-complete.bash
username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/3.4.0/lib/bazel/bin/bazel version && \
alias bazel='/usr/local/bazel/3.4.0/lib/bazel/bin/bazel'
```

6. Checkout MediaPipe repository.
11 changes: 5 additions & 6 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -8,15 +8,14 @@ nav_order: 1

--------------------------------------------------------------------------------

## Cross-platform ML solutions made simple
## Live ML anywhere

[MediaPipe](https://google.github.io/mediapipe/) is the simplest way for researchers
and developers to build world-class ML solutions and applications for mobile,
desktop/cloud, web and IoT devices.
[MediaPipe](https://google.github.io/mediapipe/) offers cross-platform, customizable
ML solutions for live and streaming media.

![accelerated.png](images/accelerated_small.png) | ![cross_platform.png](images/cross_platform_small.png)
:------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------:
***End-to-End acceleration***: *built-in fast ML inference and processing accelerated even on common hardware* | ***Build one, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT*
***End-to-End acceleration***: *Built-in fast ML inference and processing accelerated even on common hardware* | ***Build once, deploy anywhere***: *Unified solution works across Android, iOS, desktop/cloud, web and IoT*
![ready_to_use.png](images/ready_to_use_small.png) | ![open_source.png](images/open_source_small.png)
***Ready-to-use solutions***: *Cutting-edge ML solutions demonstrating full power of the framework* | ***Free and open source***: *Framework and solutions both under Apache 2.0, fully extensible and customizable*

@@ -93,7 +92,7 @@ run code search using

## Publications

* [MediaPipe 3D Face Transform](https://mediapipe.page.link/face-geometry-blog)
* [MediaPipe 3D Face Transform](https://developers.googleblog.com/2020/09/mediapipe-3d-face-transform.html)
in Google Developers Blog
* [Instant Motion Tracking With MediaPipe](https://developers.googleblog.com/2020/08/instant-motion-tracking-with-mediapipe.html)
in Google Developers Blog
2 changes: 1 addition & 1 deletion docs/solutions/face_mesh.md
Original file line number Diff line number Diff line change
@@ -277,7 +277,7 @@ only works for a single face. For visual reference, please refer to *Fig. 4*.
* TensorFlow Blog:
[Face and hand tracking in the browser with MediaPipe and TensorFlow.js](https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html)
* Google Developers Blog:
[MediaPipe 3D Face Transform](https://mediapipe.page.link/face-geometry-blog)
[MediaPipe 3D Face Transform](https://developers.googleblog.com/2020/09/mediapipe-3d-face-transform.html)
* Paper:
[Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://arxiv.org/abs/1907.06724)
([poster](https://docs.google.com/presentation/d/1-LWwOMO9TzEVdrZ1CS1ndJzciRHfYDJfbSxH_ke_JRg/present?slide=id.g5986dd4b4c_4_212))
9 changes: 6 additions & 3 deletions docs/tools/tracing_and_profiling.md
Original file line number Diff line number Diff line change
@@ -26,9 +26,10 @@ To enable tracing and profiling of a mediapipe graph:
1. The profiling library must be linked to the framework.
2. Tracing and profiling must be enabled in the graph configuration.

The profiling library is linked to the framework by default. If needed,
the profiling library can be omitted from the framework using the bazel
command line option: `--define MEDIAPIPE_PROFILING=0`.
The profiling library is linked to the framework by default for Desktop.
If needed, it can be omitted from the framework using the bazel command line
option: `--define MEDIAPIPE_PROFILING=0`. For other platforms, you can use the
bazel command line option `--define MEDIAPIPE_PROFILING=1` to link it.

To enable tracing and profiling, the `CalculatorGraphConfig` (in
[calculator.proto](https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator.proto))
@@ -38,6 +39,7 @@ is a simple setup that turns on tracing and keeps 100 seconds of timing events:
```
profiler_config {
trace_enabled: true
enable_profiler: true
trace_log_interval_count: 200
}
```
@@ -147,6 +149,7 @@ we record ten intervals of half a second each. This can be overridden by adding
```bash
profiler_config {
trace_enabled: true
enable_profiler: true
trace_log_path: "/sdcard/profiles/"
}
```
Loading