This repository has been archived by the owner on Aug 9, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 31
Headless chrome creation script and readme file #229
Merged
harold-wang
merged 5 commits into
opendistro-for-elasticsearch:dev
from
harold-wang:headless_chromium
Dec 8, 2020
Merged
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
8207cd5
Headless chrome creation script and readme file
harold-wang fd2a78d
Rename README_chrome_headless.md to README.md
harold-wang 7ebf21e
Add README.md
harold-wang aaef3fd
add comments on how to run chrome to get pdf and png report
harold-wang 3d89dd6
Change test web from google.com to https://opendistro.github.io/for-e…
harold-wang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
## Chrome Binaries for Kibana Reports used by Puppeteer | ||
Headless Chrome for Linux and Mac are chrome binaries which are significantly smaller than the standard binaries shipped by Google and Puppeteer. | ||
Chrome binary can be built from shell script build_headless_chrome.sh for Mac, Linux x64 and Linux arm64, | ||
output of script is called headless_shell. | ||
| ||
|
||
## Puppeteer's Chrome version | ||
|
||
Find the puppeteer version used in Kibana node_modules.json and get the associated chrome SHA to build from crrev.com and puppeteer repositories. Puppeteer 1.9 uses rev 674921 with commit sha as 312d84c8ce62810976feda0d3457108a6dfff9e6) | ||
|
||
## headless Chrome folder structure | ||
-chromium | ||
|-chromium | ||
|-chromium | ||
|-src | ||
|-out | ||
|-headless | ||
|-headless_shell # output of scripts | ||
| ||
## How to generate the headless_chrome | ||
This is a shell script to set environment variable, download the source code and build the executable. | ||
| ||
## Commands to create headless_chrome | ||
Run below command to create headless_shell for each platform | ||
| ||
headless-chrome.sh chrome-version-SHA (arch_name (arm64)) | ||
. Mac x64: ./build_headless_chrome.sh <chrome-version-SHA> | ||
. Linux x64: ./build_headless_chrome.sh <chrome-version-SHA> | ||
. Linux arm64: ./build_headless_chrome.sh <chrome-version-SHA> arm64 | ||
| ||
# How to call in Command line: | ||
>> please use what you used for testing this << replace me >> | ||
|
||
## Headless Chromium for MAC | ||
# Files: | ||
headless_shell | ||
libswiftshader_libGLESv2.dylib | ||
|
||
## Headless Chromium for Linux (arm64 and x64) | ||
# Files: | ||
headless_shell | ||
swiftshader | ||
|-libEGL.so | ||
|-libEGL.so.TOC | ||
|-libGLESv2.so | ||
|-libGLESv2.so.TOC | ||
# Additional libaries: | ||
libnss3-dev | ||
fonts-liberation | ||
libfontconfig | ||
(installation command example : sudo apt update && apt install libnss3-dev fonts-liberation libfontconfig) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These dependencies are for Ubuntu, for CentOS I had to install a lot more
I listed them in this PR #222, could you update it here? |
173 changes: 173 additions & 0 deletions
173
kibana-reports/rendering-engine/headless-chrome/build_headless_chrome.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
#!/bin/bash | ||
|
||
# Initializes a Linux environment. This need only be done once per | ||
# machine. The OS needs to be a flavor that supports apt get, such as Ubuntu. | ||
|
||
function generateArgs { | ||
if [ $1 == 'linux' ]; then | ||
echo 'import("//build/args/headless.gn") | ||
is_component_build = false | ||
remove_webcore_debug_symbols = true | ||
enable_nacl = false | ||
is_debug = false | ||
symbol_level = 0 | ||
use_kerberos = false' > args.gn | ||
elif [ $1 == 'darwin' ]; then | ||
echo '#args configuration | ||
|
||
icu_use_data_file = false | ||
v8_use_external_startup_data = false | ||
remove_webcore_debug_symbols = true | ||
use_kerberos = false | ||
use_libpci = false | ||
use_pulseaudio = false | ||
use_udev = false | ||
is_debug = false | ||
symbol_level = 0 | ||
is_component_build = false | ||
enable_nacl = false | ||
enable_print_preview = false | ||
enable_basic_printing = false | ||
enable_remoting = false | ||
use_alsa = false | ||
use_cups = false | ||
use_dbus = false | ||
use_gio = false | ||
' > args.gn | ||
fi | ||
} | ||
|
||
ARGC=("$#") | ||
|
||
if [ $ARGC -lt 1 ]; | ||
then | ||
echo "format: build_headless_chrome.sh {chrome_source_version} (arch_name)" | ||
echo "Mac x64: ./build_headless_chrome.sh 312d84c8ce62810976feda0d3457108a6dfff9e6" | ||
echo "Linux x64: ./build_headless_chrome.sh 312d84c8ce62810976feda0d3457108a6dfff9e6" | ||
echo "Linux arm64: ./build_headless_chrome.sh 312d84c8ce62810976feda0d3457108a6dfff9e6 arm64" | ||
exit | ||
fi | ||
|
||
source_version=$1 | ||
|
||
if [ $ARGC -lt 2 ]; | ||
then | ||
arch_name="x64" | ||
else | ||
arch_name=$2 | ||
fi | ||
|
||
if ! [ -x "$(command -v python)" ]; then | ||
echo "Python is not found, please install python or setup python environment properly" | ||
exit | ||
fi | ||
|
||
# Launch the cross-platform init script using a relative path | ||
# from this script's location. | ||
mkdir -p ~/chromium | ||
|
||
if [ "$#" -eq 2 ]; then | ||
arch_name=$2 | ||
fi | ||
|
||
current_folder=$(pwd) | ||
|
||
# find the current platform | ||
platform_name='unknown' | ||
if [[ "$OSTYPE" == "linux-gnu"* ]]; then | ||
platform_name='linux' | ||
elif [[ "$OSTYPE" == "darwin"* ]]; then | ||
platform_name='darwin' | ||
elif [[ "$OSTYPE" == "win32" ]]; then | ||
platform_name='windows' | ||
fi | ||
|
||
if [[ "$platform_name" == "unknown" ]]; then | ||
echo "platform is" $platform_name | ||
exit | ||
fi | ||
|
||
echo "source_version = " $source_version | ||
echo "platform_name = " $platform_name | ||
echo "arch_name = " $arch_name | ||
generateArgs $platform_name | ||
|
||
# Configure git | ||
git config --global core.autocrlf false | ||
git config --global core.filemode false | ||
git config --global branch.autosetuprebase always | ||
cd chromium | ||
|
||
# Grab Chromium's custom build tools, if they aren't already installed | ||
# (On Windows, they are installed before this Python script is run) | ||
if ! [ -d "depot_tools" ] | ||
then | ||
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git | ||
fi | ||
|
||
# Put depot_tools on the path so we can properly run the fetch command | ||
export PATH="$PATH:${HOME}/chromium/depot_tools" | ||
echo ${HOME}/chromium/depot_tools | ||
|
||
# Fetch the Chromium source code | ||
|
||
if [ -d 'chromium' ]; then | ||
echo "chromium src aready exists, please delete it and retry..." | ||
exit | ||
fi | ||
|
||
mkdir -p chromium | ||
cd chromium | ||
pwd | ||
|
||
# Build Linux deps | ||
echo "fetching chromium..." | ||
fetch chromium | ||
|
||
|
||
# Build Linux deps | ||
|
||
cd src | ||
if [[ arch_name -eq "arm64" ]]; then | ||
./build/linux/sysroot_scripts/install-sysroot.py --arch=$arch_name | ||
./build/install-build-deps.sh | ||
fi | ||
|
||
|
||
# Set to "arm" to build for ARM on Linux | ||
echo 'Building Chromium ' $source_version ' for ' $arch_name | ||
|
||
# Sync the codebase to the correct version, syncing master first | ||
# to ensure that we actually have all the versions we may refer to | ||
echo 'Syncing source code' | ||
|
||
|
||
git checkout -f master | ||
git fetch -f origin | ||
gclient sync --with_branch_heads --with_tags --jobs 16 | ||
git checkout $source_version | ||
gclient sync --with_branch_heads --with_tags --jobs 16 | ||
gclient runhooks | ||
echo "current_folder :" $current_folder | ||
|
||
platform_build_args=$current_folder'/args.gn' | ||
#platform_build_args=$current_folder/chromium/build_chromium/$platform_name/args.gn | ||
|
||
outputDir='headless' | ||
mkdir -p 'out/headless' | ||
|
||
echo "platform_build_args :" $platform_build_args | ||
|
||
cp $platform_build_args 'out/headless/args.gn' | ||
echo "platform_build_args :" $platform_build_args | ||
echo 'target_cpu = '\"$arch_name\" >> 'out/headless/args.gn' | ||
|
||
gn gen out/headless | ||
|
||
autoninja -C out/headless headless_shell | ||
|
||
if [[ ($platform_name != "Windows" && $arch_name != 'arm64') ]]; then | ||
echo 'Optimizing headless_shell' | ||
mv out/headless/headless_shell out/headless/headless_shell_raw | ||
strip -o out/headless/headless_shell out/headless/headless_shell_raw | ||
fi |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's the purpose of these two lines?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this was a review comment/ @harold-wang can you pls update here. thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, saw the commits