installation.md

Installation Guide

Prerequisite

• Scavenger is a combination of various server components(API, Frontend, Collector) and agent component. To install scavenger, you need JDK 21 or latest version.
• Scavenger uses 2 ports. if some ports are blocked by your firewall, you should ask the network admin to open the following ports in the firewall. These ports can be configured by the configuration.
  • Agent: Any ===> Collector: This is dependent on your tomcat configuration. By default, it is set as 8080.
  • Agent: Any ===> Collector: This is dependent on your grpc configuration. By default, it is set as 8080.
  • API: This is dependent on your tomcat configuration. By default, it is set as 8081.

Install Collector

Build Collector

Download Collector from the following link. https://github.com/naver/scavenger/releases

Or, You can also build it by yourself.

1. Build Scavenger manually from the Git clone.
2. Run ./gradlew assemble -p scavenger-collector

Installation

Start Collector using the following command.

• e.g) java -jar scavenger-collector.jar

Configuration

You can override any configuration values with -D option.

• java -Darmeria.port=8080 -jar scavenger-collector.jar

To change the http(default 8080) and grpc(default 8080) ports used by Collector, you can change the settings below.

• -Darmeria.port: http, grpc uses 8080 by default

To change the max message size of gRPC in Collector, change the setting below.

• -Darmeria.max-message-size: 10MB by default
  • Supports B, KB, MB, GB, and TB units.

The collector provides two profiles: h2 and local(default)
The profile can be enabled with spring.profiles.active configuration.

The collector uses in-memory H2 in local profile.
This means that the data is initialised every time the collector application starts.
If you want to work around this using file mode, you can enable the h2 profile.

• java -Dspring.profiles.active=h2 -jar scavenger-collector.jar

If you've built your own MySQL, follow the steps below.

1. Install MySQL Server using the official MySQL guide.
2. Log into MySQL Server.
3. Create MySQL Database.

   CREATE DATABASE scavenger;

4. Use the -D option to set the following four properties.
   • spring.profiles.active=mysql
   • spring.datasource.url={mysql url}
   • spring.datasource.username={mysql user name}
   • spring.datasource.password={mysql password}
   • spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

When Building Manually

You can modify default configuration values or add new profiles under scavenger-collector/src/main/resources/.

Install API

Build API

Download API from the following link. https://github.com/naver/scavenger/releases

Or, You can also build it by yourself.

1. Build Scavenger manually from the Git clone.
2. Run ./gradlew assemble -p scavenger-api

Installation

Start API using the following command. When you start the API, you need to set the value scavenger.collector-server-url with the -D option so that the API knows the collector's url.

• e.g) java -Dscavenger.collector-server-url=http://localhost:8080 -jar scavenger-api-boot.jar

Configuration

You can override any configuration values with -D option.

• java -jar scavenger-api-boot.jar

To change the http(default 8081) ports used by API, you can change the settings below.

• server.port: Tomcat uses 8081 by default

The API provides two profiles: h2 and local(default)
The profile can be enabled with spring.profiles.active configuration.

The API uses in-memory H2 in local profile.
This means that the data is initialised every time the API application starts, and the data is independent of the collector because it is using a different h2 than the collector.
If you have enabled the h2 profile on the collector to work around this, you can enable the h2 profile on the API as well.

• java -Dspring.profiles.active=h2 -jar scavenger-api-boot.jar

If you've built your own MySQL, follow the steps below.

If you have already deployed MySQL when installing collector, you can skip ahead to step 4.

1. Install MySQL Server using the official MySQL guide.
2. Log into MySQL Server.
3. Create MySQL Database.

   CREATE DATABASE scavenger;

4. Use the -D option to set the following four properties.
   • spring.profiles.active=mysql
   • spring.datasource.url={mysql url}
   • spring.datasource.username={mysql user name}
   • spring.datasource.password={mysql password}
   • spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

When Building Manually

You can modify default configuration values or add new profiles under scavenger-api/src/main/resources/.

Install Agent

Build Agent

• Download the latest version from https://repo1.maven.org/maven2/com/navercorp/scavenger/scavenger-agent-java/{VERSION}/scavenger-agent-java-{VERSION}.jar
  • You can find out what latest VERSION is in maven central

You can also build it yourself.

1. Build Scavenger manually from the Git clone.
2. Run ./gradlew assemble -p scavenger-agent-java

Configuration

• By default, this is done using the scavenger.conf file.
• It can also be configured via Java system properties and environment variables, but the agent is developed in such a way that it will not work if the scavenger.conf file does not exist.
• Therefore, it is recommended to use scavenger.conf as the default and only to override settings.
• The scavenger.conf can be set and downloaded via the UI.

• Example scavenger.conf

apiKey=eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183
serverUrl=http://10.106.93.41:8081
environment=dev
appName=apiserver
codeBase=webapps/ROOT/WEB-INF
packages=com.navercorp

• Scavenger configuration

configuration	default	Description
enabled	true	Option to enable or disable Scavenger.
apiKey (*)		The key used to identify the project. ex) ec69466e-aaaa-bbbb-cccc-3021386f997e
serverUrl (*)		URL of the Scavenger collector to send collection data to. ex) http://collector-test.scavenger.navercorp.com
appName (*)		A name for the application. ex) foo_apiserver
packages (*)		a list of packages to scan. Packages are separated by , and all subpackages are scanned. ex) io.scavenger.javaagent, io.scavenger.dashboard
codeBase		In Scavenger, a codebase is a representation of the code being scaned at the time of collection. If you change the content of the code you're scaning, or change settings such as which packages are scaned, Scavenger will treat it as a new codebase and upload it. When a new codebase is uploaded, new methods are inserted and methods from the old codebase are removed by a garbage collector that runs at regular intervals. Each method is identified using a signature, which is a function signal Call information is also recorded dependent on the method in each codebase, so call information for methods whose signatures have not changed is preserved between codebase uploads. This option specifies the location of the codebase. If not specified, the Scavenger agent will use the default classpath of the class loader to scan for classes. If your application loads classes on its own from a path other than the default classpath, such as Tomcat, you must explicitly specify this option as shown in the example below for proper ingestion. In your respective deployment environment, you can set this option as follows. jar file(executable jar) - Specify a directory containing a single JAR file, or specify the JAR file itself. ex) path/to/example.jar extracted jar(layertools) - Specify /lib, /classes in BOOT-INF ex) BOOT-INF/lib, BOOT-INF/classes war file(web archive) - Specify the WEB-INF directory. ex) path/to/WEB-INF Gradle worksapce - build/libs/ Maven workspace - target/ If you're using Tomcat, you can specify the path to the webapp where the code you want to scan resides, such as codeBase = /home1/irteam/service/tomcat/webapps/ROOT/WEB-INF.
excludePackages		Specifies which of the subpackages specified in packages you want to exclude from tracking. ex) foo.bar, baz.qux
annotations		Target sub methods of the annotated class to scan. ex) @org.springframework.web.bind.annotation.RestController, @org.springframework.stereotype.Controller
additionalPackages		Specifies the packages excluded by annotations that you want to include in the scan. ex) io.scavenger.javaagent, io.scavenger.dashboard
excludeByRegex		Specifies regular expressions you want to exclude from tracking. This option performs filtering on the result of packages - excludedPackages + additionalPackages. As this may impact performance, so it is recommended to use package-level filtering wherever possible. ex) ^com\.example\.exclude(\..*)?$
additionalByRegex		Specifies regular expressions excluded by annotations that you want to include in the scan. This option may impact performance, so it is recommended to use package-level filtering wherever possible. ex) ^com\.example\.additional(\..*)?$
methodVisibility	protected	Filter what to scan based on the access controller of the method. Values are public, protected, package-private, and private, with the cost increasing the further back you go. Package-private and private are unlikely to be used in practice, as they can be easily removed using static code analysis in the IDE.
excludeConstructors	false	Specifies whether constructors should be excluded from scaning. For Spring applications, we recommend setting this to true to exclude constructors of beans from collection because they are called automatically.
excludeGetterSetter	false	Specifies whether getters and setters should be excluded from scaning. Methods in the form of componentN() that are created by the data class of your Kotlin are also excluded. These are often automatically generated by libraries like Lombok, so we recommend setting them to true to exclude them from collection.
asyncCodeBaseScanMode	false	Scavenger first aggregates the codebase before instrumentation. This causes your application to be delayed from booting by Scavenger. We recommend setting this to true after you have verified that your Scavenger settings are working properly.
environment		Specifies the application environment values. This can be used if you maintain different codebases for different deployment stages, as they are collected as if they were separate codebases based on that value. ex) dev / alpha / beta / real In the Scavenger UI, data collected in separate environments can be specified multiple times when taking a snapshot to check for dead code all at once.
appVersion	unspecified	Specify the application version. This option does not affect collection. Instead, it is useful if you want to specify which version of the application you are scaning. ex) 1.0.0
hostname		Specifies that it should be recorded differently from the existing hostname. This option does not affect ingestion, but can be useful when determining which host the agent ran on.
debugMode	false	You can check the logs to see which methods are being traced and whether they are being called. Enabling this mode will have a serious impact on the performance of your application, and we do not recommend using it outside of a development environment.
maxMethodsCount	100000	Sets a maximum limit on the number of methods that can be tracked.

Installation

• Setting up the agent via -javaagent:path/to/scavenger-agent.jar
• Specify the config file via -Dscavenger.configuration=path/to/scavenger.conf
• To change the max message size of gRPC in agent, change the setting below.
• -Dscavenger.max-message-size: 10MB by default
  • Supports B, KB, MB, GB, and TB units.

We recommend tuning your codeBase configuration for minimizing tracing ranges before changing scavenger.max-message-size.

export JAVA_OPTS="$JAVA_OPTS -Dscavenger.configuration=$CATALINA_BASE/conf/scavenger.conf"
export JAVA_OPTS="$JAVA_OPTS -javaagent:/home1/irteam/scavenger-agent.jar"