Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update the AD settings for opensearch. #47

Merged
merged 11 commits into from
May 21, 2021
Merged

update the AD settings for opensearch. #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7996ed5
update the AD settings for opensearch.
May 17, 2021
2077548
Update settings name prefix from opensearch to plugins.
May 18, 2021
dfbbea6
Update the version of checkstyle to keep sync with opensearch.
May 18, 2021
9e398fd
Update the Opensearch from 1.0.0-beta1 to the main branch to take the
May 19, 2021
eb4b0ab
Merge branch 'opensearch-project:main' into main
spbjss May 19, 2021
6e5c9e9
Merge branch 'main' of https://github.com/spbjss/anomaly-detection
May 19, 2021
8fec893
1. Update the opensearch, common-utils and job-scheduler version from…
May 19, 2021
40b4d99
Update the opensearch version from 1.0.0 to 1.0.0-rc1
May 20, 2021
45ffd32
Update the version informations.
May 20, 2021
5d368a6
Update the rc1 related version config
May 20, 2021
4325929
Update checkstyle version to 8.29 to follow opensearch 1.x
May 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/CI.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,37 +33,37 @@ jobs:
with:
repository: 'opensearch-project/OpenSearch'
path: OpenSearch
ref: 'main'
ref: '1.x'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay!

- name: Build OpenSearch
working-directory: ./OpenSearch
run: ./gradlew publishToMavenLocal -Dbuild.version_qualifier=beta1 -Dbuild.snapshot=false
run: ./gradlew publishToMavenLocal -Dbuild.version_qualifier=rc1 -Dbuild.snapshot=false

# dependencies: common-utils
- name: Checkout common-utils
uses: actions/checkout@v2
with:
ref: '1.0.0-beta1'
ref: 'main' # TODO: update to the right branch name once it's ready. e.g. 1.x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add instructions for our branching strategy for plugins in the plugins repo. @dblock
I can take care of it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: opensearch-project/opensearch-plugins#35

repository: 'opensearch-project/common-utils'
path: common-utils
- name: Build common-utils
working-directory: ./common-utils
run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-beta1
run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-rc1

# dependencies: job-scheduler
- name: Checkout job-scheduler
uses: actions/checkout@v2
with:
ref: '1.0.0-beta1'
ref: 'main' # TODO: update to the right branch name once it's ready. e.g. 1.x
repository: 'opensearch-project/job-scheduler'
path: job-scheduler

- name: Build job-scheduler
working-directory: ./job-scheduler
run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-beta1 -Dbuild.snapshot=false
run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-rc1 -Dbuild.snapshot=false
- name: Assemble job-scheduler
working-directory: ./job-scheduler
run: |
./gradlew assemble -Dopensearch.version=1.0.0-beta1 -Dbuild.snapshot=false
./gradlew assemble -Dopensearch.version=1.0.0-rc1 -Dbuild.snapshot=false
echo "Creating ../src/test/resources/job-scheduler ..."
mkdir -p ../src/test/resources/job-scheduler
pwd
Expand All @@ -75,11 +75,11 @@ jobs:

- name: Build and Run Tests
run: |
./gradlew build -Dopensearch.version=1.0.0-beta1
./gradlew build -Dopensearch.version=1.0.0-rc1

- name: Publish to Maven Local
run: |
./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-beta1
./gradlew publishToMavenLocal -Dopensearch.version=1.0.0-rc1

- name: Upload Coverage Report
uses: codecov/codecov-action@v1
Expand Down
6 changes: 3 additions & 3 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import org.opensearch.gradle.test.RestIntegTestTask
buildscript {
ext {
opensearch_group = "org.opensearch"
opensearch_version = System.getProperty("opensearch.version", "1.0.0-beta1")
opensearch_version = System.getProperty("opensearch.version", "1.0.0-rc1")
common_utils_version = '1.0.0.0-beta1'
job_scheduler_version = '1.0.0.0-beta1'
}
Expand Down Expand Up @@ -368,8 +368,8 @@ checkstyle {
dependencies {
compile "org.opensearch:opensearch:${opensearch_version}"
compileOnly "org.opensearch.plugin:opensearch-scripting-painless-spi:${opensearch_version}"
compileOnly "com.amazon.opendistroforelasticsearch:opensearch-job-scheduler-spi:${job_scheduler_version}"
compile "com.amazon.opendistroforelasticsearch:common-utils:${common_utils_version}"
compileOnly "org.opensearch:opensearch-job-scheduler-spi:${job_scheduler_version}"
compile "org.opensearch:common-utils:${common_utils_version}"
compile group: 'com.google.guava', name: 'guava', version:'29.0-jre'
compile group: 'org.apache.commons', name: 'commons-math3', version: '3.6.1'
compile group: 'com.google.code.gson', name: 'gson', version: '2.8.6'
Expand Down
16 changes: 8 additions & 8 deletions src/main/java/org/opensearch/ad/AnomalyDetectorJobRunner.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,16 +71,16 @@
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.common.xcontent.XContentType;
import org.opensearch.commons.InjectSecurity;
import org.opensearch.commons.authuser.User;
import org.opensearch.jobscheduler.spi.JobExecutionContext;
import org.opensearch.jobscheduler.spi.LockModel;
import org.opensearch.jobscheduler.spi.ScheduledJobParameter;
import org.opensearch.jobscheduler.spi.ScheduledJobRunner;
import org.opensearch.jobscheduler.spi.schedule.IntervalSchedule;
import org.opensearch.jobscheduler.spi.utils.LockService;
import org.opensearch.threadpool.ThreadPool;

import com.amazon.opendistroforelasticsearch.commons.InjectSecurity;
import com.amazon.opendistroforelasticsearch.commons.authuser.User;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.JobExecutionContext;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.LockModel;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.ScheduledJobParameter;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.ScheduledJobRunner;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.IntervalSchedule;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.utils.LockService;
import com.google.common.base.Throwables;
import com.google.common.collect.ImmutableList;

Expand Down
32 changes: 29 additions & 3 deletions src/main/java/org/opensearch/ad/AnomalyDetectorPlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@
import org.opensearch.ad.rest.RestStatsAnomalyDetectorAction;
import org.opensearch.ad.settings.AnomalyDetectorSettings;
import org.opensearch.ad.settings.EnabledSetting;
import org.opensearch.ad.settings.LegacyOpenDistroAnomalyDetectorSettings;
import org.opensearch.ad.stats.ADStat;
import org.opensearch.ad.stats.ADStats;
import org.opensearch.ad.stats.StatNames;
Expand Down Expand Up @@ -172,6 +173,9 @@
import org.opensearch.common.xcontent.XContentParserUtils;
import org.opensearch.env.Environment;
import org.opensearch.env.NodeEnvironment;
import org.opensearch.jobscheduler.spi.JobSchedulerExtension;
import org.opensearch.jobscheduler.spi.ScheduledJobParser;
import org.opensearch.jobscheduler.spi.ScheduledJobRunner;
import org.opensearch.monitor.jvm.JvmService;
import org.opensearch.plugins.ActionPlugin;
import org.opensearch.plugins.Plugin;
Expand All @@ -185,9 +189,6 @@
import org.opensearch.threadpool.ThreadPool;
import org.opensearch.watcher.ResourceWatcherService;

import com.amazon.opendistroforelasticsearch.jobscheduler.spi.JobSchedulerExtension;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.ScheduledJobParser;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.ScheduledJobRunner;
import com.amazon.randomcutforest.serialize.RandomCutForestSerDe;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
Expand Down Expand Up @@ -637,6 +638,31 @@ public List<Setting<?>> getSettings() {

List<Setting<?>> systemSetting = ImmutableList
.of(
LegacyOpenDistroAnomalyDetectorSettings.MAX_SINGLE_ENTITY_ANOMALY_DETECTORS,
LegacyOpenDistroAnomalyDetectorSettings.MAX_MULTI_ENTITY_ANOMALY_DETECTORS,
LegacyOpenDistroAnomalyDetectorSettings.MAX_ANOMALY_FEATURES,
LegacyOpenDistroAnomalyDetectorSettings.REQUEST_TIMEOUT,
LegacyOpenDistroAnomalyDetectorSettings.DETECTION_INTERVAL,
LegacyOpenDistroAnomalyDetectorSettings.DETECTION_WINDOW_DELAY,
LegacyOpenDistroAnomalyDetectorSettings.AD_RESULT_HISTORY_ROLLOVER_PERIOD,
LegacyOpenDistroAnomalyDetectorSettings.AD_RESULT_HISTORY_MAX_DOCS,
LegacyOpenDistroAnomalyDetectorSettings.MAX_RETRY_FOR_UNRESPONSIVE_NODE,
LegacyOpenDistroAnomalyDetectorSettings.COOLDOWN_MINUTES,
LegacyOpenDistroAnomalyDetectorSettings.BACKOFF_MINUTES,
LegacyOpenDistroAnomalyDetectorSettings.BACKOFF_INITIAL_DELAY,
LegacyOpenDistroAnomalyDetectorSettings.MAX_RETRY_FOR_BACKOFF,
LegacyOpenDistroAnomalyDetectorSettings.AD_RESULT_HISTORY_RETENTION_PERIOD,
LegacyOpenDistroAnomalyDetectorSettings.MODEL_MAX_SIZE_PERCENTAGE,
LegacyOpenDistroAnomalyDetectorSettings.MAX_ENTITIES_PER_QUERY,
LegacyOpenDistroAnomalyDetectorSettings.MAX_ENTITIES_FOR_PREVIEW,
LegacyOpenDistroAnomalyDetectorSettings.INDEX_PRESSURE_SOFT_LIMIT,
LegacyOpenDistroAnomalyDetectorSettings.MAX_PRIMARY_SHARDS,
LegacyOpenDistroAnomalyDetectorSettings.FILTER_BY_BACKEND_ROLES,
LegacyOpenDistroAnomalyDetectorSettings.MAX_CACHE_MISS_HANDLING_PER_SECOND,
LegacyOpenDistroAnomalyDetectorSettings.MAX_BATCH_TASK_PER_NODE,
LegacyOpenDistroAnomalyDetectorSettings.BATCH_TASK_PIECE_INTERVAL_SECONDS,
LegacyOpenDistroAnomalyDetectorSettings.MAX_OLD_AD_TASK_DOCS_PER_DETECTOR,
LegacyOpenDistroAnomalyDetectorSettings.BATCH_TASK_PIECE_SIZE,
AnomalyDetectorSettings.MAX_SINGLE_ENTITY_ANOMALY_DETECTORS,
AnomalyDetectorSettings.MAX_MULTI_ENTITY_ANOMALY_DETECTORS,
AnomalyDetectorSettings.MAX_ANOMALY_FEATURES,
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/opensearch/ad/model/ADTask.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@
import org.opensearch.common.xcontent.ToXContentObject;
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.commons.authuser.User;

import com.amazon.opendistroforelasticsearch.commons.authuser.User;
import com.google.common.base.Objects;

/**
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/opensearch/ad/model/AnomalyDetector.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@
import org.opensearch.common.xcontent.ToXContentObject;
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.commons.authuser.User;
import org.opensearch.index.query.QueryBuilder;
import org.opensearch.index.query.QueryBuilders;

import com.amazon.opendistroforelasticsearch.commons.authuser.User;
import com.google.common.base.Objects;
import com.google.common.collect.ImmutableList;

Expand Down
12 changes: 6 additions & 6 deletions src/main/java/org/opensearch/ad/model/AnomalyDetectorJob.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,13 @@
import org.opensearch.common.xcontent.ToXContentObject;
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.commons.authuser.User;
import org.opensearch.jobscheduler.spi.ScheduledJobParameter;
import org.opensearch.jobscheduler.spi.schedule.CronSchedule;
import org.opensearch.jobscheduler.spi.schedule.IntervalSchedule;
import org.opensearch.jobscheduler.spi.schedule.Schedule;
import org.opensearch.jobscheduler.spi.schedule.ScheduleParser;

import com.amazon.opendistroforelasticsearch.commons.authuser.User;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.ScheduledJobParameter;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.CronSchedule;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.IntervalSchedule;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.Schedule;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.ScheduleParser;
import com.google.common.base.Objects;

/**
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/opensearch/ad/model/AnomalyResult.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@
import org.opensearch.common.xcontent.ToXContentObject;
import org.opensearch.common.xcontent.XContentBuilder;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.commons.authuser.User;

import com.amazon.opendistroforelasticsearch.commons.authuser.User;
import com.google.common.base.Objects;

/**
Expand Down
3 changes: 1 addition & 2 deletions src/main/java/org/opensearch/ad/rest/handler/IndexAnomalyDetectorActionHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@
import org.opensearch.common.xcontent.NamedXContentRegistry;
import org.opensearch.common.xcontent.XContentFactory;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.commons.authuser.User;
import org.opensearch.index.query.BoolQueryBuilder;
import org.opensearch.index.query.QueryBuilder;
import org.opensearch.index.query.QueryBuilders;
Expand All @@ -77,8 +78,6 @@
import org.opensearch.search.builder.SearchSourceBuilder;
import org.opensearch.transport.TransportService;

import com.amazon.opendistroforelasticsearch.commons.authuser.User;

/**
* Anomaly detector REST action handler to process POST/PUT request.
* POST request is for creating anomaly detector.
Expand Down
5 changes: 2 additions & 3 deletions src/main/java/org/opensearch/ad/rest/handler/IndexAnomalyDetectorJobActionHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,10 @@
import org.opensearch.common.xcontent.NamedXContentRegistry;
import org.opensearch.common.xcontent.XContentFactory;
import org.opensearch.common.xcontent.XContentParser;
import org.opensearch.jobscheduler.spi.schedule.IntervalSchedule;
import org.opensearch.jobscheduler.spi.schedule.Schedule;
import org.opensearch.rest.RestStatus;

import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.IntervalSchedule;
import com.amazon.opendistroforelasticsearch.jobscheduler.spi.schedule.Schedule;

/**
* Anomaly detector job REST action handler to process POST/PUT request.
*/
Expand Down
Loading