Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support OpenSearch 2 by removing deprecated mapping types #3

Merged
merged 12 commits into from
Sep 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/2.4.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@1.x
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@main
with:
cluster-version: 1
cluster-version: 2
mauricebolhuis marked this conversation as resolved.
Show resolved Hide resolved
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.4
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/2.5.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@1.x
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@main
with:
cluster-version: 1
cluster-version: 2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.5
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/2.6.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@1.x
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@main
with:
cluster-version: 1
cluster-version: 2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.6
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/2.7.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@1.x
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@main
with:
cluster-version: 1
cluster-version: 2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.7
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/jruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ jobs:
sudo sysctl -w vm.swappiness=1
sudo sysctl -w fs.file-max=262144
sudo sysctl -w vm.max_map_count=262144
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@1.x
- uses: opensearch-project/opensearch-ruby/.github/actions/opensearch@main
with:
cluster-version: 1
cluster-version: 2
- uses: ruby/setup-ruby@v1
with:
ruby-version: jruby-9.3
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 1.0.0

* Support OpenSearch 2 by removing deprecated type and include_type_name parameters

## 0.1.1

* Pin opensearch-ruby to '~> 1.1' because OpenSearch 2 is not supported yet
Expand Down
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
source 'https://rubygems.org'

gem "rake", "~> 12"
gem "opensearch-ruby", '~> 1.0'
gem "opensearch-ruby", '~> 2.0'
gem "pry"
gem "ansi"
gem "cane"
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ The libraries are compatible with Ruby 2.4 and higher.

We follow Ruby’s own maintenance policy and officially support all currently maintained versions per [Ruby Maintenance Branches](https://www.ruby-lang.org/en/downloads/branches/).

The version numbers follow the OpenSearch major versions. Currently the `main` branch is compatible with version `1.x` of the OpenSearch stack.
The version numbers follow the OpenSearch major versions. Currently the `main` branch is compatible with version `2.x` of the OpenSearch stack.

| Rubygem | | OpenSearch |
|:-------------:|:-:| :-----------: |
| main | → | 1.x |
| main | → | 2.x |

## Usage

Expand Down
6 changes: 3 additions & 3 deletions opensearch-model/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The library version numbers follow the OpenSearch major versions. The `main` bra

| Rubygem | | OpenSearch |
|:-------------:|:-:| :-----------: |
| main | → | 1.x |
| main | → | 2.x |

## Installation

Expand Down Expand Up @@ -523,10 +523,10 @@ class Indexer
case operation.to_s
when /index/
record = Article.find(record_id)
Client.index index: 'articles', type: 'article', id: record.id, body: record.__opensearch__.as_indexed_json
Client.index index: 'articles', id: record.id, body: record.__opensearch__.as_indexed_json
when /delete/
begin
Client.delete index: 'articles', type: 'article', id: record_id
Client.delete index: 'articles', id: record_id
rescue OpenSearch::Transport::Transport::Errors::NotFound
logger.debug "Article not found, ID: #{record_id}"
end
Expand Down
1 change: 0 additions & 1 deletion opensearch-model/examples/activerecord_article.rb
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ class Article < ActiveRecord::Base

client.indices.delete index: 'articles' rescue nil
client.bulk index: 'articles',
type: 'article',
body: Article.all.as_json.map { |a| { index: { _id: a.delete('id'), data: a } } },
refresh: true

Expand Down
3 changes: 1 addition & 2 deletions opensearch-model/examples/couchbase_article.rb
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,12 @@ class Article < Couchbase::Model

client.indices.delete index: 'articles' rescue nil
client.bulk index: 'articles',
type: 'article',
body: Article.find(['1', '2', '3']).map { |a|
{ index: { _id: a.id, data: a.attributes } }
},
refresh: true

response = Article.search 'foo', index: 'articles', type: 'article';
response = Article.search 'foo', index: 'articles';

Pry.start(binding, prompt: lambda { |obj, nest_level, _| '> ' },
input: StringIO.new('response.records.to_a'),
Expand Down
1 change: 0 additions & 1 deletion opensearch-model/examples/mongoid_article.rb
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ def as_indexed_json(options={})

client.indices.delete index: 'articles' rescue nil
client.bulk index: 'articles',
type: 'article',
body: Article.all.map { |a| { index: { _id: a.id, data: a.attributes } } },
refresh: true

Expand Down
1 change: 0 additions & 1 deletion opensearch-model/examples/ohm_article.rb
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ def records

Article.__opensearch__.client.indices.delete index: 'articles' rescue nil
Article.__opensearch__.client.bulk index: 'articles',
type: 'article',
body: Article.all.map { |a| { index: { _id: a.id, data: a.attributes } } },
refresh: true

Expand Down
1 change: 0 additions & 1 deletion opensearch-model/examples/riak_article.rb
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ class Article

client.indices.delete index: 'articles' rescue nil
client.bulk index: 'articles',
type: 'article',
body: Article.all.map { |a|
{ index: { _id: a.key, data: JSON.parse(a.robject.raw_data) } }
}.as_json,
Expand Down
6 changes: 2 additions & 4 deletions opensearch-model/lib/opensearch/model/importing.rb
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,9 @@ module ClassMethods
#
# Article.import refresh: true
#
# @example Import the records into a different index/type than the default one
# @example Import the records into a different index than the default one
#
# Article.import index: 'my-new-index', type: 'my-other-type'
# Article.import index: 'my-new-index'
#
# @example Pass an ActiveRecord scope to limit the imported records
#
Expand Down Expand Up @@ -141,7 +141,6 @@ def import(options={}, &block)
errors = []
refresh = options.delete(:refresh) || false
target_index = options.delete(:index) || index_name
target_type = options.delete(:type) || document_type
transform = options.delete(:transform) || __transform
pipeline = options.delete(:pipeline)
return_value = options.delete(:return) || 'count'
Expand All @@ -161,7 +160,6 @@ def import(options={}, &block)
__find_in_batches(options) do |batch|
params = {
index: target_index,
type: target_type,
body: __batch_to_bulk(batch, transform)
}

Expand Down
17 changes: 4 additions & 13 deletions opensearch-model/lib/opensearch/model/indexing.rb
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,12 @@ def as_json(options={})
# Wraps the [index mappings](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html)
#
class Mappings
attr_accessor :options, :type
attr_accessor :options

# @private
TYPES_WITH_EMBEDDED_PROPERTIES = %w(object nested)

def initialize(type = nil, options={})
@type = type
def initialize(options={})
@options = options
@mapping = {}
end
Expand Down Expand Up @@ -87,11 +86,7 @@ def indexes(name, options={}, &block)
end

def to_hash
if @type
{ @type.to_sym => @options.merge( properties: @mapping ) }
else
@options.merge( properties: @mapping )
end
@options.merge( properties: @mapping )
end

def as_json(options={})
Expand Down Expand Up @@ -152,7 +147,7 @@ module ClassMethods
# when it doesn't already define them. Use the `__opensearch__` proxy otherwise.
#
def mapping(options={}, &block)
@mapping ||= Mappings.new(document_type, options)
@mapping ||= Mappings.new(options)

@mapping.options.update(options) unless options.empty?

Expand Down Expand Up @@ -372,7 +367,6 @@ def index_document(options={})
request = { index: index_name,
id: id,
body: document }
request.merge!(type: document_type) if document_type

client.index(request.merge!(options))
end
Expand All @@ -393,7 +387,6 @@ def index_document(options={})
def delete_document(options={})
request = { index: index_name,
id: self.id }
request.merge!(type: document_type) if document_type

client.delete(request.merge!(options))
end
Expand Down Expand Up @@ -434,7 +427,6 @@ def update_document(options={})
request = { index: index_name,
id: self.id,
body: { doc: attributes } }
request.merge!(type: document_type) if document_type

client.update(request.merge!(options))
end
Expand All @@ -461,7 +453,6 @@ def update_document_attributes(attributes, options={})
request = { index: index_name,
id: self.id,
body: { doc: attributes } }
request.merge!(type: document_type) if document_type

client.update(request.merge!(options))
end
Expand Down
7 changes: 3 additions & 4 deletions opensearch-model/lib/opensearch/model/searching.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ def initialize(klass, query_or_payload, options={})
@options = options

__index_name = options[:index] || klass.index_name
__document_type = options[:type] || klass.document_type

case
# search query: ...
Expand All @@ -54,9 +53,9 @@ def initialize(klass, query_or_payload, options={})
end

if body
@definition = { index: __index_name, type: __document_type, body: body }.update options
@definition = { index: __index_name, body: body }.update options
else
@definition = { index: __index_name, type: __document_type, q: q }.update options
@definition = { index: __index_name, q: q }.update options
end
end

Expand All @@ -71,7 +70,7 @@ def execute!

module ClassMethods

# Provides a `search` method for the model to easily search within an index/type
# Provides a `search` method for the model to easily search within an index
# corresponding to the model settings.
#
# @param query_or_payload [String,Hash,Object] The search request definition
Expand Down
2 changes: 1 addition & 1 deletion opensearch-model/lib/opensearch/model/version.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@

module OpenSearch
module Model
VERSION = "0.1.1"
VERSION = "1.0.0"
end
end
2 changes: 1 addition & 1 deletion opensearch-model/opensearch-model.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Gem::Specification.new do |s|
s.required_ruby_version = '>= 2.4'

s.add_dependency 'activesupport', '> 3'
s.add_dependency "opensearch-ruby", '~> 1.0'
s.add_dependency "opensearch-ruby", '~> 2.0'
s.add_dependency 'hashie'

s.add_development_dependency 'activemodel', '> 3'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
end

Article.delete_all
Article.__opensearch__.create_index!(force: true, include_type_name: true)
Article.__opensearch__.create_index!(force: true)

Article.create!(title: 'Test', body: '', clicks: 1)
Article.create!(title: 'Testing Coding', body: '', clicks: 2)
Expand Down Expand Up @@ -160,17 +160,6 @@
end
end

describe '#id' do

let(:search_result) do
Article.search('title:test')
end

it 'returns the type' do
expect(search_result.results.first.type).to eq('article')
end
end

describe '#each_with_hit' do

let(:search_result) do
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
end

MyNamespace::Book.delete_all
MyNamespace::Book.__opensearch__.create_index!(force: true, include_type_name: true)
MyNamespace::Book.__opensearch__.create_index!(force: true)
MyNamespace::Book.create!(title: 'Test')
MyNamespace::Book.__opensearch__.refresh_index!
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
add_index(:answers, :question_id) unless index_exists?(:answers, :question_id)

clear_tables(Question)
ParentChildSearchable.create_index!(force: true, include_type_name: true)
ParentChildSearchable.create_index!(force: true)

q_1 = Question.create!(title: 'First Question', author: 'John')
q_2 = Question.create!(title: 'Second Question', author: 'Jody')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,7 @@
context 'when a document is indexed' do

let(:search_result) do
ArticleWithCustomSerialization.__opensearch__.client.get(index: 'article_with_custom_serializations',
type: '_doc',
id: '1')
ArticleWithCustomSerialization.__opensearch__.client.get(index: 'article_with_custom_serializations', id: '1')
end

it 'applies the serialization when indexing' do
Expand All @@ -66,7 +64,6 @@

let(:search_result) do
ArticleWithCustomSerialization.__opensearch__.client.get(index: 'article_with_custom_serializations',
type: '_doc',
id: article.id)
end

Expand Down
Loading