diff --git a/CHANGELOG.md b/CHANGELOG.md index a684d2ad2..f19df867a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,6 +95,7 @@ - Any way to manually clear/reset the changelog history? [\#568](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/568) - Present the entire changelog in its own view [\#569](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/569) ([mhenrixon](https://github.com/mhenrixon)) +- Give user full control over adding middleware [\#566](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/566) ([mhenrixon](https://github.com/mhenrixon)) **Fixed bugs:** @@ -165,6 +166,10 @@ - Documentation incorrect for `delete\_by\_digest` [\#547](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/547) - Locked jobs after kill -9 with while\_executing lock [\#546](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/546) +**Merged pull requests:** + +- Support apartment [\#540](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/540) ([mhenrixon](https://github.com/mhenrixon)) + ## [v7.0.0.beta26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta26) (2020-10-28) [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0.beta26) @@ -311,6 +316,7 @@ - Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon)) - Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon)) +- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon)) **Closed issues:** @@ -372,6 +378,10 @@ - until\_expired is not setting TTL [\#468](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/468) - Fix bug where expiration wasn't set until unlock [\#481](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/481) ([mhenrixon](https://github.com/mhenrixon)) +**Merged pull requests:** + +- Allow v7 configuration to work [\#479](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/479) ([mhenrixon](https://github.com/mhenrixon)) + ## [v6.0.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.20) (2020-03-22) [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta11...v6.0.20) @@ -552,6 +562,7 @@ - Redis is busy running script and script never terminates [\#441](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/441) - Make the ruby reaper plain ruby [\#443](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/443) ([mhenrixon](https://github.com/mhenrixon)) +- Fix sidekiq develop [\#426](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/426) ([mhenrixon](https://github.com/mhenrixon)) **Closed issues:** @@ -590,9 +601,9 @@ **Fixed bugs:** - Allow Sidekiq::Context to be used for logging [\#429](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/429) ([mhenrixon](https://github.com/mhenrixon)) -- Fix sidekiq develop [\#426](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/426) ([mhenrixon](https://github.com/mhenrixon)) - Reap as many orphans as advertised [\#403](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/403) ([mhenrixon](https://github.com/mhenrixon)) - Reaper should remove :INFO keys [\#399](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/399) ([mhenrixon](https://github.com/mhenrixon)) +- Convert v5 locks when needed [\#375](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/375) ([mhenrixon](https://github.com/mhenrixon)) **Merged pull requests:** @@ -1409,7 +1420,6 @@ - Do not unlock on sidekiq shutdown [\#87](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/87) ([deltaroe](https://github.com/deltaroe)) - Remove no-op code, protect global space from test code [\#86](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/86) ([stevenjonescgm](https://github.com/stevenjonescgm)) - Remove unique lock when executing and clearing jobs in sidekiq fake mode [\#83](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/83) ([crberube](https://github.com/crberube)) -- Fix tests. Tests with latest sidekiq versions and ruby versions [\#82](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/82) ([simonoff](https://github.com/simonoff)) - Duplicate Payload logging configuration [\#81](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/81) ([jprincipe](https://github.com/jprincipe)) - output log if not unique [\#79](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/79) ([sonots](https://github.com/sonots)) - Checking Sidekiq::Testing.inline? on testing strategy and connector [\#75](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/75) ([Draiken](https://github.com/Draiken)) @@ -1425,6 +1435,7 @@ **Merged pull requests:** +- Fix tests. Tests with latest sidekiq versions and ruby versions [\#82](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/82) ([simonoff](https://github.com/simonoff)) - Use ConnectionPool blocks to ensure exclusive connection. Closes \#66. [\#67](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/67) ([adstage-david](https://github.com/adstage-david)) ## [v3.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v3.0.10) (2014-11-19) @@ -1506,6 +1517,7 @@ - Clarify README about unique expiration [\#36](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/36) ([spacemunkay](https://github.com/spacemunkay)) - Add option to make jobs unique on all queues [\#32](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/32) ([robinmessage](https://github.com/robinmessage)) - Fix homepage in gemspec [\#29](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/29) ([tmaier](https://github.com/tmaier)) +- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau)) ## [v2.7.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v2.7.0) (2013-11-24) @@ -1524,7 +1536,6 @@ - Make unlock/yield order configurable. [\#21](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/21) ([endofunky](https://github.com/endofunky)) - Rely on Sidekiq's String\#constantize extension instead of rolling our own [\#19](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/19) ([disbelief](https://github.com/disbelief)) - Attempt to constantize String `worker\_class` arguments passed to client middleware [\#17](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/17) ([disbelief](https://github.com/disbelief)) -- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau)) - Allow worker to specify which arguments to include in uniquing hash [\#12](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/12) ([sax](https://github.com/sax)) - Add support for unique when using Sidekiq's delay function [\#11](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/11) ([eduardosasso](https://github.com/eduardosasso)) - Adding the unique prefix option [\#8](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/8) ([KensoDev](https://github.com/KensoDev)) diff --git a/Gemfile b/Gemfile index 293a2af19..f51b81d96 100644 --- a/Gemfile +++ b/Gemfile @@ -12,6 +12,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_5.0.gemfile b/gemfiles/sidekiq_5.0.gemfile index 2dc5589fc..a0657755d 100644 --- a/gemfiles/sidekiq_5.0.gemfile +++ b/gemfiles/sidekiq_5.0.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_5.1.gemfile b/gemfiles/sidekiq_5.1.gemfile index d6b811cb0..8c81d2437 100644 --- a/gemfiles/sidekiq_5.1.gemfile +++ b/gemfiles/sidekiq_5.1.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_5.2.gemfile b/gemfiles/sidekiq_5.2.gemfile index 5d478d1e1..58cced16a 100644 --- a/gemfiles/sidekiq_5.2.gemfile +++ b/gemfiles/sidekiq_5.2.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_6.0.gemfile b/gemfiles/sidekiq_6.0.gemfile index c8a4ccd1a..3a891cd7a 100644 --- a/gemfiles/sidekiq_6.0.gemfile +++ b/gemfiles/sidekiq_6.0.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_6.1.gemfile b/gemfiles/sidekiq_6.1.gemfile index fec09b09c..07518fb86 100644 --- a/gemfiles/sidekiq_6.1.gemfile +++ b/gemfiles/sidekiq_6.1.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/gemfiles/sidekiq_6.2.gemfile b/gemfiles/sidekiq_6.2.gemfile new file mode 100644 index 000000000..1ac2498bb --- /dev/null +++ b/gemfiles/sidekiq_6.2.gemfile @@ -0,0 +1,37 @@ +# This file was generated by Appraisal + +source "https://rubygems.org" + +gem "appraisal" +gem "bundler" +gem "gem-release" +gem "github-markup" +gem "rack-test" +gem "rake" +gem "rspec" +gem "rspec-its" +gem "sinatra" +gem "timecop" +gem "yard" +gem "sidekiq", "~> 6.2.0" + +platforms :mri do + gem "concurrent-ruby-ext" + gem "fasterer" + gem "github_changelog_generator" + gem "guard" + gem "guard-bundler" + gem "guard-reek" + gem "guard-rspec" + gem "guard-rubocop" + gem "hiredis" + gem "redcarpet", "~> 3.4" + gem "reek", ">= 5.3" + gem "rspec-benchmark" + gem "rubocop-mhenrixon" + gem "ruby-prof", require: false + gem "simplecov-sublime", "0.21.0", require: false + gem "travis" +end + +gemspec path: "../" diff --git a/gemfiles/sidekiq_develop.gemfile b/gemfiles/sidekiq_develop.gemfile index 4f8fddd16..6075015a4 100644 --- a/gemfiles/sidekiq_develop.gemfile +++ b/gemfiles/sidekiq_develop.gemfile @@ -9,6 +9,7 @@ gem "github-markup" gem "rack-test" gem "rake" gem "rspec" +gem "rspec-html-matchers" gem "rspec-its" gem "sinatra" gem "timecop" diff --git a/lib/sidekiq_unique_jobs/changelog.rb b/lib/sidekiq_unique_jobs/changelog.rb index 18efc9873..4d938b049 100644 --- a/lib/sidekiq_unique_jobs/changelog.rb +++ b/lib/sidekiq_unique_jobs/changelog.rb @@ -68,8 +68,8 @@ def page(cursor: 0, pattern: "*", page_size: 100) end [ - total_size, - result[0], # next_cursor + total_size.to_i, + result[0].to_i, # next_cursor result[1].map { |entry| load_json(entry[0]) }, # entries ] end diff --git a/lib/sidekiq_unique_jobs/digests.rb b/lib/sidekiq_unique_jobs/digests.rb index 0e8a3e813..8186c4a3d 100644 --- a/lib/sidekiq_unique_jobs/digests.rb +++ b/lib/sidekiq_unique_jobs/digests.rb @@ -104,8 +104,8 @@ def page(cursor: 0, pattern: SCAN_PATTERN, page_size: 100) end [ - total_size, - digests[0], # next_cursor + total_size.to_i, + digests[0].to_i, # next_cursor digests[1].map { |digest, score| Lock.new(digest, time: score) }, # entries ] end diff --git a/lib/sidekiq_unique_jobs/web.rb b/lib/sidekiq_unique_jobs/web.rb index a348f8a58..fe9d79642 100644 --- a/lib/sidekiq_unique_jobs/web.rb +++ b/lib/sidekiq_unique_jobs/web.rb @@ -19,8 +19,11 @@ def self.registered(app) # rubocop:disable Metrics/MethodLength, Metrics/AbcSize @count = (params[:count] || 100).to_i @current_cursor = params[:cursor] @prev_cursor = params[:prev_cursor] - @pagination = { pattern: @filter, cursor: @current_cursor, page_size: @count } - @total_size, @next_cursor, @changelogs = changelog.page(**@pagination) + @total_size, @next_cursor, @changelogs = changelog.page( + cursor: @current_cursor, + pattern: @filter, + page_size: @count, + ) erb(unique_template(:changelogs)) end @@ -36,8 +39,12 @@ def self.registered(app) # rubocop:disable Metrics/MethodLength, Metrics/AbcSize @count = (params[:count] || 100).to_i @current_cursor = params[:cursor] @prev_cursor = params[:prev_cursor] - @pagination = { pattern: @filter, cursor: @current_cursor, page_size: @count } - @total_size, @next_cursor, @locks = digests.page(**@pagination) + + @total_size, @next_cursor, @locks = digests.page( + cursor: @current_cursor, + pattern: @filter, + page_size: @count, + ) erb(unique_template(:locks)) end diff --git a/lib/sidekiq_unique_jobs/web/helpers.rb b/lib/sidekiq_unique_jobs/web/helpers.rb index 7f8c81b68..1adc7f35e 100644 --- a/lib/sidekiq_unique_jobs/web/helpers.rb +++ b/lib/sidekiq_unique_jobs/web/helpers.rb @@ -13,7 +13,9 @@ module Helpers VIEW_PATH = File.expand_path("../web/views", __dir__).freeze # # @return [Array] safe params - SAFE_CPARAMS = %w[cursor prev_cursor].freeze + SAFE_CPARAMS = %w[ + filter count cursor prev_cursor poll direction + ].freeze extend self @@ -67,10 +69,8 @@ def changelog # @return [String] a url safe parameter string # def cparams(options) - # stringify - options.transform_keys(&:to_s) - - params.merge(options).map do |key, value| + stringified_options = options.transform_keys(&:to_s) + params.merge(stringified_options).map do |key, value| next unless SAFE_CPARAMS.include?(key) "#{key}=#{CGI.escape(value.to_s)}" diff --git a/lib/sidekiq_unique_jobs/web/views/_paging.erb b/lib/sidekiq_unique_jobs/web/views/_paging.erb index 72e3b660d..5efd4e2e5 100644 --- a/lib/sidekiq_unique_jobs/web/views/_paging.erb +++ b/lib/sidekiq_unique_jobs/web/views/_paging.erb @@ -1,10 +1,10 @@ diff --git a/lib/sidekiq_unique_jobs/web/views/changelogs.erb b/lib/sidekiq_unique_jobs/web/views/changelogs.erb index 28677e231..ddb0b373f 100644 --- a/lib/sidekiq_unique_jobs/web/views/changelogs.erb +++ b/lib/sidekiq_unique_jobs/web/views/changelogs.erb @@ -36,7 +36,7 @@ <% @changelogs.each do |changelog| %> - + <%= safe_relative_time(changelog["time"]) %> <%= changelog["digest"] %> <%= changelog["script"] %> diff --git a/lib/sidekiq_unique_jobs/web/views/locks.erb b/lib/sidekiq_unique_jobs/web/views/locks.erb index d517d1e05..e8bee6abf 100644 --- a/lib/sidekiq_unique_jobs/web/views/locks.erb +++ b/lib/sidekiq_unique_jobs/web/views/locks.erb @@ -11,7 +11,7 @@ <%= t('Filter') %> - <% if @locks.any? && @total_size > @count.to_i %> + <% if @locks.any? && @total_size > @count %>
<%= erb unique_template(:_paging), locals: { url: "#{root_path}locks" } %>
@@ -30,20 +30,22 @@ <% @locks.each do |lock| %> - - -
- <%= csrf_tag %> - - -
- - <%= lock.key %> - <%= lock.info["lock"] %> - <%= lock.locked.count %> - <%= safe_relative_time(lock.created_at) %> - - <% end %> + + + +
+ <%= csrf_tag %> + + +
+ + <%= lock.key %> + <%= lock.info["lock"] %> + <%= lock.locked.count %> + <%= safe_relative_time(lock.created_at) %> + + + <% end %>
diff --git a/spec/sidekiq_unique_jobs/changelog_spec.rb b/spec/sidekiq_unique_jobs/changelog_spec.rb index 9a682c2d8..56ed3c60b 100644 --- a/spec/sidekiq_unique_jobs/changelog_spec.rb +++ b/spec/sidekiq_unique_jobs/changelog_spec.rb @@ -193,7 +193,7 @@ let(:page_size) { 1 } context "when no entries exist" do - it { is_expected.to match_array([0, "0", []]) } + it { is_expected.to match_array([0, 0, []]) } end context "when entries exist" do diff --git a/spec/sidekiq_unique_jobs/web_spec.rb b/spec/sidekiq_unique_jobs/web_spec.rb index 892543c71..553ce0a02 100644 --- a/spec/sidekiq_unique_jobs/web_spec.rb +++ b/spec/sidekiq_unique_jobs/web_spec.rb @@ -2,9 +2,11 @@ require "sidekiq_unique_jobs/web" require "rack/test" +require "rspec-html-matchers" RSpec.describe SidekiqUniqueJobs::Web do include Rack::Test::Methods + include RSpecHtmlMatchers def app @app ||= Rack::Builder.new do @@ -31,6 +33,8 @@ def app let(:digest_one) { "uniquejobs:9e9b5ce5d423d3ea470977004b50ff84" } let(:digest_two) { "uniquejobs:24c5b03e2d49d765e5dfb2d7c51c5929" } let(:lock_type) { :until_executed } + let(:changelog) { SidekiqUniqueJobs::Changelog.new } + let(:digests) { SidekiqUniqueJobs::Digests.new } let(:lock_info) do { type: lock_type } @@ -43,13 +47,35 @@ def app ] end + it "can paginate changelogs", sidekiq_ver: ">= 6.0" do + Array.new(190) do |idx| + expect(MyUniqueJob.perform_async(1, idx)).not_to eq(nil) + end + + get "/changelogs?filter=*&count=100" + _size, next_cursor, changelogs = changelog.page(cursor: 0, page_size: 100, pattern: "*") + + expect(last_response).to be_ok + expect(last_response.body).to have_tag("div", with: { class: "table_container" }) do + with_tag("tr.changelog-row", count: changelogs.size) + end + + get "/changelogs?filter=*&cursor=#{next_cursor}&prev_cursor=0&count=100" + + expect(last_response).to be_ok + expect(last_response.body).to have_tag("div", with: { class: "table_container" }) do + _size, _next_cursor, changelogs = changelog.page(cursor: next_cursor, page_size: 100, pattern: "*") + with_tag("tr.changelog-row", count: changelogs.size) + end + end + it "can display changelog" do lock_one.lock(jid_one, lock_info) lock_two.lock(jid_two, lock_info) get "/changelogs" - expect(last_response.status).to eq(200) + expect(last_response).to be_ok end it "can display digests" do @@ -58,19 +84,32 @@ def app get "/locks" - expect(last_response.status).to eq(200) + expect(last_response).to be_ok expect(last_response.body).to match("/locks/#{digest_one}") expect(last_response.body).to match("/locks/#{digest_two}") end - it "can paginate digests" do - Array.new(110) do |idx| + it "can paginate digests", sidekiq_ver: ">= 6.0" do + Array.new(190) do |idx| expect(MyUniqueJob.perform_async(1, idx)).not_to eq(nil) end - get "/locks" + get "/locks?filter=*&count=100" + _size, next_cursor, locks = digests.page(cursor: 0, page_size: 100, pattern: "*") - expect(last_response.status).to eq(200) + expect(last_response).to be_ok + expect(last_response.body).to have_tag("div", with: { class: "table_container" }) do + with_tag("tr.lock-row", count: locks.size) + end + + _size, next_cursor, _locks = digests.page(cursor: 0, page_size: 100, pattern: "*") + get "/locks?filter=*&cursor=#{next_cursor}&prev_cursor=0&count=100" + + expect(last_response).to be_ok + expect(last_response.body).to have_tag("div", with: { class: "table_container" }) do + _size, _next_cursor, locks = digests.page(cursor: next_cursor, page_size: 100, pattern: "*") + with_tag("tr.lock-row", count: locks.size) + end end it "can display digest" do @@ -79,7 +118,7 @@ def app get "/locks/#{digest_one}" - expect(last_response.status).to eq(200) + expect(last_response).to be_ok expect(last_response.body).to match("uniquejobs:9e9b5ce5d423d3ea470977004b50ff84") end