Hello community, here is the log from the commit of package rubygem-delayed_job for openSUSE:Factory checked in at 2015-09-27 08:40:16 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/rubygem-delayed_job (Old) and /work/SRC/openSUSE:Factory/.rubygem-delayed_job.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "rubygem-delayed_job" Changes: -------- --- /work/SRC/openSUSE:Factory/rubygem-delayed_job/rubygem-delayed_job.changes 2015-02-11 16:45:19.000000000 +0100 +++ /work/SRC/openSUSE:Factory/.rubygem-delayed_job.new/rubygem-delayed_job.changes 2015-09-27 08:39:42.000000000 +0200 @@ -1,0 +2,26 @@ +Fri Sep 25 04:28:48 UTC 2015 - coolo@suse.com + +- updated to version 4.1.1 + see installed CHANGELOG.md + + 4.1.1 - 2015-09-24 + ================== + * Fix shared specs for back-ends that reload objects + +------------------------------------------------------------------- +Wed Sep 23 04:29:11 UTC 2015 - coolo@suse.com + +- updated to version 4.1.0 + see installed CHANGELOG.md + + 4.1.0 - 2015-09-22 + ================== + * Alter `Delayed::Command` to work with or without Rails + * Allow `Delayed::Worker.delay_jobs` configuration to be a proc + * Add ability to set destroy failed jobs on a per job basis + * Make `Delayed::Worker.new` idempotent + * Set quiet from the environment + * Rescue `Exception` instead of `StandardError` in worker + * Fix worker crash on serialization error + +------------------------------------------------------------------- Old: ---- delayed_job-4.0.6.gem New: ---- delayed_job-4.1.1.gem ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ rubygem-delayed_job.spec ++++++ --- /var/tmp/diff_new_pack.uv6hMd/_old 2015-09-27 08:39:42.000000000 +0200 +++ /var/tmp/diff_new_pack.uv6hMd/_new 2015-09-27 08:39:42.000000000 +0200 @@ -24,7 +24,7 @@ # Name: rubygem-delayed_job -Version: 4.0.6 +Version: 4.1.1 Release: 0 %define mod_name delayed_job %define mod_full_name %{mod_name}-%{version} ++++++ delayed_job-4.0.6.gem -> delayed_job-4.1.1.gem ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/CHANGELOG.md new/CHANGELOG.md --- old/CHANGELOG.md 2014-12-22 14:22:31.000000000 +0100 +++ new/CHANGELOG.md 2015-09-24 23:40:11.000000000 +0200 @@ -1,3 +1,17 @@ +4.1.1 - 2015-09-24 +================== +* Fix shared specs for back-ends that reload objects + +4.1.0 - 2015-09-22 +================== +* Alter `Delayed::Command` to work with or without Rails +* Allow `Delayed::Worker.delay_jobs` configuration to be a proc +* Add ability to set destroy failed jobs on a per job basis +* Make `Delayed::Worker.new` idempotent +* Set quiet from the environment +* Rescue `Exception` instead of `StandardError` in worker +* Fix worker crash on serialization error + 4.0.6 - 2014-12-22 ================== * Revert removing test files from the gem diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/README.md new/README.md --- old/README.md 2014-12-22 14:22:31.000000000 +0100 +++ new/README.md 2015-09-24 23:40:11.000000000 +0200 @@ -1,3 +1,8 @@ +**If you're viewing this at https://github.com/collectiveidea/delayed_job, +you're reading the documentation for the master branch. +[View documentation for the latest release +(4.0.6).](https://github.com/collectiveidea/delayed_job/tree/v4.0.6)** + Delayed::Job ============ [][gem] @@ -58,14 +63,30 @@ rails generate delayed_job:active_record rake db:migrate +For Rails 4.2, see [below](#rails-42) + Development =========== In development mode, if you are using Rails 3.1+, your application code will automatically reload every 100 jobs or when the queue finishes. You no longer need to restart Delayed Job every time you update your code in development. -Rails 4 -======= -If you are using the protected_attributes gem, it must appear before delayed_job in your gemfile. +Rails 4.2 +========= +Set the queue_adapter in config/application.rb + +```ruby +config.active_job.queue_adapter = :delayed_job +``` + +See the [rails guide](http://guides.rubyonrails.org/active_job_basics.html#setting-the-backend) for more details. + +Rails 4.x +========= +If you are using the protected_attributes gem, it must appear before delayed_job in your gemfile. If your jobs are failing with: + + ActiveRecord::StatementInvalid: PG::NotNullViolation: ERROR: null value in column "handler" violates not-null constraint + +then this is the fix you're looking for. Upgrading from 2.x to 3.0.0 on Active Record ============================================ @@ -103,9 +124,17 @@ device.deliver ``` -handle_asynchronously can take as options anything you can pass to delay. In -addition, the values can be Proc objects allowing call time evaluation of the -value. For some examples: +## Parameters + +`#handle_asynchronously` and `#delay` take these parameters: + +- `:priority` (number): lower numbers run first; default is 0 but can be reconfigured (see below) +- `:run_at` (Time): run the job after this time (probably in the future) +- `:queue` (string): named queue to put this job in, an alternative to priorities (see below) + +These params can be Proc objects, allowing call-time evaluation of the value. + +For example: ```ruby class LongTasks @@ -282,6 +311,20 @@ end ``` +To set a per-job default for destroying failed jobs that overrides the Delayed::Worker.destroy_failed_jobs you can define a destroy_failed_jobs? method on the job + +```ruby +NewsletterJob = Struct.new(:text, :emails) do + def perform + emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) } + end + + def destroy_failed_jobs? + false + end +end +``` + To set a default queue name for a custom job that overrides Delayed::Worker.default_queue_name, you can define a queue_name method on the job ```ruby @@ -384,6 +427,14 @@ It is possible to disable delayed jobs for testing purposes. Set `Delayed::Worker.delay_jobs = false` to execute all jobs realtime. +Or `Delayed::Worker.delay_jobs` can be a Proc that decides whether to execute jobs inline on a per-job basis: + +```ruby +Delayed::Worker.delay_jobs = ->(job) { + job.queue != 'inline' +} +``` + You may need to raise exceptions on SIGTERM signals, `Delayed::Worker.raise_signal_exceptions = :term` will cause the worker to raise a `SignalException` causing the running job to abort and be unlocked, which makes the job available to other workers. The default for this option is false. Here is an example of changing job parameters in Rails: Files old/checksums.yaml.gz and new/checksums.yaml.gz differ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/delayed_job.gemspec new/delayed_job.gemspec --- old/delayed_job.gemspec 2014-12-22 14:22:31.000000000 +0100 +++ new/delayed_job.gemspec 2015-09-24 23:40:11.000000000 +0200 @@ -4,12 +4,12 @@ spec.description = 'Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.' spec.email = ['brian@collectiveidea.com'] spec.files = %w[CHANGELOG.md CONTRIBUTING.md LICENSE.md README.md Rakefile delayed_job.gemspec] - spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*') + spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*') spec.homepage = 'http://github.com/collectiveidea/delayed_job' spec.licenses = ['MIT'] spec.name = 'delayed_job' spec.require_paths = ['lib'] spec.summary = 'Database-backed asynchronous priority queue system -- Extracted from Shopify' spec.test_files = Dir.glob('spec/**/*') - spec.version = '4.0.6' + spec.version = '4.1.1' end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/backend/base.rb new/lib/delayed/backend/base.rb --- old/lib/delayed/backend/base.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/backend/base.rb 2015-09-24 23:40:11.000000000 +0200 @@ -10,7 +10,7 @@ def enqueue(*args) # rubocop:disable CyclomaticComplexity options = args.extract_options! options[:payload_object] ||= args.shift - options[:priority] ||= Delayed::Worker.default_priority + options[:priority] ||= Delayed::Worker.default_priority if options[:queue].nil? if options[:payload_object].respond_to?(:queue_name) @@ -32,7 +32,7 @@ new(options).tap do |job| Delayed::Worker.lifecycle.run_callbacks(:enqueue, job) do job.hook(:enqueue) - Delayed::Worker.delay_jobs ? job.save : job.invoke_job + Delayed::Worker.delay_job?(job) ? job.save : job.invoke_job end end end @@ -63,6 +63,12 @@ end end + attr_reader :error + def error=(error) + @error = error + self.last_error = "#{error.message}\n#{error.backtrace.join("\n")}" if self.respond_to?(:last_error=) + end + def failed? !!failed_at end @@ -93,7 +99,7 @@ hook :before payload_object.perform hook :success - rescue => e + rescue Exception => e # rubocop:disable RescueException hook :error, e raise e ensure @@ -139,6 +145,12 @@ end end + def destroy_failed_jobs? + payload_object.respond_to?(:destroy_failed_jobs?) ? payload_object.destroy_failed_jobs? : Delayed::Worker.destroy_failed_jobs + rescue DeserializationError + Delayed::Worker.destroy_failed_jobs + end + def fail! update_attributes(:failed_at => self.class.db_time_now) end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/backend/shared_spec.rb new/lib/delayed/backend/shared_spec.rb --- old/lib/delayed/backend/shared_spec.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/backend/shared_spec.rb 2015-09-24 23:40:11.000000000 +0200 @@ -161,7 +161,7 @@ job = described_class.enqueue(CallbackJob.new) expect(job.payload_object).to receive(:perform).and_raise(RuntimeError.new('fail')) - expect { job.invoke_job }.to raise_error + expect { job.invoke_job }.to raise_error(RuntimeError) expect(CallbackJob.messages).to eq(['enqueue', 'before', 'error: RuntimeError', 'after']) end @@ -450,6 +450,34 @@ end end + describe 'destroy_failed_jobs' do + context 'with a SimpleJob' do + before(:each) do + @job = described_class.enqueue SimpleJob.new + end + + it 'is not defined' do + expect(@job.destroy_failed_jobs?).to be true + end + + it 'uses the destroy failed jobs value on the payload when defined' do + expect(@job.payload_object).to receive(:destroy_failed_jobs?).and_return(false) + expect(@job.destroy_failed_jobs?).to be false + end + end + + context 'with a job that raises DserializationError' do + before(:each) do + @job = described_class.new :handler => '--- !ruby/struct:GoingToRaiseArgError {}' + end + + it 'falls back reasonably' do + expect(YAML).to receive(:load_dj).and_raise(ArgumentError) + expect(@job.destroy_failed_jobs?).to be true + end + end + end + describe 'yaml serialization' do context 'when serializing jobs' do it 'raises error ArgumentError for new records' do @@ -500,6 +528,7 @@ Delayed::Worker.max_run_time = 1.second job = Delayed::Job.create :payload_object => LongRunningJob.new worker.run(job) + expect(job.error).to_not be_nil expect(job.reload.last_error).to match(/expired/) expect(job.reload.last_error).to match(/Delayed::Worker\.max_run_time is only 1 second/) expect(job.attempts).to eq(1) @@ -513,6 +542,7 @@ it 'marks the job as failed' do Delayed::Worker.destroy_failed_jobs = false job = described_class.create! :handler => '--- !ruby/object:JobThatDoesNotExist {}' + expect_any_instance_of(described_class).to receive(:destroy_failed_jobs?).and_return(false) worker.work_off job.reload expect(job).to be_failed @@ -535,6 +565,7 @@ Delayed::Worker.max_attempts = 1 worker.run(@job) @job.reload + expect(@job.error).to_not be_nil expect(@job.last_error).to match(/did not work/) expect(@job.attempts).to eq(1) expect(@job).to be_failed @@ -617,6 +648,10 @@ end context 'and we want to destroy jobs' do + after do + Delayed::Worker.destroy_failed_jobs = true + end + it_behaves_like 'any failure more than Worker.max_attempts times' it 'is destroyed if it failed more than Worker.max_attempts times' do @@ -624,6 +659,13 @@ Delayed::Worker.max_attempts.times { worker.reschedule(@job) } end + it 'is destroyed if the job has destroy failed jobs set' do + Delayed::Worker.destroy_failed_jobs = false + expect(@job).to receive(:destroy_failed_jobs?).and_return(true) + expect(@job).to receive(:destroy) + Delayed::Worker.max_attempts.times { worker.reschedule(@job) } + end + it 'is not destroyed if failed fewer than Worker.max_attempts times' do expect(@job).not_to receive(:destroy) (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) } @@ -641,15 +683,35 @@ it_behaves_like 'any failure more than Worker.max_attempts times' - it 'is failed if it failed more than Worker.max_attempts times' do - expect(@job.reload).not_to be_failed - Delayed::Worker.max_attempts.times { worker.reschedule(@job) } - expect(@job.reload).to be_failed + context 'and destroy failed jobs is false' do + it 'is failed if it failed more than Worker.max_attempts times' do + expect(@job.reload).not_to be_failed + Delayed::Worker.max_attempts.times { worker.reschedule(@job) } + expect(@job.reload).to be_failed + end + + it 'is not failed if it failed fewer than Worker.max_attempts times' do + (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) } + expect(@job.reload).not_to be_failed + end end - it 'is not failed if it failed fewer than Worker.max_attempts times' do - (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) } - expect(@job.reload).not_to be_failed + context 'and destroy failed jobs for job is false' do + before do + Delayed::Worker.destroy_failed_jobs = true + end + + it 'is failed if it failed more than Worker.max_attempts times' do + expect(@job).to receive(:destroy_failed_jobs?).and_return(false) + expect(@job.reload).not_to be_failed + Delayed::Worker.max_attempts.times { worker.reschedule(@job) } + expect(@job.reload).to be_failed + end + + it 'is not failed if it failed fewer than Worker.max_attempts times' do + (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) } + expect(@job.reload).not_to be_failed + end end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/command.rb new/lib/delayed/command.rb --- old/lib/delayed/command.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/command.rb 2015-09-24 23:40:11.000000000 +0200 @@ -6,15 +6,19 @@ end end require 'optparse' +require 'pathname' module Delayed class Command # rubocop:disable ClassLength attr_accessor :worker_count, :worker_pools + DIR_PWD = Pathname.new Dir.pwd + def initialize(args) # rubocop:disable MethodLength @options = { :quiet => true, - :pid_dir => "#{Rails.root}/tmp/pids" + :pid_dir => "#{root}/tmp/pids", + :log_dir => "#{root}/log" } @worker_count = 1 @@ -37,11 +41,14 @@ @options[:max_priority] = n end opt.on('-n', '--number_of_workers=workers', 'Number of unique workers to spawn') do |worker_count| - @worker_count = worker_count.to_i rescue 1 # rubocop:disable RescueModifier + @worker_count = worker_count.to_i rescue 1 end opt.on('--pid-dir=DIR', 'Specifies an alternate directory in which to store the process ids.') do |dir| @options[:pid_dir] = dir end + opt.on('--log-dir=DIR', 'Specifies an alternate directory in which to store the delayed_job log.') do |dir| + @options[:log_dir] = dir + end opt.on('-i', '--identifier=n', 'A numeric identifier for the worker.') do |n| @options[:identifier] = n end @@ -114,18 +121,19 @@ end def run(worker_name = nil, options = {}) - Dir.chdir(Rails.root) + Dir.chdir(root) Delayed::Worker.after_fork - Delayed::Worker.logger ||= Logger.new(File.join(Rails.root, 'log', 'delayed_job.log')) + Delayed::Worker.logger ||= Logger.new(File.join(@options[:log_dir], 'delayed_job.log')) worker = Delayed::Worker.new(options) worker.name_prefix = "#{worker_name} " worker.start rescue => e - Rails.logger.fatal e STDERR.puts e.message - exit 1 + STDERR.puts e.backtrace + ::Rails.logger.fatal(e) if rails_logger_defined? + exit_with_error_status end private @@ -142,5 +150,21 @@ worker_count = (worker_count || 1).to_i rescue 1 @worker_pools << [queues, worker_count] end + + def root + @root ||= rails_root_defined? ? ::Rails.root : DIR_PWD + end + + def rails_root_defined? + defined?(::Rails.root) + end + + def rails_logger_defined? + defined?(::Rails.logger) + end + + def exit_with_error_status + exit 1 + end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/message_sending.rb new/lib/delayed/message_sending.rb --- old/lib/delayed/message_sending.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/message_sending.rb 2015-09-24 23:40:11.000000000 +0200 @@ -31,8 +31,10 @@ module ClassMethods def handle_asynchronously(method, opts = {}) - aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1 # rubocop:disable PerlBackrefs - with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}" + aliased_method = method.to_s.sub(/([?!=])$/, '') + punctuation = $1 # rubocop:disable PerlBackrefs + with_method = "#{aliased_method}_with_delay#{punctuation}" + without_method = "#{aliased_method}_without_delay#{punctuation}" define_method(with_method) do |*args| curr_opts = opts.clone curr_opts.each_key do |key| diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/recipes.rb new/lib/delayed/recipes.rb --- old/lib/delayed/recipes.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/recipes.rb 2015-09-24 23:40:11.000000000 +0200 @@ -38,17 +38,17 @@ desc 'Stop the delayed_job process' task :stop, :roles => lambda { roles } do - run "cd #{current_path};#{rails_env} #{delayed_job_command} stop #{args}" + run "cd #{current_path} && #{rails_env} #{delayed_job_command} stop #{args}" end desc 'Start the delayed_job process' task :start, :roles => lambda { roles } do - run "cd #{current_path};#{rails_env} #{delayed_job_command} start #{args}" + run "cd #{current_path} && #{rails_env} #{delayed_job_command} start #{args}" end desc 'Restart the delayed_job process' task :restart, :roles => lambda { roles } do - run "cd #{current_path};#{rails_env} #{delayed_job_command} restart #{args}" + run "cd #{current_path} && #{rails_env} #{delayed_job_command} restart #{args}" end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/syck_ext.rb new/lib/delayed/syck_ext.rb --- old/lib/delayed/syck_ext.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/syck_ext.rb 2015-09-24 23:40:11.000000000 +0200 @@ -29,7 +29,7 @@ # Constantize the object so that ActiveSupport can attempt # its auto loading magic. Will raise LoadError if not successful. name.constantize - "Struct::#{ name }" + "Struct::#{name}" end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/tasks.rb new/lib/delayed/tasks.rb --- old/lib/delayed/tasks.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/tasks.rb 2015-09-24 23:40:11.000000000 +0200 @@ -19,7 +19,7 @@ :min_priority => ENV['MIN_PRIORITY'], :max_priority => ENV['MAX_PRIORITY'], :queues => (ENV['QUEUES'] || ENV['QUEUE'] || '').split(','), - :quiet => false + :quiet => ENV['QUIET'] } @worker_options[:sleep_delay] = ENV['SLEEP_DELAY'].to_i if ENV['SLEEP_DELAY'] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/lib/delayed/worker.rb new/lib/delayed/worker.rb --- old/lib/delayed/worker.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/lib/delayed/worker.rb 2015-09-24 23:40:11.000000000 +0200 @@ -39,6 +39,7 @@ self.delay_jobs = DEFAULT_DELAY_JOBS self.queues = DEFAULT_QUEUES self.read_ahead = DEFAULT_READ_AHEAD + @lifecycle = nil end reset @@ -97,13 +98,29 @@ end def self.lifecycle - @lifecycle ||= Delayed::Lifecycle.new + # In case a worker has not been set up, job enqueueing needs a lifecycle. + setup_lifecycle unless @lifecycle + + @lifecycle + end + + def self.setup_lifecycle + @lifecycle = Delayed::Lifecycle.new + plugins.each { |klass| klass.new } end def self.reload_app? defined?(ActionDispatch::Reloader) && Rails.application.config.cache_classes == false end + def self.delay_job?(job) + if delay_jobs.is_a?(Proc) + delay_jobs.arity == 1 ? delay_jobs.call(job) : delay_jobs.call + else + delay_jobs + end + end + def initialize(options = {}) @quiet = options.key?(:quiet) ? options[:quiet] : true @failed_reserve_count = 0 @@ -112,7 +129,9 @@ self.class.send("#{option}=", options[option]) if options.key?(option) end - plugins.each { |klass| klass.new } + # Reset lifecycle on the offhand chance that something lazily + # triggered its creation before all plugins had been registered. + self.class.setup_lifecycle end # Every worker has a unique name which by default is the pid of the process. There are some @@ -121,7 +140,7 @@ # it crashed before. def name return @name unless @name.nil? - "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue "#{@name_prefix}pid:#{Process.pid}" # rubocop:disable RescueModifier + "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue "#{@name_prefix}pid:#{Process.pid}" end # Sets the name of the worker. @@ -181,7 +200,8 @@ # Do num jobs and return stats on success/failure. # Exit early if interrupted. def work_off(num = 100) - success, failure = 0, 0 + success = 0 + failure = 0 num.times do case reserve_and_run_one_job @@ -190,7 +210,7 @@ when false failure += 1 else - break # leave if no work could be done + break # leave if no work could be done end break if stop? # leave if we're exiting end @@ -200,18 +220,18 @@ def run(job) job_say job, 'RUNNING' - runtime = Benchmark.realtime do + runtime = Benchmark.realtime do Timeout.timeout(max_run_time(job).to_i, WorkerTimeout) { job.invoke_job } job.destroy end job_say job, format('COMPLETED after %.4f', runtime) - return true # did work + return true # did work rescue DeserializationError => error - job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}" + job.error = error failed(job) - rescue => error + rescue Exception => error # rubocop:disable RescueException self.class.lifecycle.run_callbacks(:error, self, job) { handle_failed_job(job, error) } - return false # work failed + return false # work failed end # Reschedule the job in the future (when a job fails). @@ -236,7 +256,7 @@ say "Error when running failure callback: #{error}", 'error' say error.backtrace.join("\n"), 'error' ensure - self.class.destroy_failed_jobs ? job.destroy : job.fail! + job.destroy_failed_jobs? ? job.destroy : job.fail! end end end @@ -268,7 +288,7 @@ protected def handle_failed_job(job, error) - job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}" + job.error = error job_say job, "FAILED (#{job.attempts} prior attempts) with #{error.class.name}: #{error.message}", 'error' reschedule(job) end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/metadata new/metadata --- old/metadata 2014-12-22 14:22:31.000000000 +0100 +++ new/metadata 2015-09-24 23:40:11.000000000 +0200 @@ -1,7 +1,7 @@ --- !ruby/object:Gem::Specification name: delayed_job version: !ruby/object:Gem::Version - version: 4.0.6 + version: 4.1.1 platform: ruby authors: - Brandon Keepers @@ -15,7 +15,7 @@ autorequire: bindir: bin cert_chain: [] -date: 2014-12-22 00:00:00.000000000 Z +date: 2015-09-24 00:00:00.000000000 Z dependencies: - !ruby/object:Gem::Dependency name: activesupport @@ -84,6 +84,7 @@ - spec/autoloaded/instance_clazz.rb - spec/autoloaded/instance_struct.rb - spec/autoloaded/struct.rb +- spec/daemons.rb - spec/delayed/backend/test.rb - spec/delayed/command_spec.rb - spec/delayed/serialization/test.rb @@ -117,7 +118,7 @@ version: '0' requirements: [] rubyforge_project: -rubygems_version: 2.4.4 +rubygems_version: 2.4.8 signing_key: specification_version: 4 summary: Database-backed asynchronous priority queue system -- Extracted from Shopify @@ -126,6 +127,7 @@ - spec/autoloaded/instance_clazz.rb - spec/autoloaded/instance_struct.rb - spec/autoloaded/struct.rb +- spec/daemons.rb - spec/delayed/backend/test.rb - spec/delayed/command_spec.rb - spec/delayed/serialization/test.rb diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/autoloaded/instance_struct.rb new/spec/autoloaded/instance_struct.rb --- old/spec/autoloaded/instance_struct.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/autoloaded/instance_struct.rb 2015-09-24 23:40:11.000000000 +0200 @@ -1,5 +1,6 @@ module Autoloaded - class InstanceStruct < ::Struct.new(nil) + InstanceStruct = ::Struct.new(nil) + class InstanceStruct def perform end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/autoloaded/struct.rb new/spec/autoloaded/struct.rb --- old/spec/autoloaded/struct.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/autoloaded/struct.rb 2015-09-24 23:40:11.000000000 +0200 @@ -1,6 +1,7 @@ # Make sure this file does not get required manually module Autoloaded - class Struct < ::Struct.new(nil) + Struct = ::Struct.new(nil) + class Struct def perform end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/daemons.rb new/spec/daemons.rb --- old/spec/daemons.rb 1970-01-01 01:00:00.000000000 +0100 +++ new/spec/daemons.rb 2015-09-24 23:40:11.000000000 +0200 @@ -0,0 +1,2 @@ +# Fake "daemons" file on the spec load path to allow spec/delayed/command_spec.rb +# to test the Delayed::Command class without actually adding daemons as a dependency. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/delayed/command_spec.rb new/spec/delayed/command_spec.rb --- old/spec/delayed/command_spec.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/delayed/command_spec.rb 2015-09-24 23:40:11.000000000 +0200 @@ -2,6 +2,129 @@ require 'delayed/command' describe Delayed::Command do + let(:options) { [] } + let(:logger) { double('Logger') } + + subject { Delayed::Command.new options } + + before do + allow(Delayed::Worker).to receive(:after_fork) + allow(Dir).to receive(:chdir) + allow(Logger).to receive(:new).and_return(logger) + allow_any_instance_of(Delayed::Worker).to receive(:start) + allow(Delayed::Worker).to receive(:logger=) + allow(Delayed::Worker).to receive(:logger).and_return(nil, logger) + end + + shared_examples_for 'uses --log-dir option' do + context 'when --log-dir is specified' do + let(:options) { ['--log-dir=/custom/log/dir'] } + + it 'creates the delayed_job.log in the specified directory' do + expect(Logger).to receive(:new).with('/custom/log/dir/delayed_job.log') + subject.run + end + end + end + + describe 'run' do + it 'sets the Delayed::Worker logger' do + expect(Delayed::Worker).to receive(:logger=).with(logger) + subject.run + end + + context 'when Rails root is defined' do + let(:rails_root) { Pathname.new '/rails/root' } + let(:rails) { double('Rails', :root => rails_root) } + + before do + stub_const('Rails', rails) + end + + it 'runs the Delayed::Worker process in Rails.root' do + expect(Dir).to receive(:chdir).with(rails_root) + subject.run + end + + context 'when --log-dir is not specified' do + it 'creates the delayed_job.log in Rails.root/log' do + expect(Logger).to receive(:new).with('/rails/root/log/delayed_job.log') + subject.run + end + end + + include_examples 'uses --log-dir option' + end + + context 'when Rails root is not defined' do + let(:rails_without_root) { double('Rails') } + + before do + stub_const('Rails', rails_without_root) + end + + it 'runs the Delayed::Worker process in $PWD' do + expect(Dir).to receive(:chdir).with(Delayed::Command::DIR_PWD) + subject.run + end + + context 'when --log-dir is not specified' do + it 'creates the delayed_job.log in $PWD/log' do + expect(Logger).to receive(:new).with("#{Delayed::Command::DIR_PWD}/log/delayed_job.log") + subject.run + end + end + + include_examples 'uses --log-dir option' + end + + context 'when an error is raised' do + let(:test_error) { Class.new(StandardError) } + + before do + allow(Delayed::Worker).to receive(:new).and_raise(test_error.new('An error')) + allow(subject).to receive(:exit_with_error_status) + allow(STDERR).to receive(:puts) + end + + it 'prints the error message to STDERR' do + expect(STDERR).to receive(:puts).with('An error') + subject.run + end + + it 'exits with an error status' do + expect(subject).to receive(:exit_with_error_status) + subject.run + end + + context 'when Rails logger is not defined' do + let(:rails) { double('Rails') } + + before do + stub_const('Rails', rails) + end + + it 'does not attempt to use the Rails logger' do + subject.run + end + end + + context 'when Rails logger is defined' do + let(:rails_logger) { double('Rails logger') } + let(:rails) { double('Rails', :logger => rails_logger) } + + before do + stub_const('Rails', rails) + end + + it 'logs the error to the Rails logger' do + expect(rails_logger).to receive(:fatal).with(test_error) + subject.run + end + end + end + end + describe 'parsing --pool argument' do it 'should parse --pool correctly' do command = Delayed::Command.new(['--pool=*:1', '--pool=test_queue:4', '--pool=mailers,misc:2']) @@ -40,13 +163,13 @@ expect(Dir).to receive(:mkdir).with('./tmp/pids').once [ - ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :queues => []}], - ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}], - ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}], - ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}], - ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :queues => ['test_queue']}], - ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :queues => %w[mailers misc]}], - ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :queues => %w[mailers misc]}] + ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => []}], + ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}], + ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}], + ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}], + ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => ['test_queue']}], + ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => %w[mailers misc]}], + ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :log_dir => './log', :queues => %w[mailers misc]}] ].each do |args| expect(command).to receive(:run_process).with(*args).once end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/message_sending_spec.rb new/spec/message_sending_spec.rb --- old/spec/message_sending_spec.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/message_sending_spec.rb 2015-09-24 23:40:11.000000000 +0200 @@ -62,6 +62,7 @@ class FairyTail attr_accessor :happy_ending def self.princesses; end + def tell @happy_ending = true end @@ -118,5 +119,25 @@ end.not_to change(fairy_tail, :happy_ending) end.to change { Delayed::Job.count }.by(1) end + + it 'does delay when delay_jobs is a proc returning true' do + Delayed::Worker.delay_jobs = ->(_job) { true } + fairy_tail = FairyTail.new + expect do + expect do + fairy_tail.delay.tell + end.not_to change(fairy_tail, :happy_ending) + end.to change { Delayed::Job.count }.by(1) + end + + it 'does not delay the job when delay_jobs is a proc returning false' do + Delayed::Worker.delay_jobs = ->(_job) { false } + fairy_tail = FairyTail.new + expect do + expect do + fairy_tail.delay.tell + end.to change(fairy_tail, :happy_ending).from(nil).to(true) + end.not_to change { Delayed::Job.count } + end end end diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/performable_method_spec.rb new/spec/performable_method_spec.rb --- old/spec/performable_method_spec.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/performable_method_spec.rb 2015-09-24 23:40:11.000000000 +0200 @@ -68,7 +68,7 @@ story = Story.create expect(story).to receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError)) expect(story).to receive(:tell).and_raise(RuntimeError) - expect { story.delay.tell.invoke_job }.to raise_error + expect { story.delay.tell.invoke_job }.to raise_error(RuntimeError) end it 'delegates failure hook to object' do @@ -98,7 +98,7 @@ story = Story.create expect(story).to receive(:error).with(an_instance_of(Delayed::Job), an_instance_of(RuntimeError)) expect(story).to receive(:tell).and_raise(RuntimeError) - expect { story.delay.tell }.to raise_error + expect { story.delay.tell }.to raise_error(RuntimeError) end it 'delegates failure hook to object' do diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/sample_jobs.rb new/spec/sample_jobs.rb --- old/spec/sample_jobs.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/sample_jobs.rb 2015-09-24 23:40:11.000000000 +0200 @@ -1,4 +1,5 @@ -class NamedJob < Struct.new(:perform) +NamedJob = Struct.new(:perform) +class NamedJob def display_name 'named_job' end @@ -22,11 +23,12 @@ cattr_accessor :runs @runs = 0 def perform - raise 'did not work' + raise Exception, 'did not work' end end -class CustomRescheduleJob < Struct.new(:offset) +CustomRescheduleJob = Struct.new(:offset) +class CustomRescheduleJob cattr_accessor :runs @runs = 0 def perform diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/spec/worker_spec.rb new/spec/worker_spec.rb --- old/spec/worker_spec.rb 2014-12-22 14:22:31.000000000 +0100 +++ new/spec/worker_spec.rb 2015-09-24 23:40:11.000000000 +0200 @@ -154,4 +154,22 @@ @worker.say(@text, Delayed::Worker.default_log_level) end end + + describe 'plugin registration' do + it 'does not double-register plugins on worker instantiation' do + performances = 0 + plugin = Class.new(Delayed::Plugin) do + callbacks do |lifecycle| + lifecycle.before(:enqueue) { performances += 1 } + end + end + Delayed::Worker.plugins << plugin + + Delayed::Worker.new + Delayed::Worker.new + Delayed::Worker.lifecycle.run_callbacks(:enqueue, nil) {} + + expect(performances).to eq(1) + end + end end