How to create a Sidekiq service application without Rails

Sometimes you want to create a small Sidekiq service to process requests, without all the extra bloat (and memory usage) from a Rails framework. It turns out that creating such service is not all that hard to do. Create your application using the folders and files below as your starting point.

Gemfile

# Gemfile
source 'https://rubygems.org'
ruby "2.5.0"
git_source(:github) do |repo_name|
  repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
  "https://github.com/#{repo_name}.git"
end

gem "sidekiq"

config/sidekiq.yml

---
:concurrency: <%= ENV["CONCURRENCY"] || 25 %>
:queues:
 - default

config/application.rb

# config/application.rb

ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile", __FILE__)

require "bundler/setup" if File.exist?(ENV["BUNDLE_GEMFILE"])
Bundler.require

ENV["RACK_ENV"] ||= "development"

module Example
 class Application
   def self.root
     @root ||= File.dirname(File.expand_path('..', __FILE__))
   end

   def self.env
     @env ||= ENV["RAILS_ENV"] || ENV["RACK_ENV"] || ENV["ENV"] || "development"
   end

   def self.logger
     @logger ||= Logger.new(STDOUT)
   end
 end
end

require_relative "initializers/sidekiq"

config/initializers/sidekiq.rb

# config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: ENV["REDIS_URL"] }
end

app/workers/my_worker.rb

# app/workers/my_worker.rb

class MyWorker
  include Sidekiq::Worker

  def perform(*args)
    # Do work here
  end
end

Once you have created those files, then you can run the following on the command line:

bundle exec sidekiq -e ${RACK_ENV:-development} -r ./config/application.rb -C ./config/sidekiq.yml

If you want to enqueue jobs from another application using Sidekiq, its as easy as:

Sidekiq::Client.enqueue_to "default", MyWorker, "param_1" => "1", "param_2" => "2"

When you have all this working, then you can start adding things like your database connections and other gems and code to the worker. Let me know if this helps anyone or if you run into any problems!

Installing mysql2 Gem error targeting Mac OS X 10.5 or later

I recently tried to install the mysql2 gem and was presented with the following error. I was on El Capitan 10.11.6 and using ruby 1.9.3-p125.

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/Users/…/.rvm/rubies/ruby-1.9.3-p125/bin/ruby extconf.rb
checking for rb_thread_blocking_region()… yes
checking for rb_wait_for_single_fd()… yes
checking for mysql.h… yes
checking for errmsg.h… yes
checking for mysqld_error.h… yes
creating Makefile

make

compiling client.c
couldn’t understand kern.osversion `15.6.0’
compiling mysql2_ext.c
couldn’t understand kern.osversion `15.6.0’
compiling result.c
couldn’t understand kern.osversion `15.6.0’
linking shared-object mysql2/mysql2.bundle
couldn’t understand kern.osversion `15.6.0’
ld: -rpath can only be used when targeting Mac OS X 10.5 or later
collect2: ld returned 1 exit status
make: *** [mysql2.bundle] Error 1

Gem files will remain installed in /Users/…/vendor/bundle/ruby/1.9.1/gems/mysql2-0.3.11 for inspection.
Results logged to /Users/…/vendor/bundle/ruby/1.9.1/gems/mysql2-0.3.11/ext/mysql2/gem_make.out

An error occurred while installing mysql2 (0.3.11), and Bundler cannot continue.
Make sure that `gem install mysql2 -v ‘0.3.11’` succeeds before bundling.

After some searching I found out the solution was to install the gem by setting the Mac Deployment Target as so:

MACOSX_DEPLOYMENT_TARGET=10.9 gem install mysql2

Hope this helps someone else in the future!

How to trim a string and fill remaining space in Ruby

So I needed a way to take a string, and depending on its size, shorten it and fill the remaining space with some arbitrary sequence of characters. Ljust would work, except for the fact that it will fill a string up to a given length, but I only needed to do this when over a certain size. Here is an example:

str = "Somereallylongstring"
if str.length > 10
  puts str[0..7] + "…"
else
  puts str
end
# => "Somerea..."

This is the basic idea of what I wanted to do. I decide to make it cleaner and override the String class like so:

class String
  def lfill(len = self.length, fill = ".")
    tmp = self[0..(len – 3)] + "..." if self.length – 3 > len
    return tmp || self
  end
end

nstr = "Somereallylongstring"
puts str.lfill(10)
#=> "Somerea..."

 

How to merge strings just like ActiveRecord conditions do

While you can achieve the same functionality using sprintf, this may provide a cleaner approach and one that you are more familiar with. This will allow you to build a string the same way you can use ActiveRecord and the :conditions option.

Basically how this works is by overriding the Array class and adding a method to merge the string and values together into unified string! Enough talk, lets see some code:

class Array
  def merge
    statement, *values = self
    expected = statement.count("?")
    provided = values.size
    unless expected.eql?(provided)
      raise "wrong number of bind variables (#{provided} for #{expected}) in: #{statement}"
    end
    bound = values.dup
    statement.gsub("?") { bound.shift }
  end
end

As you can see, if you do not provide the right number of values for the statement, it will raise and error. Here is how you would use it:

puts ["Hello ?, how are you", "John"].merge
#=> Hello John, how are you

Likewise, you can use variables to hold values:

message = "Hello ?, how are you"
name = "John"
puts [message, name].merge
#=> Hello John, how are you

This will also work with multiple values:

puts ["Hello ?, ? and ?, how are you", "John", "Joe", "Jim"].merge
#=> Hello John, Joe, and Jim, how are you

The only downside to this currently is that you cannot use a ? in the string you are merging, as it will think its a binding character.

NoMethodError on nil.each?

Have you ever have to iterate over an array from say the params hash? You probably have added some code to make sure that data exists in the hash element first before doing the loop. This is a cool little trick to help condense some code. Normally I would write this:

if params[:accounts] and params[:accounts].size > 0
  params[:accounts].each do |account|
    # Do something with account here
  end
end

As you can see, kind of ugly. Here is the change:

(params[:accounts] || []).each do |account|
  # Do something with account
end

AHH!!! Much nicer!

Moving files in Rails using String

Since ruby allow for really easy overriding of objects and classes, I decided to make it easier and cleaner to move my files. Here is how you currently do it:

FileUtils.move("/path/to/file", "/path/to/new/file")

I know this isn’t a huge deal, but I would like to clean it up a little bit. I used an initializer (RAILS_ROOT/config/initializers/core_extensions.rb) to override the String class as such:

class String
  def move(to = nil)
    if to && File.exist?(self)
      FileUtils.move(self, to)
    end
  end
end

This allows me to do the following to move files:

"/path/to/file".move("/path/to/new/file")

I though it was pretty cool!

Easy Hash to XML and back conversion in Ruby

Instead of spending hours trying to write something custom to convert your hash into xml or vice versa, you can do it in one line. If you are already in a Rails environment, you are good to go, but if using ruby, include the ActiveSupport gem:

require 'rubygems'
require 'active_support'

Now that you have, we can convert a xml to a hash:

hash = Hash.from_xml(xml_stream)

And a hash to xml is stupid simple:

xml = hash.to_xml

There you have it, simple xml to hash and hash to xml!

Capistrano, CVS, and connection refused issue

If you ever find yourself using CVS as your repository, and are trying to deploy your projects via capistrano, you may run into a connection refused issue. The easiest way I found a solution around this was to modify the capistrano file: “/capistrano-2.1.0/lib/capistrano/recipes/deploy/scm/cvs.rb”. This of course is in your ruby gems folder.I could have added just the change to my project, but I am lazy, and need this for multiple CVS project deploying from this machine. The change I made was on line 145, which I added the CVS_RSH=ssh line:

"export CVS_RSH=ssh && mkdir -p #{ dest } && cd #{ dest }"

Rails Log Analyzer (RAWK)

Came across a sweet rails log analyzer. Doesn’t require that you use syslog or anything like the other log parsers that are out there. This one will work right on your development or production log files. Its called RAWK and you can find it here: http://rubyforge.org/frs/?group_id=2517&release_id=15246

%d bloggers like this: