Configuring Rails and Splunk (Part 2)

Posted by dansketcher on August 30, 2010
Rails, splunk / No Comments

As my wrestle with configuring Splunk continues, I discovered the following helpful article explaining what happens where when forwarding data to a Splunk server.

Basically, it depends. Which is not surprising, given how complicated Splunk is, but it’s good to have on hand!

Combined with details on configuring linebreaking for multi-line events, perhaps I can get my rails logs handled as one entity in the near future…

Configuring Rails and Splunk (Part 1)

Posted by dansketcher on August 19, 2010
log4r, Rails, splunk / No Comments

To increase our application intelligence, I’ve started installing Splunk. However, what was not clear to me was the best way of getting my Web/App Rails logs into Splunk. The Web server is on a different box (as is the DB server) so I wanted to know the best way of getting the log data to the Splunk host.

Posts like this imply that Syslog-ng is a good way of getting data to Splunk, but the hidden detail is that if the Splunk server is down, the data is lost. A better way (thanks to the helpful folk on IRC) is to set up a Splunk forwarder on the Web/App/DB hosts and go from there. I went initially with setting them up as (free licenced) Standard Forwarders, so that the web interface is still running for configuration. Remember that any firewall (iptables) will need to be opened for the management ports (8000, 8089 ) and the forward port (9997).

Something else I discovered (but can no longer find in the docs) is that if you configure data sources using the web interface on a Forwarder, but subsequently change the instance to a LightForwarder, you also have to move the configuration files from $SPLUNK_HOME/etc/apps/search/local/ to $SPLUNK_HOME/etc/system/local/

Thinking-Sphinx :with => @geodist returns no results

Posted by dansketcher on January 01, 2010
gems, Rails, rant, software development / No Comments

Having just recently started using Thinking Sphinx, I’ve been toying with the geograhical search that it provides to simply do distance-based searches while querying other information. The Geo-searching part of the manual shows how to do searches limiting distance:

# Searching for places within 10km
Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => 0..10_000}
# Searching for places sorted by closest first
Place.search "pancakes", :geo => [@lat, @lng],
  :order => "@geodist ASC, @relevance DESC"

Unfortunately, the first style of query where @geodist is limited in range – when used as-is – does not work I found in my testing that I could order by @geodist as per the second query but when I added the :with range, I had no results.

The solution was actually simple, but really annoying. You have to use Floats instead of Integers for the range numbers! So changing the first query to:

Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => 0.0..10_000.0}

# or more neatly

Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => Range.new(0.0, 10_000.0)}

makes it all work. Now to get someone to update those docs!
Update: Updated!

Workling support for Synchronous AMQP RabbitMQ Clients and Amazon SQS Queues

Posted by dansketcher on November 16, 2009
gems, Rails, software development / No Comments

As a part of contracting work I have been doing for TouchLocal, I have just opensourced some code I wrote to support new Workling clients. As you may know, Workling is a Rails-oriented system for performing asynchronous processing and optionally returning data from these background workers. However, because of the implementation of the original AMQP client, you could not use RabbitMQ queues from non-evented Mongrel or Phusion Passenger servers (only evented Mongrel or Thin).

Building on the work of celdee-bunny and famoseagle-carrot, I implemented a RabbitMQ workling client that could be used from within Phusion Passenger and Mongrel. The Synchronous AMQP Workling client allows RabbitMQ to be used from Workling without requiring complicated changes to deployment scenarios. Also, I implemented the Return Store functionality, so that RabbitMQ users can get data back from the workers, just like when using Starling.

Additionally, it was useful at the time to add support for an Amazon SQS Workling client, more as an exercise in testing its performance than anything else. As with the SyncAmqpClient, support for the Return Store is present. One of the discoveries in working with Amazon SQS (via the RightAws gem) was the discovery that the default key structure for Workling (which uses colon characters for segment delimiters) is not supported by Amazon. As a result, if you define the keys used for AWS configuration, even if you don’t use them, they will change the Workling key structure. This is not a problem for new implementations, but for existing deployments adding this will mean that Workling cannot see the old queues and you may not be able to access them without removing the AWS configuration… not a deal breaker, but something to be aware of.

So, the TouchLocal github account holds the version of Workling that has these two implementation for now, at least until my pull request to the main branch is accepted :)

touchlocal-openx gem released

Posted by dansketcher on November 04, 2009
gems, hosting, Rails, software development / No Comments

After the last post, I spent a bit of time integrating my work (along with other upgrades) into a fork of the openx code on github and have released a gem version of it on GemCuttter.org

As you may or may not know, GemCutter.org will be the new default gem source by becoming rubygems.org. This is pretty exciting, because previously you had to have you project registered on rubygems.org in order to publish to it, or use the non-standard github gem host. Good stuff.

In any case, the new OpenX gem can be installed now by executing

sudo gem install touchlocal-openx --source "http://gemcutter.org"

# Load it using
require 'rubygems'
gem 'touchlocal-openx' 
require 'openx'

In Rails, include it like this in your Rails::Initializer block:

  config.gem "touchlocal-openx", :lib => "openx", :source => "http://gemcutter.org"

I’m a proud parent!

Minimum OpenX XMLRPC Ruby Client

Posted by dansketcher on September 29, 2009
hosting, Rails, software development / No Comments

2009-11-04 Update: Gem version released

For TouchLocal I am currently reworking some of the internal advertising systems to use OpenX. That way we get better reporting and better reliability, but we can use things like OpenX Direct Selection to make the best use of our existing infrastructure.

While there are 2 Ruby projects (1 a more recent fork of the other), both are oriented around the API for administering OpenX rather than serving ads via the API. So, here is the minimum you need to do from Ruby to get a banner served:

(Based on http://www.openx.org/en/docs/tutorials/Advanced+XML-RPC)

require 'xmlrpc/client'
# The settings are the  HTTP Headers - the PHP client sets many, but this is the minimum requirement
settings = {:cookies => [], :remote_addr => 'localhost'}

# 'what' in our case is a Direct Selection
params = [
  what = 'Plumber',
  campaignId = 0,
  target = '',
  source = '',
  withText = false,
  xmlContext = [{'!=' => 'campaignid:2'}]

b = nil
  server = XMLRPC::Client.new2("http://www.example.com/www/delivery/axmlrpc.php")
  b = server.call('openads.view', settings, *params)
rescue Exception => e
  puts e.message

In particular, note the content of the xmlContext parameter – that took me quite a few hours of digging through the PHP to work out. It’s an array of hashes, that use the keys “==” and “!=” to include or exclude banners based on the values. The values must be of the form “type:id”, with type being one of “campaignid”, “clientid”, “companionid”, and “bannerdid” (although technically, anything other than the first 3 is treated as a banner ID as of version 2.8.1 of OpenX)

I hope that saves someone the pain that I just went through to discover it!


Posted by dansketcher on April 06, 2009
Rails, software development / No Comments

So often in Ruby on Rails I find myself needing to convert a variable to a Boolean, and there is no built in way of doing it. So, how about this:

class Boolean
  def self.parse(obj)
    %w(true t 1 y).include?(obj.to_s.downcase.strip)

We use this at work and it’s a nice neat way of ensuring non-repetitive code reuse :)

Howto get capistrano-ext to run migrations as the correct environment

Posted by dansketcher on March 18, 2009
Rails / No Comments

If like me, you use the Capistrano Multistage extensions in the capistrano-ext plugin, you’d have noticed to your horror at some point that if you deploy to an environment other than production, your migrations will still run against the production database.

As the extension has no native support for this change in behaviour, the following method override will do the trick:

namespace :deploy do
  desc "Invoke the db migration using the correct stage"
  task :migrate, :roles => :app do
    send(run_method, "cd #{release_path} && rake db:migrate RAILS_ENV=#{stage} ")

While I’m here, here’s a simple trick I came up with: at work we use SVN branches heavily, some for each Sprint and then for Releases. As each release has its own branch, we need to change the Capistrano repository URL for each release, which is tedious and easy to forget.

Instead of the manual change, why not just ask SVN to tell you the current URL for the current path, which is in our case the root of the app? If, during the cap deploy, we run some ruby code to call ‘svn info’, like so (the first line is not necessary – it’s just there for context):

set :scm, :subversion
lines = %x[svn info]

The ‘svn info’ command obviously relies on the svn executable being in your path, but it is required for the rest of deployment so I’ll assume that’s already taken care of. The output of the command is something like:

Path: .
URL: https://svn.example.com/svn/project/branches/release_12345
Repository Root: https://svn.example.com/svn
Repository UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Revision: 1359
Node Kind: directory
Schedule: normal
Last Changed Author: dan
Last Changed Rev: 13582
Last Changed Date: 2009-03-17 22:34:44 +0000 (Thu, 19 Feb 2009)

the %x[] command takes this output and puts it in a long string with line breaks. Then, because this output is well formed, we can use a little regex-fu to get out the parh of the repository, like so (full commands):

lines = %x[svn info]
set :repository, /URL\: (.*)\n/.match(lines)[1]

The regular expression we use here is looking for a string that is between “URL: ” and “\n”, which as you would see from the output above, is:


Just what we were after!

Running one migration by hand

Posted by dansketcher on March 25, 2008
Rails / Comments Off on Running one migration by hand

Sometimes, you want to run a single migration by hand (to test it maybe…)

You can do this from the command line:

ruby script/runner 'require "db/migrate/005_create_blogs"; CreateBlogs.migrate(:down)'
ruby script/runner 'require "db/migrate/005_create_blogs"; CreateBlogs.migrate(:up)'

DRYing up Actions with a page_errorhandler

Posted by dansketcher on March 09, 2008
Rails / Comments Off on DRYing up Actions with a page_errorhandler

When I am developing page actions, I quite often find myself approaching a standard get/post action in the same way. This is particularly because I want to validate a number of ActiveRecord models, only progressing if they are ALL valid, and showing all errors at once

Of course, that means that I need to have a standard way of doing this, and a standard way of reporting it. It also implies the use of transactions, so that all saves are rolled back on failure. I have

So to begin with, an (hypothetical) example of my approach is:

def edit_user
@user = User.find(session[:user_id])
@address = @user.address
if request.post?
# note that I do not want to reference the objects in the
# method call - so that the changes are available to render
@user.transaction do
results = []
results << @user.save
results << @address.save
raise "Validation failed" if results.include?(false)
redirect_to :action => :show_user and return
rescue Exception
logger.warn{"Transaction terminated : #{e.message}"}
logger.warn{"@user : #{(@user.errors.full_messages.join('; ') rescue nil)}"}
logger.warn{"@address : #{(@address.errors.full_messages.join('; ') rescue nil)}"}

The first thing that is useful about this is that because all objects are saved together and the transaction is not terminated unless ONE of them is invalid, we will not redirect and the edit_user template will be rendered. Also, the validation errors are written to the log “just in case”

Also, note that I am using the log4r syntax of using braces instead of brackets. In Log4r, if the logger is not logging at the level that is specified, the code inside the braces will not be run. In the example above, if we are in production mode and we are not logging DEBUG, the e.backtrace method is actually not executed.

From here, lets DRY it up.

First, there’s those validation messages. There’s quite a lot of code here. I’ve approached this from 2 angles. First, in environment.rb, I put this code in

class ActiveRecord::Base
def full_messages
self.errors.full_messages.join('; ') rescue nil

and in application.rb

def validation_message(*objects)
str = "Validation error :"
objects.each{ |obj| str << "\n#{obj.class.name} => #{obj.full_messages}" }

This means that the validation message can now be handled by simply doing this in the rescue block:

logger.warn{validation_message(@user, @address)}

Next, because I use this structure regularly, I created a page_errorhandler method to contain the exception block that does this:

def page_errorhandler(*objects, &block)
rescue Exception => e
logger.warn{"Transaction terminated : #{e.message}"}

Combining these with our method above:

def edit_user
@user = User.find(session[:user_id])
@address = @user.address
if request.post?
page_errorhandler(@user, @address) do
# note that I do not want to reference the objects in the
# method call - so that the changes are available to render
@user.transaction do
results = []
results << @user.save results << @address.save raise "Validation failed" if results.include?(false) end redirect_to :action => :show_user and return

Neatens things up nicely!