Sketchpad – Dan Sketcher’s personal blog

Having just recently started using Thinking Sphinx, I’ve been toying with the geograhical search that it provides to simply do distance-based searches while querying other information. The Geo-searching part of the manual shows how to do searches limiting distance:

# Searching for places within 10km
Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => 0..10_000}
# Searching for places sorted by closest first
Place.search "pancakes", :geo => [@lat, @lng],
  :order => "@geodist ASC, @relevance DESC"

Unfortunately, the first style of query where @geodist is limited in range – when used as-is – does not work I found in my testing that I could order by @geodist as per the second query but when I added the :with range, I had no results.

The solution was actually simple, but really annoying. You have to use Floats instead of Integers for the range numbers! So changing the first query to:

Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => 0.0..10_000.0}

# or more neatly

Place.search "pancakes", :geo => [@lat, @lng],
  :with => {"@geodist" => Range.new(0.0, 10_000.0)}

makes it all work. Now to get someone to update those docs!
Update: Updated!

As a part of contracting work I have been doing for TouchLocal, I have just opensourced some code I wrote to support new Workling clients. As you may know, Workling is a Rails-oriented system for performing asynchronous processing and optionally returning data from these background workers. However, because of the implementation of the original AMQP client, you could not use RabbitMQ queues from non-evented Mongrel or Phusion Passenger servers (only evented Mongrel or Thin).

Building on the work of celdee-bunny and famoseagle-carrot, I implemented a RabbitMQ workling client that could be used from within Phusion Passenger and Mongrel. The Synchronous AMQP Workling client allows RabbitMQ to be used from Workling without requiring complicated changes to deployment scenarios. Also, I implemented the Return Store functionality, so that RabbitMQ users can get data back from the workers, just like when using Starling.

Additionally, it was useful at the time to add support for an Amazon SQS Workling client, more as an exercise in testing its performance than anything else. As with the SyncAmqpClient, support for the Return Store is present. One of the discoveries in working with Amazon SQS (via the RightAws gem) was the discovery that the default key structure for Workling (which uses colon characters for segment delimiters) is not supported by Amazon. As a result, if you define the keys used for AWS configuration, even if you don’t use them, they will change the Workling key structure. This is not a problem for new implementations, but for existing deployments adding this will mean that Workling cannot see the old queues and you may not be able to access them without removing the AWS configuration… not a deal breaker, but something to be aware of.

So, the TouchLocal github account holds the version of Workling that has these two implementation for now, at least until my pull request to the main branch is accepted :)

After the last post, I spent a bit of time integrating my work (along with other upgrades) into a fork of the openx code on github and have released a gem version of it on GemCuttter.org

As you may or may not know, GemCutter.org will be the new default gem source by becoming rubygems.org. This is pretty exciting, because previously you had to have you project registered on rubygems.org in order to publish to it, or use the non-standard github gem host. Good stuff.

In any case, the new OpenX gem can be installed now by executing

sudo gem install touchlocal-openx --source "http://gemcutter.org"

# Load it using
require 'rubygems'
gem 'touchlocal-openx'
require 'openx'

In Rails, include it like this in your Rails::Initializer block:

  config.gem "touchlocal-openx", :lib => "openx", :source => "http://gemcutter.org"

I’m a proud parent!

2009-11-04 Update: Gem version released

For TouchLocal I am currently reworking some of the internal advertising systems to use OpenX. That way we get better reporting and better reliability, but we can use things like OpenX Direct Selection to make the best use of our existing infrastructure.

While there are 2 Ruby projects (1 a more recent fork of the other), both are oriented around the API for administering OpenX rather than serving ads via the API. So, here is the minimum you need to do from Ruby to get a banner served:

(Based on http://www.openx.org/en/docs/tutorials/Advanced+XML-RPC)

require 'xmlrpc/client'
# The settings are the  HTTP Headers - the PHP client sets many, but this is the minimum requirement
settings = {:cookies => [], :remote_addr => 'localhost'}

# 'what' in our case is a Direct Selection
params = [
  what = 'Plumber',
  campaignId = 0,
  target = '',
  source = '',
  withText = false,
  xmlContext = [{'!=' => 'campaignid:2'}]
]

b = nil
begin
  server = XMLRPC::Client.new2("http://www.example.com/www/delivery/axmlrpc.php")
  b = server.call('openads.view', settings, *params)
  b['html']
rescue Exception => e
  puts e.message
end

In particular, note the content of the xmlContext parameter – that took me quite a few hours of digging through the PHP to work out. It’s an array of hashes, that use the keys “==” and “!=” to include or exclude banners based on the values. The values must be of the form “type:id”, with type being one of “campaignid”, “clientid”, “companionid”, and “bannerdid” (although technically, anything other than the first 3 is treated as a banner ID as of version 2.8.1 of OpenX)

I hope that saves someone the pain that I just went through to discover it!

So often in Ruby on Rails I find myself needing to convert a variable to a Boolean, and there is no built in way of doing it. So, how about this:

class Boolean
  def self.parse(obj)
    %w(true t 1 y).include?(obj.to_s.downcase.strip)
  end
end

We use this at work and it’s a nice neat way of ensuring non-repetitive code reuse :)

If like me, you use the Capistrano Multistage extensions in the capistrano-ext plugin, you’d have noticed to your horror at some point that if you deploy to an environment other than production, your migrations will still run against the production database.

As the extension has no native support for this change in behaviour, the following method override will do the trick:

namespace :deploy do
  desc "Invoke the db migration using the correct stage"
  task :migrate, :roles => :app do
    send(run_method, "cd #{release_path} && rake db:migrate RAILS_ENV=#{stage} ")
  end
end

While I’m here, here’s a simple trick I came up with: at work we use SVN branches heavily, some for each Sprint and then for Releases. As each release has its own branch, we need to change the Capistrano repository URL for each release, which is tedious and easy to forget.

Instead of the manual change, why not just ask SVN to tell you the current URL for the current path, which is in our case the root of the app? If, during the cap deploy, we run some ruby code to call ’svn info’, like so (the first line is not necessary – it’s just there for context):

set :scm, :subversion
lines = %x[svn info]

The ’svn info’ command obviously relies on the svn executable being in your path, but it is required for the rest of deployment so I’ll assume that’s already taken care of. The output of the command is something like:

Path: .
URL: https://svn.example.com/svn/project/branches/release_12345
Repository Root: https://svn.example.com/svn
Repository UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Revision: 1359
Node Kind: directory
Schedule: normal
Last Changed Author: dan
Last Changed Rev: 13582
Last Changed Date: 2009-03-17 22:34:44 +0000 (Thu, 19 Feb 2009)

the %x[] command takes this output and puts it in a long string with line breaks. Then, because this output is well formed, we can use a little regex-fu to get out the parh of the repository, like so (full commands):

lines = %x[svn info]
set :repository, /URL\: (.*)\n/.match(lines)[1]

The regular expression we use here is looking for a string that is between “URL: ” and “\n”, which as you would see from the output above, is:

https://svn.example.com/svn/project/branches/release_12345

Just what we were after!

Sometimes, you want to run a single migration by hand (to test it maybe…)

You can do this from the command line:


ruby script/runner 'require "db/migrate/005_create_blogs"; CreateBlogs.migrate(:down)'
ruby script/runner 'require "db/migrate/005_create_blogs"; CreateBlogs.migrate(:up)'


When I am developing page actions, I quite often find myself approaching a standard get/post action in the same way. This is particularly because I want to validate a number of ActiveRecord models, only progressing if they are ALL valid, and showing all errors at once

Of course, that means that I need to have a standard way of doing this, and a standard way of reporting it. It also implies the use of transactions, so that all saves are rolled back on failure. I have

So to begin with, an (hypothetical) example of my approach is:


def edit_user
@user = User.find(session[:user_id])
@address = @user.address
if request.post?
begin
# note that I do not want to reference the objects in the
# method call - so that the changes are available to render
@user.transaction do
results = []
results << @user.save
results << @address.save
raise "Validation failed" if results.include?(false)
end
redirect_to :action => :show_user and return
rescue Exception
logger.warn{"Transaction terminated : #{e.message}"}
logger.warn{"@user : #{(@user.errors.full_messages.join('; ') rescue nil)}"}
logger.warn{"@address : #{(@address.errors.full_messages.join('; ') rescue nil)}"}
logger.debug{e.backtrace}
end
end
end


The first thing that is useful about this is that because all objects are saved together and the transaction is not terminated unless ONE of them is invalid, we will not redirect and the edit_user template will be rendered. Also, the validation errors are written to the log “just in case”

Also, note that I am using the log4r syntax of using braces instead of brackets. In Log4r, if the logger is not logging at the level that is specified, the code inside the braces will not be run. In the example above, if we are in production mode and we are not logging DEBUG, the e.backtrace method is actually not executed.

From here, lets DRY it up.

First, there’s those validation messages. There’s quite a lot of code here. I’ve approached this from 2 angles. First, in environment.rb, I put this code in


class ActiveRecord::Base
def full_messages
self.errors.full_messages.join('; ') rescue nil
end
end


and in application.rb

def validation_message(*objects)
str = "Validation error :"
objects.each{ |obj| str << "\n#{obj.class.name} => #{obj.full_messages}" }
str
end


This means that the validation message can now be handled by simply doing this in the rescue block:


logger.warn{validation_message(@user, @address)}


Next, because I use this structure regularly, I created a page_errorhandler method to contain the exception block that does this:


def page_errorhandler(*objects, &block)
begin
yield
rescue Exception => e
logger.warn{"Transaction terminated : #{e.message}"}
logger.warn{validation_message(*objects)}
logger.debug{e.backtrace}
end
end


Combining these with our method above:


def edit_user
@user = User.find(session[:user_id])
@address = @user.address
if request.post?
page_errorhandler(@user, @address) do
# note that I do not want to reference the objects in the
# method call - so that the changes are available to render
@user.transaction do
results = []
results << @user.save
results << @address.save
raise "Validation failed" if results.include?(false)
end
redirect_to :action => :show_user and return
end
end
end


Neatens things up nicely!

One of the things that I like about the .net web application setup is the concept of Master Pages. The most common usage of this is if you have a Home Page that has a slightly different layot from the rest of the site, but you want to share the base DIV structure and assets without having to copy them into another layout… Keep it DRY!

Although there is no “out of the box” solution for sharing common layout setups in Rails except for using Partials, there is in fact a way of replicating this functionality. First, you need to be familiar with the ActionView::Helpers::CaptureHelper class and the methods in it.

So, what you do is create a container rhtml layout that contains the common layout elements, such as the base HTML and div structure.

container.rhtml
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<body>
<%= yield :layout -%>
</div>
</body>
</html>


From there, you can create as many other layouts that reference that container, such as one called application.rhtml:
<% @content_for_layout = capture do %>
<div id='a_new_div' />
<%= yield :layout %>
<% end %>
<%= render 'layouts/container', { 'content_for_layout' => @content_for_layout } %>


Note what happens there – the capture method is used to catch the rendering of this layout file. Then, instead of just allowing it to render, this output is injected into the container.rhtml file. In this way, we have chained the inner layout (application.rhtml) into the outer (container.rhtml) layout.

In our example, the “<div id=’a_new_div’ />” will be injected, along with the data from the view, into the container. There is nothing stopping you from doing this to another layer, although I doubt how often you’d need to do that!

From our example above, we could also easily create a home.rhtml file that had slightly different layout from our application.rhtml file and use that from our HomeController without concern – all of the pages would then render in the correct layout while giving us the benefit of a shared outer container!

I have used this in production, and it works a treat both in terms of performance and code maintainability.

So, JRuby looks pretty cool, and I’ve got some code written that uses a weird BouncyCastle encryption implementation going in my rails app. Works well, the WAR deployment works (with goldspike) and everything’s going swimmingly.

Until, of course, I tried to integrate with Another Project that uses the ActiveSalesforce. Behind the scenes, ActiveSalesforce makes SSL encrypted connections to Salesforce API endpoints and performs operations. This was working fine until I started the app under JRuby. All of a sudden, the salesforce connections stopped working.

A bit of digging later, and I discovered that the JRuby-OpenSSL implementation is causing reads of the HTTP responses to stall. JRuby HTTP is fine, Ruby HTTPS is fine, but JRuby HTTPS is not. Boo.

It’s a bug, now. I hope it’s fixed soon!