I’ve just started playing with SELinux on Centos, and while the idea is great, it’s not exactly what I’m used to. Take for example adding Phusion Passenger to Apache. When I first restarted, I got this

[root@localhost modules]# /etc/init.d/httpd restart
Stopping httpd:                                            [FAILED]
Starting httpd: httpd: Syntax error on line 210 of /etc/httpd/conf/httpd.conf: Syntax error on line 1 of /etc/httpd/conf.d/passenger.conf: Cannot load /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/passenger-2.2.9/ext/apache2/mod_passenger.so into server: /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/passenger-2.2.9/ext/apache2/mod_passenger.so: failed to map segment from shared object: Permission denied
                                                           [FAILED]

Huh?

So it turns out after reading through some google results and then subsequently man httpd_selinux that the following will fix it:

chcon -R -h -t httpd_sys_script_exec_t /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/passenger-2.2.9/ext/apache2/mod_passenger.so

And so it does. “httpd_sys_script_exec_t” allows Apache to execute the SO, and we’re all good to restart.

I’ve been playing around with deprec for automatically provisioning new servers for my Rails applications. However, I hit a strange problem where certain command-line auto-completions did not work; I’d get an error message like this:

$ ls
temp

$ cd [hit tab to get autocomplete]
-sh: <( compgen -d -- '' ): No such file or directory
-sh: <( compgen -f -X  -- '' ): No such file or directory

The confusing thing was that only the unpriviliged user exhibited this behaviour, not the root user.

I tried reinstalling the bash-completion package to no avail (even an aptitude purge then aptitude install did not work). Eventually I came across this rather long post on bugs.debian.org where someone else has the same problem. Essentially it comes down to the unpriviliged user using /bin/sh as the default shell; under these circumstances autocompletion is supposed to not function (although presumably without error). This is despite /bin/sh being a symlink to /bin/bash

The fix, then, is actually quite simple. Change the default shell for all your unprivileged users to bash:

# usermod -s /bin/bash dansketcher

and then when you make new users, make sure you select bash explicitly:

# useradd newuser -m -s /bin/bash

As we enhanced the caching strategy through the site, and the Rails support for Memcache improved, we changed the Rails cache to use Memcache. This by implication meant that the default cache we had been using for actions (the filestore cache) was no longer used and the infrastructure team could not flush the cache in the same way it previously had.

So that we could use the Rails.cache helper and still have the benefit of the Action cache, we changed our production environment configuration to look like this:

# Namespace for memcache to automatically flush on app version updates.
# APP_REVISION is updated automatically via asking the Version Control System
ns = "dir_#{RAILS_ENV}_#{$persona}_#{APP_REVISION}"

# MC_ADDRESSES constant is put into the root namespace - we use it for other things
::MC_ADDRESSES = ['10.0.0.1:11211', '10.0.0.2:11211', '10.0.0.3:11211']

# Rails.cache is memcache
config.cache_store = :mem_cache_store, MC_ADDRESSES,{:namespace => ns}

# Force the ActionController cache store to be the file one as per the previous default.
config.action_controller.cache_store = :file_store, "#{RAILS_ROOT}/tmp/cache"

Happy days.

This class implements an interface to the Zeus ZXTM API, allowing (in its current state) the ability to programatically change active nodes and manage other details of the commonly changed data in the Load Balancer. On top of that, there is an included deploy.rb file that shows the deployment recipes that can be simply and easily added to your own Capistrano configuration to use this class quickly and easily.

Enjoy.

I’ve just gone through extracting the magic that makes it possible to do this programatically, released as DataFabric::Initializer. I hope it helps someone other than me!

git rebase --onto master branch1 branch2

Unfortunately for me, I seem to have either done it from the wrong place or done the wrong thing for my situation, because git rewound branch2 to the revision that it was branched from master and all the later commits were inaccessible!

However, I was not going to give up quite so easily, and I figured that once a revision was committed, it had to be somewhere.. Fortunately for me, I manged to find this post on how to recover lost commits in a git repository – I subsequently ran:

git reflog show
# find the commit with the last set of changes in your branch -
# good commit comments are so useful!
git co b6654bd
git co -b branch2_2

With my branch recovered, I did a simple rebase of the new (old) branch onto master and merged it, and all was well.

TypeError in BusinessController#view

can't dup NilClass

C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2189:in `dup'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2189:in `scoped_methods'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2193:in `current_scoped_methods'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:2183:in `scope'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:1548:in `find_every'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:1588:in `find_one'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:1574:in `find_from_ids'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/base.rb:616:in `find'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/associations/belongs_to_association.rb:44:in `find_target'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/associations/association_proxy.rb:240:in `load_target'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/associations/association_proxy.rb:112:in `reload'
C:/development/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activerecord-2.3.2/lib/active_record/associations.rb:1231:in `user'
C:/development/community_s0024_upgrade_upgrade/webapps/touchlocal/app/controllers/application_controller.rb:178:in `check_ticket_and_session'

Of course, that error means nothing as it is. What the underlying cause is though, is that the class reloader is trying to unload a class but it can’t because of included modules that can’t be unloaded. The fix is to use the keyword unloadable in the model to tell the reloader to force a reload on each request:

class User < ActiveRecord::Base
  unloadable

  ...
end

All fixed!

As a result, we went on a third rewrite effort. This time we went back to basics, and decided to use log4r to output a special log format that would be parsed and inserted into the tracking tables aynchronously. This would imply a delay, but only in the vicinity of a few minutes. The main design problems were ensuring stability of the processing platform and also the ability to retry files that might have been missed or errored. The repeatability was achieved by using a combination of a hash of the tracking data + the timestamp + some random information, and a (large) table that tracks this timestamps and hash combination (given that hashes can sometimes collide, it makes sense to add the timestamp as a factor).

The backend process to parse and record the information from this log file format was written in Ruby of course. In order to achieve stability I decided to use the Ruby Daemons gem. This handles PID file management and lots of other neat things for writing a daemon so the basics of long-running processes were not my ‘problem’ as such. So that these Ruby processes could scale up, I ensured as I was writing it that it would be aware of the potential error conditions of multiple processes, such as one process moving a file when another was looking for it (race conditions, etc). While the Daemons gem uses the fork() implementation on *nix for the standard headless run mode, it also support a non-background run mode which works on Windows. I also chose to use ActiveRecord, reusing the AR models from the Merb application.

One of my personal goals was to ensure the daemon was as self-contained as possible. For me, this meant that I wanted the Sysadmin to be able to check it out and run the start command and have it work on a standard base Ruby installation.

Here’s the initialisation code for the Daemon:

################
### REQUIRES ###
################

# This loads gems in vendor/gems (abstracted out so it can be used in rake tasks that haven't loaded ENV yet)
require "lib/local_gem_loader"

require 'rubygems'
require 'daemons'
require 'activerecord'
require 'json/pure'

require 'erb'
require 'cgi'

# +require+ all the models
model_path = File.expand_path(File.join(File.dirname(__FILE__), 'models'))
$LOAD_PATH.unshift model_path
Dir.glob(File.join(model_path, '*.rb')) do |file|
  require file
end

# Log4r does not work because the Daemons gem closes all open file descriptors.
#require 'log4r'
#require 'config/logger'
#logger = ::DEFAULT_LOGGER

#################
### CONSTANTS ###
#################

SLEEP_TIME_SECONDS = 5
FILE_NAME = 'tracking_daemon.rb' # name to report as the process

#####################
### CONFIGURATION ###
#####################

database_configuration_file = File.expand_path(File.join(File.dirname(__FILE__), 'config', 'database.yml'))
database_configuration = YAML::load(ERB.new(IO.read(database_configuration_file)).result)

######################
### INITIALISATION ###
######################

# Parse out the RAILS_ENV=production setting
ARGV.each do |arg|
  if arg.include?('=')
    key, val = arg.split('=', 2)
    ENV[key] ||= val
  elsif database_configuration.keys.include?(arg)
    ENV['RAILS_ENV'] ||= arg
  end
end
RAILS_ENV = (ENV['RAILS_ENV'] || "development").dup
puts "Starting #{FILE_NAME} daemon in #{RAILS_ENV} mode"

ActiveRecord::Base.configurations = database_configuration
ActiveRecord::Base.establish_connection RAILS_ENV

root_files_path       = File.expand_path(File.join(File.dirname(__FILE__), 'files'))
incoming_files_path   = File.expand_path(File.join(root_files_path, 'incoming_files'))

options = {
             :multiple   => true,
             :ontop      => false,
             :backtrace  => true,
             :log_output => true,
             :monitor    => true
           }

##############
### DAEMON ###
##############

Daemons.run_proc(FILE_NAME, options) do
  loop do
    # 1. Get the next file to process
    Dir.glob(File.join(incoming_files_path, '*')) do |incoming_file|
    end

    #...

    puts "Sleep #{SLEEP_TIME_SECONDS} sec" if RAILS_ENV == "development"
    sleep(SLEEP_TIME_SECONDS)
  end
end

The local_gem_loader is something of my own invention, based on the vendor/gems loader that was introduced in Rails 2. I wrote it initially to enable our (then) Rails 1.2.6 app to have vendored gems. It was very useful here to allow me to meet my desire to have this thing be checked out from SVN and started. Here it is – it’s pretty simple really:

# Load the gems in /vendor/gems
standard_dirs = ['rails', 'plugins']
gems          = Dir[File.join(__FILE__, "vendor/*/**") ]
if gems.any?
  gems.each do |dir|
    next if standard_dirs.include?(File.basename(dir))
    lib = File.join(dir, 'lib')
    $LOAD_PATH.unshift(lib) if File.directory?(lib)
    src = File.join(dir, 'src')
    $LOAD_PATH.unshift(src) if File.directory?(src)
  end
end

After including that line, I was able to vendor all the gems I needed (activerecord, json-pure, and even daemons) in the vendor/gems directory I created. After that, the ./models directory is loaded with the lines

# +require+ all the models
model_path = File.expand_path(File.join(File.dirname(__FILE__), 'models'))
$LOAD_PATH.unshift model_path
Dir.glob(File.join(model_path, '*.rb')) do |file|
  require file
end

Also note that as per the behaviour of how Daemons is designed, as it starts it closes all open file descriptors. While I read this in the documentation, I still tried to integrate Log4r, and spent a very confused hour wondering why all my log files were erroring on write… anyhoo…

After ensuring the models are in the load path and are ready to go, ActiveRecord needs to be initialised. I added the ability to at runtime choose the environment to write to database-wise, just like Rails does. This is achieved here:

# First load the Rails config/database.yml
database_configuration_file = File.expand_path(File.join(File.dirname(__FILE__), 'config', 'database.yml'))
database_configuration = YAML::load(ERB.new(IO.read(database_configuration_file)).result)

# Parse out the RAILS_ENV=production setting, which can be either in the form
# ruby tracking_daemon.rb start RAILS_ENV=production or
# ruby tracking_daemon.rb start production
# Note that the environments allowed are vaildated against the ones available in the database.yml
ARGV.each do |arg|
  if arg.include?('=')
    key, val = arg.split('=', 2)
    ENV[key] ||= val
  elsif database_configuration.keys.include?(arg)
    ENV['RAILS_ENV'] ||= arg
  end
end

RAILS_ENV = (ENV['RAILS_ENV'] || "development").dup
puts "Starting #{FILE_NAME} daemon in #{RAILS_ENV} mode"

ActiveRecord::Base.configurations = database_configuration
ActiveRecord::Base.establish_connection RAILS_ENV

At the end of this we have loaded the database config and connected to the database. Happy days. The only thing left is to start the daemon, which I chose to do in an inline fashion. Note that Daemons allows you to have the process report whatever name you like in the process lists, but I went with the name of the file itself for clarity:

root_files_path       = File.expand_path(File.join(File.dirname(__FILE__), 'files'))
incoming_files_path   = File.expand_path(File.join(root_files_path, 'incoming_files'))

options = {
             :multiple   => true, # allow multiple concurrent of the same
             :ontop      => false, # daemonise
             :backtrace  => true, # show full failure info
             :log_output => true,
             :monitor    => true # instantiate a monitor to restart as required
           }

Daemons.run_proc(FILE_NAME, options) do
  loop do
    # 1. Get the next file to process
    Dir.glob(File.join(incoming_files_path, '*')) do |incoming_file|
    end

    #...

    puts "Sleep #{SLEEP_TIME_SECONDS} sec" if RAILS_ENV == "development"
    sleep(SLEEP_TIME_SECONDS)
  end
end

All in all it’s been a great success – what was previously using a few backend servers running full whack to process all the incoming information, was now using 2 daemons on a single server in each datacentre. They hardly even show up on the top list. Excellent stuff.

http://tjulo.blogspot.com/2007/03/wordpress-and-source-code-posts.html

So, edit the file wp-includes/js/tinymce/plugins/wordpress/editor_plugin.js.

Comment out the section that replaces pre content:

/* var startPos = -1;
while ((startPos = content.indexOf('
', startPos+1);
var innerPos = content.indexOf('>', startPos+1);
var chunkBefore = content.substring(0, innerPos);
var chunkAfter = content.substring(endPos);

var innards = content.substring(innerPos, endPos);
innards = innards.replace(/\n/g, '
');
content = chunkBefore + innards + chunkAfter;
}*/

Then I had to update my CSS so that pre used the same class as code and then change my posts over to use pre instead of code.

Next step is to fix code too so that I don't have to think about it.

It does seem to have the side-effect of not automatically handling the escaping of < and > symbols outside those tags though, which for this post where I started with <code> and <pre> instead of code and pre, made things a bit of a mess.