Rails

Using Raw SQL snippets to create Thinking-Sphinx Indexes

We have used Thinking-Sphinx in several projects to implement search engines. Thinking-Sphinx is a wrapper in Ruby to use ActiveRecord with Sphinx. But in the project I’m working right now I had to solve a different problem: the user should be able to search for formatted values, however without using the formatting characters, i.e, suppose a field containing 111.444.777-35, the user should be able to find it searching for 11144477735 (or a substring of it).

The default behaviour for Thinking-Sphinx is to index the exact values for each column it scans, but we can by-pass it. The Thinking-Sphinx index definition allows us to use raw SQL to define which values should be indexed for each attribute. In my case, I used the native PostgreSQL regular expressions functions to get rid of the punctuation characters, indexing only the “clean” version of each value. My database column is called “entity_number”:

define_index

Continue reading

How to send a large quantity of email in Rails, using ar_mailer

Radu Cojocaru

email-newsletterSending a large number of emails is not an easy task. It can take a lot of time so you need to do it asynchronously. Also, if you are on a shared hosting, you might be limited to a certain number of emails per hour, so you need to divide your list of email addresses in batches and send them at certain intervals.

There’s a plugin that can help us with this: ar_mailer.

This plugin adds a new deliver method to ActionMailer::Base :activerecord. In this delivery method, emails are not actually delivered, they are stored into a database table. A ruby script ar_sendmail can then be run (in a cron job) in order to send the emails saved in the database.

Step by step intructions on how to use the plugin:

1) Install the plugin

./script/plugin install git://github.com/adzap/ar_mailer.git

Add following line to config/environment.rb:

config.gem "adzap-ar_mailer",

Continue reading

Code blocks with optional arguments in Ruby 1.8.x

Ruby 1.9 has some cool new features. One of them is the ability to define default values for the arguments passed to code blocks, like in the following example:

pow = proc { |a, b = 2| a**b }
 
pow.call 3, 3
# 27
 
pow.call 3
# 9

This is very useful, for instance, when we dynamically create new methods using metaprogramming and want some of the arguments for these methods to be optional.

class MyMath
  class < < self
    define_method :pow do |base, exponent = 2|
      base**exponent
    end
  end
end
 
MyMath.pow 3 # 9
MyMath.pow 2, 3 # 8

But in Ruby 1.8.x we can’t do that, we can’t define default values the arguments of a code block. So what if your application runs on 1.8.x and you need to dynamically create methods with optional arguments? There is a solution, not so elegant as the

Continue reading

How long will it take to understand Rails 3 in and out?

Brad Midgley

According to David Heinemeier Hansson, you will never understand it all. Case in point? David himself.

That was the gist of his answer to how a new change to the way rails 3 renders actions in the browser would be handled for older browsers.

DHH gets involved in the parts he cares about and doesn’t worry about the exact details that have been delegated to someone else. He says we should not worry about knowing the whole framework because he doesn’t either. We don’t need to live up to an impossible standard.

Continue reading

Linux kiosk

Brad Midgley

I recently did some maintenance on a linux-based kiosk project. It runs linux and has a rails application serving content. Firefox is running in a kiosk mode and there is no keyboard or mouse present.

In adding a touchscreen from a different vendor, I found some details that can help make a kiosk application more responsive to touchscreen taps.

If you use linux in a touchscreen implementation, I recommend staying away from linux drivers offered by the vendor if possible. These were very out of date and X refused to load them. Use evtouch if possible. Install the package:

apt-get install xserver-xorg-input-evtouch

Note that most of the parameters to the driver are meant to discourage it from giving us complicated mouse emulation behavior like mousovers, middle/right clicks, dragging, etc. It needs to be just the simplest it can be for kiosk operation. People will push buttons too long, unintentionally try to drag

Continue reading

processing large xml data files

Brad Midgley

Getting to Railsconf 2009-26XML is often used to convey very large data sets. A perfect example is wikipedia’s data. The full wikipedia datasets are available in several different slices.

This dataset is about 24GB uncompressed for all articles in English. This is an excellent set to push the limits on your parser. Clearly, you can’t load a file like this into memory to process it. To handle a large file like this, you need to use sax mode. In this mode, methods in your code are called in the process of scanning through the xml document. The parser does not retain any state as it goes, so this mode scales up to whatever dataset size you need.

Nokogiri, rexml, and direct libxml interfaces in ruby all expose a sax mode for their interfaces. These differ in the way the

Continue reading

rails app producing “error or missing database”

Brad Midgley

You see an error like this in your logs: ActiveRecord::StatementInvalid (SQLite3::SQLException: SQL logic error or missing database: INSERT INTO…). The statement it prints out is working fine when you run it directly.

First, one thing you can tell from this error is you are running the application using a different user id than you are testing with. Are you completely sure you want to run sqlite3 on some kind of production system? :)

If you really are sure, then address the problem. The other identity can’t read/write the database file AND folder.

Continue reading

New graphing options for metric_fu

Output from reek code smell metric in metric_fu

Metric-fu is a handy set of rake tasks for running various metrics on your project’s ruby code and tracking the overall health of your code base. It can give you some interesting insights into things like how good the code coverage of your tests is and how well-designed your code is. It can also track the progress of your code over time, so you can see if things are getting better or worse.

Until recently, the only graphing tool available for metric_fu was gruff, which requires rmagick, which can be quite challenging to install and has been reported to be a resource hog and quite unstable.

Over the last few days I’ve been working on some updates to metric_fu that eliminate the dependency on rmagick by offering graphing with

Continue reading

Using cucumber and webrat for remote web testing

Cucumber and webrat serve as a powerful combination of tools for testing your web applications, but in its most common mode webrat can only test your application locally. Is it possible, you may ask, to use webrat to test a remote web site? The answer is yes, with a little tweaking. Webrat has a configuration option that tells it to use mechanize (a screen-scraping tool for ruby) instead of the built-in rails view testing system.

Continue reading

RadiantCMS customized

Brad Midgley

Kolob Canyon greenery

We are working on a very interesting project right now that allows us to adapt RadiantCMS to serve several different functions in a big picture system.

Radiant is a powerful content management system that has very good design for factoring out redundancy in your web content. Think of the Don’t-Repeat-Yourself principle applied to the web content, where we unfortunately put up with duplication far too often.

I was impressed with the hierarchy of web content in Radiant: layouts, snippets, ways to bring this together with tags. The tags reminded me of JSP work in past projects, but in this case it feels like borrowing a good idea and making it do more.

The extension design is interesting and allows us to focus customization work outside the core of the system. There are also a lot of interesting extensions already available.

Radiant runs

Continue reading