Rack::Utils and CGI escape and unescape performance boost
As performance boosts are about speed, we’ll start with the benchmarks. Here is a run of spec/bench.rb, from the url_escape source tree.
Escape
| - | user | system | total | real |
|---|---|---|---|---|
| URLEscape::escape | 0.200000 | 0.000000 | 0.200000 | ( 0.196100) |
| CGI::escape | 3.830000 | 0.010000 | 3.840000 | ( 3.828438) |
| Rack::Utils::escape | 3.880000 | 0.010000 | 3.890000 | ( 3.880745) |
Unescape
| - | user | system | total | real |
|---|---|---|---|---|
| URLEscape::unescape | 0.090000 | 0.000000 | 0.090000 | ( 0.089190) |
| CGI::unescape | 2.820000 | 0.000000 | 2.820000 | ( 2.816234) |
| Rack::Utils::unescape | 3.140000 | 0.000000 | 3.140000 | ( 3.137291) |
URLEscape provides these two methods as a C extension, suitable for
use on ruby 1.8.6-8 and 1.9.1+; tested on linux, XP, and Vista.
The jruby version uses the java stdlib’s java.net.URLEncoder and
URLDecoder. We only see a 200-700% increase with this change, and would like to improve
on those numbers.
Why?
Josh Susser initially noticed the ability to overload #escape and #unescape while testing a client application. At the same time, we had just come across a bottleneck when regression testing FXC (a web app which serves configuration information to the FreeSWITCH softswitch) where requests were being delayed in our rack middleware, which parses the POST data sent by FreeSWITCH and routes requests to the ramaze application for processing. The delay was noticeable under loads of only 50 req/second; where rack became the bottleneck, not ramaze, the db, or any other factor. Adding the above library (on linux, with ruby 1.9.1) removed the delay in rack, pushing the work back to to the web app (or database) where it’s free to be as slow as it must. Optimally we’d like to perform at a speed equal to the database, making it the final bottleneck in a dynamic application.
Installation and Usage
To use URLEscape standalone
Install with one of the following methods:
- gem install url_escape
- get the tarball from RubyForge
- get the source from GitHub and rake install in the source top-level.
Then simply require “url_escape” and you have access to URLEscape.escape(string) and URLEscape.unescape(string)
To use URLEscape’s escape/unescape in place of CGI or Rack::Utils versions
gem install rackfastescape
or
gem install cgifastescape
This will install url_escape if it’s not already installed, as well.
You can optionally install rackfastescape or cgifastescape from rubyforge’s tarball or the github source (rake install as with urlescape). If you use tarball or source rake install, *you will have to manually install urlescape first.*
Once installed, simply use
require "rack_fast_escape"to replace Rack::Utils version, or
require "cgi_fast_escape"to replace the CGI version.
What else?
The ability of large posts to slow down a web application cannot be removed by just speeding up the POST parser. In order to alleviate the risk of such large POSTs being used to deny a service, firewall or web server throttling or limiting is a more reliable protection to enable. Here are a few examples:
Lighttpd: http://lighttpd.net
- Offers mod_evasive which limits connections per ip, as well as the ability to limit the data rate per connection.
Nginx: http://nginx.org
- Flexible limiting system, per vhost, per user, per connection.
Netfilter/QoS (linux): http://l7-filter.sourceforge.net/
Allow classifiation of HTTP packets so iptables/tc or whatever utility you’d like can have the info it needs about the HTTP protocol to make limiting/dropping/queueing decisions
Others: Apache, Squid, Litespeed, many others will have various methods of limiting size and frequency of requests.
Side Note
When speccing these libraries, a few implementation differences came to light which we’ll highlight here.
- Rack::Utils and CGI both throw errors on a mixed ASCII and Unicode string in ruby 1.9.1 and above
- Java’s URLDecoder and URLEncoder do not escape or unescape mixed ASCII/Unicode properly.
URLEscape (the C version) handles these cases properly, though we don’t expect you’d see them much in proper requests.
Thanks
To Evan Phoenix, Josh Susser, Trey Dempsey, Jayson Vaughn, Michael Fellinger, Kevin Berry, and all the other contributors of ideas and support who made this product a reality.
License
Nothing to fear, it’s MIT
Queuetastic released!
Looking for a way to monitor Asterisk queues in real-time and have queue reporting available? Without shelling out licensing costs?
Welcome to Queuetastic! An Asterisk Queue Real-time Monitoring, Reporting, and Analysis tool designed with Ruby on Rails.
Queuetastic primarily performs 3 major functions.
- Monitor Asterisk Queues in real-time using AJAX through the Asterisk Manager Interface
- Migrate and load Asterisk Queue data into a database of your choosing
- Extract queue and agent reports
Why Queuetastic?
- Asterisk Guru’s Queue Stats simply would not install in my environment. It couldn’t parse my queue_log properly. The source code was closed too. Boo, Next please…..
- Asterisk Queue Activity did install, but didn’t have the features I was looking for. Plus it’s web views are in Spanish and I don’t speak Spanish.
- I did not want to shell out for an expensive commercial license to monitor and report on an open-source PBX. No, Queuemetrics “free 2 Agent license” wouldn’t cut it either.
- There should be an open-source application for this!
- So I decided to write my own web-app using Ruby on Rails
Screenshots
The Dashboard

Live View

Download Queuetastic
To download a tarball release:
To download from upstream using git:
git clone git://rubyists.com/gits/queuetastic queuetastic
or
git clone git://github.com/thedonvaughn/queuetastic.git queuetastic
Wiki
For installation and support check out Queuetastic’s Wiki
CalendarHelper Meet Builder::XmlMarkup
Your choices for calendar builders for rails aren't many, but the CalendarHelper plugin originally by Jeremy Voorhis has worked well for me—that is until I needed to drastically change the HTML it outputs. Immediately after digging in to the code I found the following comment: "TODO Use some kind of builder instead of straight HTML." Read on for my conversion of CalendarHelper to use Builder::XmlMarkup.
CalendarHelper
It's usage is simple. It mixes in to ActionView::Base, adding a method "calendar" which takes a hash of options and yields to a block whose return value is rendered as each date cell. In addition to the standard concerns for a calendar, it allows for configurable links in the header, css classes on various elements, day of the week abbreviations, and most importantly day of the week ordering. The last feature causes a nice twist in the code, but we'll deal with that in a moment. First let's look at the code as it is originally.
This gives you a feel for how the HTML is generated.
# TODO Use some kind of builder instead of straight HTML cal = %(<table class="#{options[:table_class]}" border="0" cellspacing="0" cellpadding="0">) cal << %(<thead><tr>) if options[:previous_month_text] or options[:next_month_text] cal << %(<th colspan="2">#{options[:previous_month_text]}</th>) colspan=3 else colspan=7 end cal << %(<th colspan="#{colspan}" class="#{options[:month_name_class]}">#{Date::MONTHNAMES[options[:month]]}</th>) cal << %(<th colspan="2">#{options[:next_month_text]}</th>) if options[:next_month_text] cal << %(</tr><tr class="#{options[:day_name_class]}">)
This is a fairly common pattern: accumulate the output in a variable and return the result at the end. It works fine for small code blocks. Many of my helper methods use this same pattern. However, when the output grows in complexity the code can be difficult to maintain. It requires extra effort to make the HTML output readable within the code—here docs do help. And even then the outputted HTML isn't always pretty. Enter Builder::XmlMarkup. It exists to make XML generation simple and allow the structure of the code to mirror the structure of the outputted data.
Builder::XmlMarkup
Builder::XmlMarkup takes advantage of ruby's immensely useful method_missing. method_missing serves as a catch all for any instance method that is not defined in a class. Through it's magic ruby allows for things like:
xhtml = []
cal = Builder::XmlMarkup.new(:target => xhtml, :indent => 2)
cal.table(:class => options[:table_class], :border => 0, :cellspacing => 0, :cellpadding => 0) do
cal.thead do
cal.tr do
Any method called on the builder object is converted to a tag name. The method can be called with or without a block. When invoked with a block any calls to the builder contained in the block are nested within the block's tag. When invoked without a block the first argument is treated as the contents of the block. Either one can be invoked with a hash representing the attributes for the tag.
Converting the existing code to use the builder was a straight forward process of going through the code and replacing it with the corresponding builder. Unfortunately, the existing code's output statements did not logically follow the code nesting. In some places tags were started within loops and terminated at a different level. Rather than add conditionals to the logic so that it followed the structure of the document it was outputting, I took an opportunity to refactor the algorithm it used, which presents us with the twist I mentioned earlier.
Algorithm-smalgorithm
A month does not always start on the first day nor does it end on the last day of a week. The calendar then can be thought of as containing three distinct sections: days from the previous month, days within the month, and days from the next month.
This is how the existing code treats a month, as three distinct sections. Each section's loop may or may not terminate the row tr tag, which is where the nesting problem from above occurs.
To fit the problem in to the confines of our builder—each week's row tr tag must be a single block--we have to rethink the algorithm. A month calendar can be thought of as a two dimensional array. The first dimension representing the weeks of the month, and the second representing the days of a week. This fits nicely with the requirements of our builder, but we only know the first and last day of the month which do not necessarily start on the first and end of the last day of the week. What we need to find given the first and last day of the month, is the date of the first and last day of the week.
Finding this is easy enough. The Date object gives us the day of the week in the wday method. Counting is done with Sunday equal to zero. So we take the first day of the month's week day and subtract that many days to find the date of the start of the week.
first = Date.civil(options[:year], options[:month], 1)
first = first - first.wday
After using the corresponding algorithm to find the last day of the last week of the month we can then iterate from the new first day to the new last day of the month in 7 day chunks to build the calendar.
The Twist
But wait. Remember the twist? CalendarHelper allows us to define which day is the first day of the week. If the first day of the week is not Sunday, zero, the logic breaks. When counting days the week we have a series: 0, 1, 2, 3, 4, 5, 6, representing each day from Sunday through Saturday. When we change the start of the week—say to Wednesday—we end up with the new series Wed., Thurs., Fri., Sat., Sun., Mon., Tues., or 3, 4, 5, 6, 0, 1, 2.
What we want is the offset, the number of days, from the start of the week to the date of the start of the month within that week. Using that offset we can subtract it from the first of the month to find the date of the first day of that week just like we did above.
Our offset can be thought of as counting the days of the week from 0, 1, 2...6. We need a way to convert from the rotated series 3, 4, 5, 6, 0, 1, 2 to the zero based one. Enter the modulus operator, %. Our friend the modulus operator which gives the remainder of the division of the two operands, is wonderful for repeated, bounded series. If we have a series with n -> inf and the statement n % C with C the number of elements in our series we get 0, 1, 2, ... C, 0, 1, 2, ... C ... Useful but how does this help us?
0, 1, 2, Huh?
We want to convert from 3, 4, 5, 6, 0, 1, 2 to 0, 1, 2, 3, 4, 5, 6. Let's say the start of the month falls on a Friday, 5. If we take start of the shifted series, 3, and subtract it from the size of the series, 7, and take that sum and add it to the day, 5, and modulo that with size of the series, 7, we get
= (5 + (7 - 3)) % 7 = (5 + 4) % 7 = 9 % 7 = 2
2, which is counted as the third item in the series from zero to six, which when subtracted from the start of the month gives us the date of the start of the week.
But wait. Huh?
If we were to have the series start at 0 and apply the same algorithm we have
= (5 + (7 - 0)) % 7 = 12 % 7 = 5
which is just the next repetition of our series. By subtracting 3 we are accounting for the shift factor within our original series. Mmmmm math. Tasty.
To find the last day of the week we start with the last day of the week, 6, and subtract from it the offset between the last day of the month and the first day of the week. So now we have
first = Date.civil(options[:year], options[:month], 1)
last = Date.civil(options[:year], options[:month], -1)
first -= ((first.wday + (7 - options[:first_day_of_week])) % 7)
last += 6 - ((last.wday + (7 - options[:first_day_of_week])) % 7)
Full Code
I've submitted my changes to the CalendarHelper developers. This or something similar should show up soon in the calendar helper svn.
require 'date'
require 'builder'
# CalendarHelper allows you to draw a databound calendar with fine-grained CSS formatting
module CalendarHelper
VERSION = '0.3.0'
# Returns an HTML calendar. In its simplest form, this method generates a plain
# calendar (which can then be customized using CSS) for a given month and year.
# However, this may be customized in a variety of ways -- changing the default CSS
# classes, generating the individual day entries yourself, and so on.
#
# The following options are required:
# :year # The year number to show the calendar for.
# :month # The month number to show the calendar for.
#
# The following are optional, available for customizing the default behaviour:
# :table_class => "calendar" # The class for the <table> tag.
# :month_name_class => "monthName" # The class for the name of the month, at the top of the table.
# :other_month_class => "otherMonth" # Not implemented yet.
# :day_name_class => "dayName" # The class is for the names of the weekdays, at the top.
# :day_class => "day" # The class for the individual day number cells.
# This may or may not be used if you specify a block (see below).
# :abbrev => (0..2) # This option specifies how the day names should be abbreviated.
# Use (0..2) for the first three letters, (0..0) for the first, and
# (0..-1) for the entire name.
# :day_names => nil # An optional array of names for the days of the week.
# :first_day_of_week => 0 # Renders calendar starting on Sunday. Use 1 for Monday, and so on.
# :accessible => true # Turns on accessibility mode. This suffixes dates within the
# # calendar that are outside the range defined in the <caption> with
# # <span class="hidden"> MonthName</span>
# # Defaults to false.
# # You'll need to define an appropriate style in order to make this disappear.
# # Choose your own method of hiding content appropriately.
#
# :show_today => false # Highlights today on the calendar using the CSS class 'today'.
# # Defaults to true.
# :previous_month_text => nil # Displayed left of the month name if set
# :next_month_text => nil # Displayed right of the month name if set
# :pass_builder => false # Pass the builder as the second argument to the block. Ignore the
# the text that is returned from the block. You are responsible
# for adding a 'td' to the builder output.
#
# For more customization, you can pass a code block to this method, that will get one argument, a Date object,
# and return a values for the individual table cells. The block can return an array, [cell_text, cell_attrs],
# cell_text being the text that is displayed and cell_attrs a hash containing the attributes for the <td> tag
# (this can be used to change the <td>'s class for customization with CSS).
# This block can also return the cell_text only, in which case the <td>'s class defaults to the value given in
# +:day_class+. If the block returns nil, the default options are used.
#
# Example usage:
# calendar(:year => 2005, :month => 6) # This generates the simplest possible calendar.
# calendar({:year => 2005, :month => 6, :table_class => "calendar_helper"}) # This generates a calendar, as
# # before, but the <table<'s class
# # is set to "calendar_helper".
# calendar(:year => 2005, :month => 6, :abbrev => (0..-1)) # This generates a simple calendar but shows the
# # entire day name ("Sunday", "Monday", etc.) instead
# # of only the first three letters.
# calendar(:year => 2005, :month => 5) do |d| # This generates a simple calendar, but gives special days
# if listOfSpecialDays.include?(d) # (days that are in the array listOfSpecialDays) one CSS class,
# [d.mday, {:class => "specialDay"}] # "specialDay", and gives the rest of the days another CSS class,
# else # "normalDay". You can also use this highlight today differently
# [d.mday, {:class => "normalDay"}] # from the rest of the days, etc.
# end
# end
#
# An additional 'weekend' class is applied to weekend days.
#
# For consistency with the themes provided in the calendar_styles generator, use "specialDay" as the CSS class for marked days.
#
def calendar(options = {}, &block)
raise(ArgumentError, "No year given") unless options.has_key?(:year)
raise(ArgumentError, "No month given") unless options.has_key?(:month)
defaults = {
:table_class => 'calendar',
:month_name_class => 'monthName',
:other_month_class => 'otherMonth',
:day_name_class => 'dayName',
:day_class => 'day',
:abbrev => (0..2),
:day_names => nil,
:first_day_of_week => 0,
:accessible => false,
:show_today => true,
:previous_month_text => nil,
:next_month_text => nil,
:pass_builder => false
}
options = defaults.merge options
first = Date.civil(options[:year], options[:month], 1)
last = Date.civil(options[:year], options[:month], -1)
first -= ((first.wday + (7 - options[:first_day_of_week])) % 7)
last += 6 - ((last.wday + (7 - options[:first_day_of_week])) % 7)
today = first
if(options[:day_names])
day_names = options[:day_names]
else
i = 0
day_names = Date::DAYNAMES.inject(Array.new(7)) { |a, d| a[(i + (7 - options[:first_day_of_week])) % 7] = d; i += 1; a }
end
xhtml = []
cal = Builder::XmlMarkup.new(:target => xhtml, :indent => 2)
cal.table(:class => options[:table_class], :border => 0, :cellspacing => 0, :cellpadding => 0) do
cal.thead do
cal.tr do
if options[:previous_month_text] or options[:next_month_text]
cal.th(options[:previous_month_text], :colspan => 3)
colspan = 3
else
colspan = 7
end
cal.th("#{Date::MONTHNAMES[options[:month]]} #{options[:year]}", :colspan => colspan, :class => options[:month_name_class])
cal.th(options[:next_month_text], :colspan => 3, :class => 'rightMonth') if options[:next_month_text]
end
cal.tr(:class => options[:day_name_class]) do
day_names.each { |d| cal.th(d[options[:abbrev]], :scope => 'col') }
end
end
cal.tbody do
while(today <= last) do
cal.tr do
1.upto(7) do
if(today.month != options[:month].to_i)
css_class = options[:other_month_class] + ([0, 6].include?(today.wday) ? ' weekendDay' : '')
if(options[:accessible])
cal.td(:class => css_class) do
cal << today.day.to_s
cal.span(Date::MONTHNAMES[d.month], :class => 'hidden')
end
else
cal.td(today.day.to_s, :class => css_class)
end
else
if(block_given? && options[:pass_builder])
block.call(today, cal)
else
cell_text, cell_attrs = block.call(today) if block_given?
cell_text ||= today.day.to_s
cell_attrs ||= { :class => options[:day_class] }
cell_attrs[:class] += " weekendDay" if [0, 6].include?(today.wday)
cell_attrs[:class] += " today" if (today == Date.today) and options[:show_today]
cal.td(cell_attrs) { cal << cell_text }
end
end
today += 1
end
end
end
end
end
xhtml.join
end
end
ActionView rcsv template handler
The way the rjs and builder template handlers work in rails is nice. I do a lot of work with csv for generating reports, and having all the csv generation code in the controller doesn't seem to fit the MVC paradigm. I dug in to how ActionView template handlers work in Rails 2.1 and came up with the following solution.
The code uses the FasterCSV gem so you'll need that installed. One gotcha i noticed is that if format.csv is before format.html in the respond_to block the request gets sent incorrectly.
Code
RAILS_ROOT/config/initializers/template_handlers.rb
require 'rcsv_template_handler'
ActionView::Template.register_template_handler :rcsv, ActionView::TemplateHandlers::RCSV
RAILS_ROOT/lib/rcsv_template_handler.rb
require 'fastercsv'
module ActionView
module TemplateHandlers
class RCSV < TemplateHandler
include Compilable
def self.line_offset
2
end
def compile(template)
content_type_handler = (@view.send!(:controller).respond_to?(:response) ? "controller.response" : "controller")
"#{content_type_handler}.content_type ||= Mime::CSV
" +
"if controller.request.env['HTTP_USER_AGENT'] =~ /msie/i
" +
" controller.headers['Pragma'] = 'public'
" +
" #{content_type_handler}.content_type ||= Mime::TEXT
" +
" controller.headers['Cache-Control'] = 'no-cache, must-revalidate, post-check=0, pre-check=0'
" +
" controller.headers['Expires'] = '0'
" +
"else
" +
" #{content_type_handler.content_type ||= Mime::CSV
" +
"end
" +
"csv_doc = ::FasterCSV.new(:row_sep = "
") do |csv|
#{template.source}
end"
end
def cache_fragment(block, name = {}, options = nil)
@view.fragment_for(block, name, options) do
eval('csv_doc', block.binding)
end
end
end
end
end
Usage
RAILS_ROOT/app/controllers/name_controller.rb
class NameController < ApplicationController
def action
@data = Data.find :all
respond_to do |format|
format.html # Render action.html.erb
format.csv # Render action.csv.rcsv
end
end
end
RAILS_ROOT/app/view/name/action.csv.rcsv
@data.each do |datum|
csv << datum
end
RJS Errors with Firebug Goodness
I use firebug for all of my javascript debugging. It’s an excellent application, but one thing that has always bothered me is that rail’s built-in RJS exception handler shows it’s output in an alert. Firebug has the very handy console.log() function. Why can’t rails use that?
Now it can.
The code is smart enough to tell if Firebug is loaded or not and gracefully falls back to alert if it isn’t. As an added bonus you can now cut and paste the content as well as see multiple errors.
Usage
Copy the code to a file that is loaded at rails start up. RAILS_ROOT/config/initializers/rjs_debug.rb would work nicely.
module ActionView
module Helpers
module JavascriptHelper
class JavaScriptGenerator
module GeneratorMethods
def to_s #:nodoc:
returning javascript = @lines * $/ do
if ActionView::Base.debug_rjs
source = javascript.dup
javascript.replace "try {
#{source}
} catch (e) "
javascript << "{ var rjs_debug = alert; if(window.console && window.console.firebug) { rjs_debug = console.log }; rjs_debug.apply(this, ['RJS error:\n\n' + e.toString()]); rjs_debug.apply(this, ['#{source.gsub('\','