Category Archives: ruby on rails

uninitialized constant MysqlCompat::MysqlRes…getting Snow Leopard, Rails, and MySQL to play nicely

Leave a reply

You’ve probably done it…set up your machine, updated Rails, installed MySQL, then attempted to start your server. You’re cruising…then blam!

 uninitialized constant MysqlCompat::MysqlRes

Then you’ve probably hit Google a few times…tried specifying  x86_64 ARCHFLAGS, installing and reinstalling MySQL and the MySQL gem, and even trying the MySQL 2.7 gem.

No dice. You and I both.

Well, hear this…you might have another out! A few weeks ago I was working with the guys over at Blue Box Group to get a Memcached instance up. They had suggested I install Homebrew, a package manager for OS X. I suggest you install it too.

Be sure to uninstall the any copies of your mysql gem; then, once you have it Homebrew running, try the following.

icarus$ sudo brew install mysql
==> Downloading ftp://ftp.cwru.edu/pub/bash/readline-6.1.tar.gz
################################################## 100.0%
==> Downloading patches
################################################## 100.0%

** snip **

icarus$ sudo gem install mysql -- --with-mysql-dir=/usr/local --with-mysql-config=/usr/local/bin/mysql_config

Supplies!

Using regular expressions? The Devil is in the details.

Leave a reply

Recently, I ran into a little roadblock as I’ve been cleaning up some bugs for one of my applications. I rely heavily on cross pollination of data between sites to drive traffic across my network. In this particular situation, a simple collection of linked image thumbnails accomplishes this task.

One of the helper methods that I use returns the file name of a supplied post object. To do this, I perform a regular expression match against the post’s URL and return the result.

# post.url(:icon) = id/10/icon/foo.jpg
# returns "icon/foo.jpg"

def filename_from_post(post)
  return post.image.url(:icon).match(/icon\/.*\.jpg)/)[0]
end

That worked great until my application would randomly throw a NoMethodError. I spent a while scratching my head, looking over my code, checking my SQL queries, and so on and so forth. Finally, I figured it out…I had forgotten to specify a case insensitive search!

# post.url(:icon) = id/10/icon/bar.JPG
# returns nil

>> post.url(:icon).match(/icon\/.*\.jpg/)
=> nil

>> post.url(:icon).match(/icon\/.*\.jpg/i)
=> #<MatchData "icon/bar.JPG">

I guess what they say is true…the Devil is in the details.

Getting around ext3 inode limitations using MD5 file paths and Paperclip interpolations

Leave a reply

I really wished somebody had suggested I use a hashed file paths when I was planning out my application. Let me be that guy.

Anybody building an application that allows file uploads should seriously consider use hashed file paths. Why? Everybody dreams of their app becoming the next big thing, but what happens when that dream turns into a reality? It may not happen overnight, but there is a limit to the number of files you can fit into one directory.

A few weeks ago I woke up to an inbox full of emails from my Rails app titled, “Errno::EMLINK (Too many links)”. After some quick research I learned most Linux distributions use the ext3 file system which has an upper limit of 32,000 links per inode. At the time of my little meltdown I had been storing user uploads in one directory. Apparently I’ve reached the limit’

Further research revealed that the most painless solution to this problem was to use hashed file paths. Instead of saving files to “files/1/resized/image.jpg” files would be stored in “files/cdd/01d/2cf/resized/image.jpg”.

I ran the idea by the guys at my data center and it they agreed that it was the correct course of action. Wanting to further understand how a hashed file path would be different than my current method, I did some more digging. Here’s how Nick, one of the data center guys explained it:

“If I understand the way an MD5 hash operates, it will use the first few hex numerals to make the first folder, then the next few for another, and do a couple of nested layers. The idea being that while each md5 sum itself is unique, there are far fewer than 32,000 combinations for each pair, triplet, etc of hex digits, so you don’t have to worry about hitting the directory limitation. For example, if you have 10 files that all start with “1c”, let’s say, they would go into the same directory, and their nested directories would be based off of the next characters in the md5. This way, what is 10 folders in a pure :id system have just become one.

In order to hit that limit again, you’d need to have 32,000 uploads that all share the same X number of characters in their md5 sum (where x is however many are being used to generate the hash), and the chances of that happening are extremely small.”

Anyways, on to the code! As I codeviously stated, I have been using Paperclip in my Rails application. Using Paperclip interpolations we can easily create a hash path and make it available to our models. First, we need to create a file named “paperclip-md5-file-paths.rb” in config/initializers/.

If you’re using an older version of Paperclip, try something like this:

Paperclip.interpolates :hash_path do |attachment, style|
  # set the FINAL_POST_ID_BEFORE_MD5_HASH to be the ID of the existing latest attachment at the time of transition.
  return "content/#{attachment.instance.id}" if attachment.instance.id < FINAL_ID

  hash = Digest::MD5.hexdigest(attachment.instance.id.to_s + 'secret')
  hash_path = ''
 
  3.times { hash_path += '/' + hash.slice!(0..2) }
  hash_path[1..12] << ''
end

If you’re using a newer version of Paperclip, this may be more useufl:

Paperclip::Attachment.interpolations[:hash_path] = lambda do |attachment, style|
  return "content/#{attachment.instance.id}" if attachment.instance.id < FINAL_ID

  hash = Digest::MD5.hexdigest(attachment.instance.id.to_s + 'secret')
  hash_path = ''
 
  3.times { hash_path += '/' + hash.slice!(0..2) }
  hash_path[1..12] << ''
end

The first line of code determines if the current PID was before, or after our migration to the hashed file paths. This was necessary in order to support the codevious 32,000 files; you may or may not need something like this. The next block of code takes the MD5 hash of our current PID with an optinal secret word, then converts the result to a string. Finally, we iterate over the hash and slices it into three pieces. The result is our “cdd/01d/2cf” path.

Now we have a :hashed_path available to us…let’s do something with it! In your model, you can now access this new path like so:

has_attached_file :image,
  :url => "/content/:hash_path/:style/:basename.:extension",
  :path => ":rails_root/public/content/:hash_path/:style/:basename.:extension",

Why this isn’t a default configuration in Paperclip is beyond me. Enjoy!

Using a Rails collection partial with a counter

2 Replies

I was reworking an application today and noticed there was a portion of the code that was rendering a counter.

# what I saw
<% @wheels.each_with_index do |wheel,count| %>
  <%= render :partial => "wheel", :locals => { :wheel => wheel, :count => count} %>
<% end %>

# you might also be doing
<% count =0 %>
<% @wheels.each do |wheel| %>
  <%= render :partial => "wheel", :locals => { :wheel => wheel, :count => count} %>
  <%= count += 1 %>
<% end %>

Instead of using a block and rendering the partial each time, let’s try something different. We’ll use:

<%= render :partial => "wheel", :collection => @wheels %>

This will allow us to iterate through each item in our @wheels collection. Rails will make each element available to the wheel partial using the “wheel” variable. If our partial was “book” our variable would be “book” as well. If you want to change the name of the variable, you can do this:

<%= render :partial => "wheel", :collection => @wheels, :as => "dinosaur" %>

Now that we’re rendering a collection, we have a helper available to us inside of the partial that returns the index. For example:

# in our view
<%= render :partial => "wheel", :collection => @wheels %>

# in our wheel partial
# returns index
<%= wheel_count %>

# another example:
# in our view
<%= render :partial => "page", :collection => @pages %>

# our page partial
# returns index
<%= page_count %>

Multiple database connections with Rails

2 Replies

Recently a client requsted that I add a new admnistration panel to an existing Rails application. I thought to myself, “That sounds simple enough?”; welll, things are never as simple. In addition to this control panel, they wanted to use an existing MySQL database and its contents which, of course, did not follow the Rails naming conventions. Anyhow, after some research here is what I came up with.

Our first stop is in the database.yml file; the file should look something like this:

# database.yml
development:
  adapter: mysql
  encoding: utf8
  username: root
  password:
  database: example_development

This shouldn’t be a suprise to you. The database specified above is the default database that all of our Active Record models will use; if we want to add another connection we will need to start by speficing the connection. This connection can be another MySQL, PostgreSQL, or even an Oracle connection.

# database.yml
development:
  adapter: mysql
  encoding: utf8
  username: root
  password:
  database: rails_development

# here we specify new connection
pixel_development:
  adapter: mysql
  encoding: utf8
  username: legacy_username
  password: legacy_password
  database: legacy_database

Now that we have the connections specified, we need to move to our model; it is in here where we redirect the request to a non-standard connection. I have created a new model called “Pixel” which will interface with a database used for tracking pixels.

# app/models/pixel.rb
class Pixel < ActiveRecord::Base
end

Unless we specificy another database connection, Rails will use the default enviroments database; in this case “development”. Let’s change this by adding the following line:

# this will use our "pixel_development" database
class Pixel < ActiveRecord::Base
  establish_connection :pixel_development
end


# but let's make this more dynamic so we can use
# develpoment, production, and test databases
class Pixel < ActiveRecord::Base
  establish_connection :pixel_#{RAILS_ENV}
end

Now we should be good to go! Any requests to the Pixel model will connect to the second database. Say the table name is set in stone and cannot be modified? No problem.

# this will use our "pixel_development" database
class Pixel < ActiveRecord::Base
  establish_connection :pixel_#{RAILS_ENV}
  set_table_name "your_table_name_here"
end

Dynamically set a domain for a Rails asset host

7 Replies

I’ve been wanting to implement an asset host for my Rails app, looking over the API I saw that Rails’ asset_host only supported a single domain. I’m running multiple domains off one Rails app, so this would have been a problem. After a little research, I came up with a solution that would work for me.

I run this as a before_filter in my Application Controller.

# for my use, @site.domain returns the user's current
# domain. you should change this to whatever method
# you use to get the domain

class ApplicationController < ActionController::Base
before_filter :find_asset_host
 
private

  def find_asset_host
    ActionController::Base.asset_host = Proc.new { |source|
        asset_hosts = %w{icarus zeus aphrodite etc etc}
     
        # disable asset host for development
        if is_development?
          ""
        else
          "http://#{asset_hosts[rand(4)]}.#{@site.domain}"
        end
      }    
  end
 
end

Simple non-model check box properties

3 Replies

Need to pass a property that isn’t associated with a model? I did. It took me a few minutes, but the solution is codetty easy.

Instead of passing the instance variable your form is using, try this.

<% form_for [:admin, @post]} do |f| %>
  <%= f.error_messages %>
 
  <p>
    <%= f.label :title %>
    <%= f.text_field :title %>
  </p>

  <p>
    <%= label(:skip_queue, "Skip queue") %>
    <%= check_box("overrides", "skip_queue") %>
  </p>

  <p>
    <%= f.submit "Submit" %>
  </p>
<% end %>

You’d access the property of the check box like so:

# log
Processing Admin::PostsController#create (for 127.0.0.1 at 2009-03-06 03:03:01) [POST]
  Parameters: {"commit"=>"Update", "post"=>{"title"=>"Title", "overrides"=>{"skip_queue"=>"1"}}

# access the property
params[:overrides][:skip_queue]

Paperclip deletes images with empty file fields update with rails 2.3

Leave a reply

Bahhh! When you update a model that contains an empty file field in Rails 2.3 the attachment is deleted. It looks like empty file fields get passed as nil and then Paperclip gets angry.

Processing Admin::PostsController#update (for 127.0.0.1 at 2009-03-05 01:21:08) [PUT]
  Parameters: {"commit"=>"Update", "post"=>{"title"=>"Post Title", "image"=>nil}, "id"=>"124"}

There are two quick monkey patches that I’ve been using, the first, was splitting off attachments to their own method (edit_attachment)…but this quickly became a nuisance. A quicker work around is to remove the attachment field, in this case “image”, from the parameters.

# PUT /posts/1
# PUT /posts/1.xml
def update
  @post = Post.find(params[:id])

  # delete the offending parameter
  params[:post].delete(:image) if params[:post][:image].nil?

  respond_to do |format|
    if @post.update_attributes(params[:post])
      flash[:notice] = 'Post was successfully updated.'
      format.html { redirect_to(admin_post_url(@post)) }
    else
      format.html { render :action => "edit" }
    end
  end
end

Watermark your images in Rails

2 Replies

I’ve been working on a project that requires me to watermark all my images. Here’s a script I put together using rmagick. I plan on releasing a skeleton application that will accept uploaded files and watermark them.

Ideally, I’d like to use paperclip to handle uploading, and write a processor subclass to handle watermarking…unfortunately I’m running into some issues. If anybody wants to lend a hand, I have the public repository up at: http://github.com/ng/paperclip-watermarking-app/tree

require 'RMagick'

# load source and watermark images
src = Magick::ImageList.new("source-image.jpg").first
watermark = Magick::Image.read("watermark-image.png").first

# create a new, blank image with a height that is the sum of
# the source and watermark
dst = Image.new(src.columns, src.rows+watermark.rows) { self.background_color = 'white' }

# place the source image on the blank image
result = dst.composite(src, Magick::NorthGravity, Magick::OverCompositeOp)

# place the watermark over the codevious transformation
result = result.composite(watermark, Magick::SouthEastGravity, Magick::OverCompositeOp)

# write it out
result.write('watermarked-image.jpg')