Queue failed Solr calls with Delayed::Job

We are using websolr.com from one of our Heroku apps and from time to time we see connection errors. This is disruptive in the UI. One option was to move all of the Solr calls to DJ tasks, but I didn’t want to have a ton of stuff in the DJ queue. The other option was to make a variation of SilentFailSessionProxy and have it queue up the jobs we cared about, specifically index and remove and raise the others.

Here is the current version of this.

require 'sunspot/session_proxy/silent_fail_session_proxy'

module Sunspot
  module SessionProxy
    class QueueFailureSessionProxy < Sunspot::SessionProxy::SilentFailSessionProxy
      QUEUE_METHODS = [:index!, :index, :remove!, :remove]

      def rescued_exception(method_name, ex, klass)
        raise ex unless ::Rails.env.production?

        if klass && QUEUE_METHODS.include?(method_name)
          HoptoadNotifier.notify(
            :error_class => "Solr Exception",
            :error_message => "Solr Exception: #{ex.message}",
            :parameters => { :model => klass, :method => method_name }
          ) if defined?(HoptoadNotifier)

          klass.delay.index! if method_name.to_s.match('index')
          klass.delay.remove_from_index! if method_name.to_s.match('remove')
        else
          raise ex
        end
      end

      SUPPORTED_METHODS.each do |method|
        module_eval(<<-RUBY)
          def #{method}(*args, &block)
            begin
              search_session.#{method}(*args, &block)
            rescue => ex
              self.rescued_exception(:#{method}, ex, args.first)
            end
          end
        RUBY
      end

    end
  end
end

Sunspot.session = Sunspot::SessionProxy::QueueFailureSessionProxy.new(Sunspot.session)

How to install private gems on heroku

Scenario: I have to setup a staging app on Heroku. Sounds pretty simple, should take an hour tops, right? Think again.

Mix in a gem from a private git repo and it gets messy. If the gem isn’t installed via version but with a commit ref you get even more fun.

Setup the app on Heroku as you normally would, then try to deploy. Everything looks good, until it tries to fetch the private gem. Then

git push heroku staging:master
-----> Heroku receiving push
-----> Rails app detected
-----> Detected Rails is not set to serve static_assets
       Installing rails3_serve_static_assets... done
-----> Gemfile detected, running Bundler version 1.0.3
       Unresolved dependencies detected; Installing...
...

<tries to install private gem>

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

Hrm… Poke around the google some and find I’m not the only one. See references to an ~/.ssh/config that looks something like

Host heroku.com
  HostName heroku.com
  User git
  ForwardAgent yes
  IdentitiesOnly yes

Sounds reasonable. Nope, no luck. More time with google and it would seem that Heroku doesn’t pass your keys out the other side. Not sure if they have a specific reason for it or it’s just the default sshd_config.

Ok, that’s not going to work. Lets try dropping the gem in vendor/cache and see if that helps. Oh, there isn’t an actual gem to cache. Lets grab the one in bundler. Nope that wont work either.

Off to build the gem so I can place it in vendor/cache.

cd ../my_gem
rake build
cp pkg/my_gem-0.0.2.gem ../rails_app/vendor/cache
cd ../rails_app
bundle install

WTF!? Oh, it saw one gem in the cache dir, I must want them all in there, right?? No, no I don’t.

Ok, well let’s place the gem in the app in a place I can get to it with bundler but it wont randomly trigger some other unwanted behavior.

gem install ../my_gem/pkg/my_gem-0.0.2.gem
gem unpack my_gem -v 0.0.2 --target vendor/private_gems

Ok now I update my Gemfile to look like

gem 'my_gem',
  :path => 'vendor/private_gems/my_gem-0.0.2'

So far it seems happy.

Though I have seen some really strange issues with it randomly Throwing the Gemfile.lock - Heroku has a read-only filesystem error which would make perfect sense other than the fact that the error came from running heroku rake db:migrate right after a successful push.

New home, new engine

If you are reading this the site has moved to Heroku and is now powered by JABE in addition to the info on github, more info on JABE coming soon here.

Snow Leopard, Passenger 3, PassengerPane and Stock Apache

I finally decided to start using Passenger and the stock Apache in OS X Snow Leopard.

Here are the steps I followed, from memory, so if they are missing anything please let me know.

install PassengerPane

rvm install ree
rvm ree
rvm gemset create APP_NAME
gem install passenger
rvmsudo passenger-install-apache2-module

Add an .rvmrc file

rvm --rvmrc --create ree@APP_NAME

Find the gems info

rvm ree@APP_NAME
rvm info | grep GEM
  GEM_HOME:     "/Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@APP_NAME"
  GEM_PATH:     "/Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@APP_NAME:/Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@global"

add those to the /etc/apache2/passenger_pane_vhosts/APP_NAME.local.conf

SetEnv GEM_HOME /Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@showcase
SetEnv GEM_PATH /Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@showcase:/Users/mmoen/.rvm/gems/ree-1.8.7-2010.02@global

Make sure the envs are setup in /etc/apache2/passenger_pane_vhosts/APP_NAME.local.conf

RailsEnv development
RackEnv development

If you make changes to routing or other places that don’t autoload, you will need to restart the instance

touch tmp/restart.txt

Known issues:

If you are running a Rails 2.x app this way you will have to delete config.ru before starting or restarting the app. Otherwise it messes up Rails.env. And as always, feel free to tell me I’m way off or if you have any input, please share it.

[edit] Update .rvmrc creation per Wayne’s suggestion

Ack.tmBundle

If you are a Textmate user and you were tired of Find in Project being a dog, so you started using Grep in Project and even it was getting slow. You absolutely owe it to yourself to check out Ack in Project. It is ridiculously fast.