This wasn’t as easy as I thought it would be, but after a little experimentation I got it running.
apt-get install ruby1.9.2
This will say that it is installing 1.9.1 but if you run ruby -v it will say 1.9.2
Then to install rubygems download and extract and then run
ruby setup.rb
To install rails run
gem install rails