Screencast: Deploying to AWS/EC2 (Capistrano), Part 3

November 24th, 2009 Leave a comment Go to comments

[Draft!]

I should say a word or two about why deploying is so annoying.

The big problem with deploying is that even if we do it every other day or so, we do it infrequently. Because we do it infrequently, we don’t really spend a lot of time refining it. We get it to work, and forget about it, hoping that our deployment scripts will work the next time. When the deployment scripts break, we typically go into “open book” mode, spending a lot of time with Google.

Like a lot of things in Rails, when Capistrano came out it was somewhat ahead of the curve. Now it’s somewhat behind the curve, but is still something of a standard. Capistrano is not getting a lot of active development, so keep your eye for a new deployment system. One that has some mindshare these days is Vlad the Deployer (http://rubyhitsquad.com/Vlad_the_Deployer.html).

So . . . the following recipe should work. But deployment scripts break. Be prepared for some scrounging to get this to work if you find you have to alter some aspect of it to your own requirements.

A few assumptions: You will need to be able to ssh to your remote host, so review that from the last couple of screencasts. You will need to remember how to use sudo. You will need to know the name of your application and the directory in which it will live — in the example below, we’re going to use the “ajax” example application, and called it “ajax-capified.”

  1. Install the Capitrano gem, and “capify” your application:
    gem install capistrano
    cd your/app/path
    capify .
    

    Note that the last command is “capify” followed by a space, followed by a period, which means: Create the Capfile and config/deploy.rb files for the project in the current directory (.).

  2. Once this is all set up, the application is going to live in /home/ubuntu/sites/ajax-capified/current. Let’s update the Passenger (see Part 2) setup now so that once everything is in place, Passenger will find the code. So ssh over, do “sudo bash”, and edit /etc/apache2/sites-available/default, and change the two occurrences of /home/ubuntu/myapp/public to /home/ubuntu/sites/ajax-capified/current/public — after doing this, mine looks like this:
    <VirtualHost *:80>
      ServerName ec2-174-129-169-212.compute-1.amazonaws.com
      DocumentRoot /home/ubuntu/sites/ajax-capified/current/public
      <Directory /home/ubuntu/sites/ajax-capified/current/public>
        AllowOverride all
        Options -MultiViews
      </Directory>
    </VirtualHost>
    

    Also, you should restart Apache right now; after this, we will restart
    the application in a different manner:

    /etc/init.d/apache2 restart
    
  3. Now, we need to make some changes to the default config/deploy.rb file. We’re going to need to go through this line-by-line. Here’s what ours looks like. You should study the comments, and compare this file to the default one that is created by the capify command.
    # Used by logging
    set :application, 'ajax-capified'
    
    # The directory on the remote system that will contain release versions
    # and the "current" release (which is a symlink to the last good release).
    # NOTE: This should be an absolute path (beginning with /).
    set :deploy_to, '/home/ubuntu/sites/ajax-capified'
    
    # If your code is checked into a repository for source control . . .
    #   set :repository,  "set your repository location here"
    #   set :scm, :subversion
    # In this example, we're going to skip source control.
    set :repository, '.'
    set :scm, :none
    set :deploy_via, :copy
    
    # Log in as this user for setup and deployment
    set :user, 'ubuntu'
    
    # We want to avoid using the root user
    set :use_sudo, false
    
    # Our single server will serve all of the available roles; Capistrano
    # is designed to allow for having the different roles be served by
    # different computers.
    server 'ec2-174-129-169-212.compute-1.amazonaws.com', :web, :app, :db, :primary => true
    
    # Specify the private key you will use
    ssh_options[:keys] = [File.join(ENV['HOME'], 'Desktop', 'e168', 'id-jgn-demo')]
    
    # For Passenger
    namespace :deploy do
     task :start do
     end
     task :stop do
     end
     desc "Restart the application in Passenger"
     task :restart, :roles => :app, :except => { :no_release => true } do
     run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
     end
    end
    
  4. For production, we’re going to switch the database to MySQL. If you did what I suggested in prior screencasts, and didn’t define a password for the root MySQL user, all you need to do is change the “production” block in config/database.yml to . . .
    production:
     adapter: mysql
     database: ajax_capified_production
     pool: 5
    
  5. Let’s create that database. ssh to your system, and . . .
    mysqladmin -u root create ajax_capified_production
    
  6. Now we’ll do cap deploy:setup — this creates the directory structure.
    cap deploy:setup
    

    At this point, you might want to ssh over to your system and see what’s there. What you should find is a “sites” directory, and inside of that, a directory named after our app (ajax-capified). Then within that, directories called releases/ and shared/. Your code will be copied into releases/ under a timestamped directory name; when the deploy is finished, a directory under ajax-capified named current/ will be symlinked to the last release that successfully deploy.

  7. and . . . finally . . .
    cap deploy:migrations
    

    This is the most important command in your arsenal, because it will run your migrations
    and deploy the latest code. Once you’re set up, this is the command you’ll
    run over and over: Make changes to your code, add migrations, etc.; test
    them in development mode, and run tests. Then when everything’s right, deploy
    to product with cap deploy:migrations

  8. That’s it! Next screencast: DNS.
  1. Ken
    December 5th, 2009 at 22:10 | #1

    Isn't step 1 carried out as root?

  2. December 6th, 2009 at 07:26 | #2

    @Ken

    No. This step is done on your local system. If you are using the "ruby_switcher.sh" as I have recommended, then your gems are not installed as root.

    If, however, you have installed a system-wide Ruby on your local system, and it's Mac OS/X or Linux, you would need to be root.

  3. Gabriel Hase
    December 10th, 2009 at 09:12 | #3

    Yeahi it worked! My project got live:
    http://ec2-174-129-155-11.compute-1.amazonaws.com/

    Will the screencast for DNS setup still come? (it's especially important since I need to generate a Google API key for a fixed domain).
    Also: Is there a possiblity to make a little screencast (or note on this page) about how to setup capistrano with subversion (or do you have a great webpage/cast that explains that)?

  4. December 10th, 2009 at 09:28 | #4

    @Gabriel Hase

    I will likely get the DNS info done over the weekend. The short version is that you can sign up at dyndns.com for free, and a subdomain on one of their domains (e.g., gabriel.dynalias.com) to your Amazon IP. Another option which I probably won't discuss is buying your own domain and setting up DNS for that.

    What do you mean by "Google API key for a fixed domain"? Do you mean obtaining an Elastic IP?

    As for Capistrano and Subversion: I am not sure I will have time for that anytime soon. Note that you can most certainly maintain your own local code in subversion, and STILL use the "no scm" option to move your code to the remote system. In fact, there are some good reasons to do this: It will keep you from being tempted to update single files in place, which to some is somewhat risky.

    The only real trick with Capistrano, Subversion, and EC2 is that the default EC2 setup uses only key authorization (no passwords). So you have to set up the remote system so that IT can check out from your remote Subversion. There are a number of ways to do this, such as using the "agent forwarding" capability in ssh; or you could just copy your key to the remote system so that it is picked up during the checkout. There are also some considerations based on your access mode into your svn repos: Is your svn repos URL like "svn+ssh://....." or "http://....."?

  5. Gabriel Hase
    December 10th, 2009 at 09:43 | #5

    So is it even easier when we already have a dns name? (I already bought tourslikemine.com with godaddy). I am still wondering if it isn't a problem since the ip's of the instances are changing (they are, right?).

    The Google Api key is generated for a domain. E.g. I can say generate a key for 127.0.0.1 which will work on all subsequent addresses (e.g. 127.0.0.1/tours). What I want is to generate the key for http://www.tourslikemine.com. But this I can only do once I setup my godaddy address with my amazon ec2 instance.

    The copy deployment works fine for now, only problem is that the tarbal is 80MB and it takes a while do deploy (i.e. upload). I guess with subversion set up it would only send the change delta (I already have my project under subversion control). My svn server uses svn:// addresses.

  6. December 10th, 2009 at 09:54 | #6

    @Gabriel Hase

    For changing IP's: Check the EC2 docs for "Elastic IP." You can get one of these through the Firefox ElasticFox dashboard. It will give you a non-changing IP. As long as it's in use, it's free. If it's not assigned to an instance, they charge per day (like $0.10/day -- I can't remember the detail).

    Then go into your GoDaddy DNS settings, and point the domain to the Elastic IP.

    This should resolve your issues with the Google API key.

    No -- It doesn't just get the deltas. Every time you deploy with Capistrano, it creates a whole new release directory with a fresh copy. If your remote system can check out from SVN, though, all of the bandwidth is going between your EC2 system and your repos, so your own machine isn't tied up.

    80MB, eh? What's in there?

  7. Gabriel Hase
    December 10th, 2009 at 09:58 | #7

    @john

    john :
    80MB, eh? What's in there?

    :) flat files with a lot of country and city names (acutally every city in the world with a population bigger than 15000 people). Sums up...

  8. December 10th, 2009 at 10:01 | #8

    @Gabriel Hase

    Can't you dump that into the database? Then you wouldn't have to move it over all the time.

  9. Gabriel Hase
    December 10th, 2009 at 10:19 | #9

    Yes, I guess on the long run, that's what I am going to do. But for now I have the DB initialisation with the city names in a data migration so it will work for the TA to setup a fresh DB.

  10. December 11th, 2009 at 00:30 | #10

    @Gabriel Hase
    here is what I did. To get this to work on my Mac Server, I changed my CNAME in my Domains panel at GoDaddy. I actually used DynDNS.com to do it with my home server (bought a Mac Mini Server just for this class). In GoDaddy, you want to sign in -> Domains -> CLICK the Domain. At the next page, you will see a link "Total DNS Control", Follow that. Click the edit button for @ and put your AWS Server address in where the IP address (most Likely) is, in the "Points To" column. Then change the CNAME for www, and change the "Points To" to @ if not alredy done.

  11. December 11th, 2009 at 00:33 | #11

    If I have my site up, is it ok to show it off a little bit? I deployed it to my personal mac server (only 2MB Upstream, but should work for beta.) Let me know, I will post the link. Still a lot of work for production ready, but it is almost worthy of a project.

  12. December 11th, 2009 at 09:09 | #12

    @Jeff Ancel

    Jeff -- Thanks for the steps. Do you really need to use a CNAME, though? Can't you just point the A record right to your IP?

  13. December 11th, 2009 at 09:09 | #13

    @Jeff Ancel

    Jeff, yes, please do share. I will add a "demos" discussion page.

  14. December 15th, 2009 at 00:08 | #14

    @john
    I think you are right. The reason I did CNAME was because I was setting up a sub-domain as to not interrupt my service since it was already in use. If I change the ARecord, then my host would no longer be hosting the current site at the normal url. I will do this once I have the site up and current data migrated over. It is going to be an interesting go at migration, going to try and do it with no downtime, though after the class.

    One more question:
    With .net MVC it was really easy to have a catch all error page (and different ones of sorts for different status messages) set up in their configuration file. I would ideally want no one to know what language this is written in. I feel disappointed when I go to a ruby site and get the Red Basic Error Screen of death. Can you point me at a good resource to figure this out encompassing every error possible?

  15. December 15th, 2009 at 00:10 | #15

    A quick example of the above...

    [link]http://hurl.me/some/dome/jibberish[/link]

    and...

    [link]http://beta.hurl.me/some/dome/jibberish[/link]

  16. Ken Busch
    December 30th, 2009 at 22:54 | #16

    Step 6 fails on Windows. I notice this bug has been reported and there is supposedly a work-around. The error message reads:
    C:/Ruby19/lib/ruby/gems/1.9.1/gems/net-ssh-2.0.16/lib/net/ssh/authentication/pageant.rb:24:in `': uninitialized constant DL::Importable (NameError)

    Reference: http://www.mail-archive.com/capistrano@googlegroups.com/msg07438.html

  17. February 7th, 2010 at 16:49 | #17

    This worked out very wellfor me. The only issues I had with the deploy.rb files were (note, I am not sure if this was specifically to do with my using GIThub or not):

    I also had to add these two lines here (deploy.rb), then it worked.
    ssh_options[:port] = 22
    ssh_options[:forward_agent] = true

  1. No trackbacks yet.