Screencast: Deploying to AWS/EC2, Part 1
Errata
- Windows users!!: See special note regarding keypairs (Step 8) below. I have received reports that the keypair generation is awkward, flaky, or broken in ElasticFox, so I’ve provided some alternative instructions at the end of this page.
- In point 11 below, I talk about opening ports up for the “Basic” security group. I suggest that you open port 3000. This isn’t in the recording, but I’d like you to do it so that you can test a quick Rails app in development mode in the next screencast.
In this screencast, I’ll demonstrate the initial AWS/EC2/S3 signup, and will launch a server. The followup screencasts will show how to set up a system for Ruby on Rails, serving the application with the Passenger module for Apache (http://www.modrails.com/). Deployment will be conducted with Capistrano (http://www.capify.org/index.php/Capistrano).
The reason this screencast is password-protected is because, while I have tried to hide all personal details, there may be an occasional display of my address, phone number, etc. If I did miss a display of such information, I trust you not to share it with anyone. Thanks.
Here are the steps. Skim these steps before watching the screencast (link at bottom).
- It is not required that you deploy your final project to EC2! So if this looks too hard or uninteresting, you needn’t do it.
- After your final project proposal has been approved, your TA will give you a credit code from Amazon that is good for $100 worth of usage. A small instance that is left running continuously for a month costs about $75. Instance usage is charged by the hour — $0.10/hour for a small instance — so gauge your usage accordingly.NOTE: Critical! If you exceed your credit code grant, you start paying real money from your credit card. We are not responsible for this, so monitor your usage. When in doubt, shut servers down! Also, we not give you an additional credit code (there aren’t any extras). Don’t ask.NOTE: The instructions here do NOT create a “production quality” server for a business, but it’s good enough for a personal project. Also, the security is merely OK. For instance, we will allow ssh connections from anywhere to make it easier to get to your server.
- Sign up for AWS, EC2, and S3. You need to sign up for each of these, in this order.
http://aws.amazon.com
http://aws.amazon.com/ec2[In the screencast, I say something about signing up for S3: You don't have to do that separately anymore.] - Now we want to add the “credit code” from your TA. In the main AWS portal area (http://aws.amazon.com / “Your Account”), go to “Payment Method,” and add your code at the bottom.
- Next stop: Security Credentials.Make note of:
- Your AWS Access Key
- Your AWS Secret Access Key
- (I recommend copying them into a text file.)
- Install the ElasticFox add-on (http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609)
- When you first run Tools/Elasticfox, you will be prompted for credentials. Enter a name for your credentials, your AWS Access Key, and your AWS Secret Access Key.
- [Windows users! See bottom of the page for an alternative way to generate a keypair.] Click on the “Keypairs” tab. Click the green key icon to create a keypair, and then enter a name for the pair. You will now be asked to save the file somewhere: My recommendation for now is to save it to your desktop. This file contains your private key. NOTE: If you lose this file, you will not be able to log into your instance. Period. So don’t lose it! On Linux and Mac OS/X, change the permissions so no one else can read it (chmod 600 file).
- From the “Regions” drop-down, pick us-east-1
- Pick the “Security Groups” tab. Click the green plus. Create a group named “Basic.” Leave the radio button at the default (”Enable SSH and RDP for this Host?”) and then click “Create Group.”
- Now select HTTP protocol from the protocol dropdown, select “Network,” and enter “0.0.0.0/0″ and click Add. Do the same for ssh. If you want to be a bit more secure, you can constrain ssh to just the “Host” radio button, and click the button for “Get My Host Address”; then ssh will only work from your current network address. Finally, also open up port 3000; for this one, you can just click the radio for “Host” and click the button for “Get My Host Address” — this means that port 3000 will only be open to your current IP address.
- Click the “Images” tab. In the text box, enter: ami-1515f67c This is the stock AMI for Ubuntu 9.10 (see http://uec-images.ubuntu.com/releases/karmic/release/).
- Right click on the line for the image, and select “Launch Instance(s) of this AMI”
- For the KeyPair drop-down, pick the one you created above; in the Security Groups section, select the Group you created.
- Click Launch.
- Now choose the “Instances” tab. Eventually, your instance will be in the “Running” state (you can click that little refresh button to get ElasticFox to update the state of all of your instances).
- Now we’re going to create a terminal session to your server. Connect from the console by ssh-ing to a server with the “public DNS name.” To get this, right-click on the instance, and pick “Copy Public DNS name to clipboard.” Then go to the console, change your directory to where the private key file is, and type what’s below, pasting your public DNS name after the @ sign:
ssh -i id-jgn-demo ubuntu@ec2-174-129-56-164.compute-1.amazonaws.com
So in the case above, my private key is in the file id-jgn-demo, and the public DNS name is: ec2-174-129-56-164.compute-1.amazonaws.com, and I’m logging in as the ubuntu user.
- Play around.
- Log out (just press Control-D).
- Now terminate the instance (right-click and choose “Terminate”).
- Congratulations! You have created a running server in the Amazon cloud!
For Windows users, especially: Generating a keypair via the AWS Console
In Step 8 above, I tell you how to generate a keypair using Elasticfox. Apparently there are problems with this under Windows.
Here’s how to do the same thing with the AWS console.
- Go to http://aws.amazon.com/console/
- Sign in
- Click “keypairs” (left-side navigation pane)
- Click “Create Key Pair” (upper-left of “Key Pairs” pane)
- Give it a name and and click “create”
- This will initiate a download
- The downloaded file (something like key.pem) is your private key; you use it the same way as I recommend in step 17 above, except that you will say “-i key.pem”; if you like, you can rename this file so that it doesn’t have the “.pem” file type.
- Now if you go back to Elasticfox and select the “KeyPairs” tab, you should see a row for your new keypair. I would recommend using Elasticfox as above, though much of the functionality is available through the web-based console.
That should be it. Again, keep this private key in a safe place. If you lose it, you won’t be able to log in to instances created with that keypair.
When following this tutorial, when I am trying to login I get the following (This is from Mac, just as described):
Permission denied (publickey).
I even chmod'ed the file as you do in the drive.
@Jeff Ancel
Jeff, what command are you using to log in? Remember that the command is:
ssh -i SOMEFILE ubuntu@SOMEPUBLICDNSNAME
where SOMEFILE is the path to your private key file, and SOMEPUBLICDNSNAME is the name of the system.
Notice that the user is "ubuntu" not "root"
Yes, I am using that file. At some place at some other point in time (unrelated to this process), I saw another error pertaining to "World Writable Directory (some directory). I was thinking this might have something to do with it.
I am going to try it from my Mac Server this evening as it doesn't have the configurations my MBP does and see if I can't get it to work from there. Also, my MBP is going to be my test machine, will let you know when I have this baby up.
@Jeff Ancel
Remember that it is quite cheap to start up a new server -- from ElasticFox, it takes about 5 seconds. So you could also start up a new server and see if you can log into it.
err, mean my Mac Server is going to be my test machine :D
Ok, I will do that and see how it goes. I am not sure I can do it from the network here at work, so it will be this evening.
@Jeff Ancel
One more comment on all this: Don't be shy about terminating instances. The whole use of EC2 is predicated on the "cheap" creation/destroying of instances. Also remember that if you leave an instance running, that's $0.10/hour. It builds up.
Is there anything like a bill overview that let's you see your bill at all times?
To be honest: I don't get these pricing schemes and I am sure I understand something wrong here. If I chose the web app sample from their official calculator (http://calculator.s3.amazonaws.com/calc5.html) it tell me a monthly bill of $6600... what's that for??? (I don't know for the US but in Switzerland simple hosting space is about $30 a month).
@Gabriel Hase
A small instance running for a month is about $75. The bandwidth and storage costs will be negligible, and you won't need EBS, load balancing, etc.
An Amazon small instance is quite beefy. If you have a site that is truly just starting, then a VPS such as slicehost.com is a good way to go. The deployment screencasts here: http://e168f08.plugh.org/screencasts/ discuss slicehost (though it's Ruby 1.8.x and Rails 2.0.x). For what it's worth, I would never recommend "shared hosting" for a Rails app.
The big "win" for EC2 is the ability to bring up servers very rapidly, and then tear them down rapidly. Example: You're running an e-commerce app with two small instances running Rails, and a medium instance running a database. The shopping season hits, and you need two extra small instances for two weeks. With Amazon, that is a trivial "on demand" setup. With most VPS's, it is hard to rent servers for less than a month, and the setup can be harder.
Another top use case for Amazon is QA. You may have a development cycle where you only need QA for a few days a month. If so, you can bring up a complete parallel system to your production footprint. E.g., say your production footprint is 3 small servers ($225/month), and you only need QA for 8 hours/day for 3 days in one month. With Amazon, that additional short-term capacity is only about $8.
Sounds reasonable, thanks.
I just setup my instance and everything works. You mention on the second (and third) screencast that there will be an ami for the course with everything pre-built. Does it exist already? (the stock ami for ubuntu has nothing in it).
@Gabriel
Nope -- unfortunately I haven't saved a pre-built AMI yet.
However . . . It is pretty quick to just go through the steps and add everything yourself.
Finally, also open up port 3000; for this one, you can just click the radio for “Host” and click the button for “Get My Host Address” — this means that port 3000 will only be open to your current IP address
Above is a quote. Not sure I am perfectly clear on this direction. What I did from the dropdown is select other, and then I set a range, port 3000 to port 3000 and then clicked the Host button. Is this approximately right or did I miss something somewhere?
Here is the actual issue that I am still having. I am going to run through the instructions from a different computer.
jeff-ancels-macbook-pro-2:hurl jeffancel$ ssh -i id_hurl-cloud ubunto@ec2-174-129-175-111.compute-1.amazonaws.com
The authenticity of host 'ec2-174-129-175-111.compute-1.amazonaws.com (174.129.175.111)' can't be established.
RSA key fingerprint is dd:a2:5d:3a:c0:40:e6:84:96:c6:ab:38:27:e8:13:45.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-174-129-175-111.compute-1.amazonaws.com,174.129.175.111' (RSA) to the list of known hosts.
Permission denied (publickey).
@Jeff Ancel
For opening up 3000: I think what you're doing is correct.
Truly, I think if you try the ElasticFox UI, you will see the dialog box I'm talking about.
@Jeff Ancel
For "permission denied (publickey)":
Make SURE you're using the right private key file (argument for -i). It should be the one downloaded from Amazon.
Here is some more information once I -v the ssh to get a better stack.
jeff-ancels-macbook-pro-2:hurl jeffancel$ ssh -v id_hurl-cloud ubunto@ec2-174-129-175-111.compute-1.amazonaws.com
OpenSSH_5.1p1, OpenSSL 0.9.7l 28 Sep 2006
debug1: Reading configuration data /etc/ssh_config
debug1: Connecting to id_hurl-cloud [63.251.179.56] port 22.
debug1: connect to address 63.251.179.56 port 22: Connection refused
debug1: Connecting to id_hurl-cloud [206.112.100.132] port 22.
debug1: connect to address 206.112.100.132 port 22: Connection refused
ssh: connect to host id_hurl-cloud port 22: Connection refused
@Jeff Ancel
Ah.
Jeff: it's ubuntu
you have: ubunto
@john
I am using the file that I created in the "Key-Value" Pairing, downloaded from Elasticfox. Is this not accurate?
@john
Lol, that is what I thought. Wasn't sure what the message was telling me. Still perusing Amazon docs and google to try and fix the issue.
Your command is still wrong. You want:
Darn. Like this:
ssh -i id_hurl-cloud ubuntu@ec2-174-129-175-111.compute-1.amazonaws.com
@john
Yeah, I was never good at spelling. *Knocks head against wall*. I guesss that's not what I should be telling my English professor ;). I was more interested at some point (in the future I guess) as to finding out how/why you went from professor to professional technical guru. I am very interested in becoming a professor one day.
@Jeff Ancel
http://e168f09.plugh.org/discussions/discussion-general/comment-page-2/#comment-874