HowTo: EC2 for Poets

Last update: 4/14/2012; 10:49:36 AM.

Most people think they can't run a server, but servers aren't any more complicated than a laptop. The main difference is that a server is always on and always connected to the Internet.

EC2 for Poets is a tutorial that shows you how to set up a server in Amazon's "cloud." All you need is a net connection, credit card, and a basic understanding of how to use computers.

Initially, the goal for EC2 for Poets was to make cloud computing less mysterious by helping people get through the process of setting up a server on Amazon EC2. The newest version is more than an experiment, it's a platform for applications. We're starting with the RIver2 news aggregator, an app that reads RSS feeds you're subscribed to every ten minutes and posts the new items at the top of the list. It's also a podcatcher and a photo aggregator, supports realtime updating and OPML reading lists.

And there are more apps you can install after getting your river up and running. A simple linkblogging tool. A photo archiver. An outline-based document management and collaborative system.

Each app is an instrument, together they form a symphony. The theme: A distributed publishing system operated by its users. This is, imho, the holy grail of the Internet. The goal we're all marching towards. With EC2 for Poets, we're getting closer.

The key idea is "operated by its users." Engineers mystify what they do, as a form of job security. Eventually people figure it out. EC2 for Poets moves in that direction. For some people it will be the Aha that helps them see how the net works.

This is also, I hope, a foundation for journalism and computer science students to lead their respective professions to the news system of the future. And users who are empowered to solve problems for themselves instead of waiting for experts or gatekeepers to do it.

The first version of this tutorial was written in Berkeley, CA in March 2009 and was revised in January 2011, September 2011 and April 2012.

Dave Winer
New York, NY

PS: This howto was announced on March 21, 2009 with a 22-minute podcast. Even if you don't install the server, it might be useful to listen to the podcast to get an idea of why this may turn out to be important.

PPS: The title derives from a class that was offered at the University of Wisconsin when I was a grad student there called Computer Science for Poets.

Creating a server on Amazon EC2

There's this great service called EC2 run by Amazon, the same Amazon you can buy books and refrigerators from.

EC2 allows you to create your own server in Amazon's server farm located somewhere "up there" in the clouds. It could be in Montana or New Jersey or Europe. You don't have to know or care where the server is.

You also don't have to talk to anyone to create a server. And when you're done with it, just shut it off. You only pay for what you use. And it doesn't cost very much. If you run a server for 8 hours it costs about $1. Update: For some people it's free for a year!

If you've ever purchased anything at Amazon, you're all set. And if you haven't, all you need is a credit card.

I think any reasonably knowledgeable user could set up a server this way. You might have to scratch your head, or read the instructions two or three times to get past the hardest parts. But you don't have to be an math major to pull it off. That's why it's called EC2 for Poets.

For people who are new to computers, this head-scratching, stare-at-the-computer process is very much part of how computers work, for all of us. No matter how long you've been doing it. It's one of the fundamental lessons of science, I think -- anything worth learning is hard. Even the simplest things sometimes, amazingly, take a long time to penetrate our thick skulls. That's why learning can be so satisfying.

BTW, I don't own any Amazon stock, and I don't make any money when you set up a server. I just feel this is a basic skill any technical user should have. It will change the way you think about computing. I'm sure of it.

One more thing -- Ken Sedgwick blazed this trail. He wrote an initial howto which I expanded on in this piece. Thanks Ken!

Requirements

1. An account at Amazon.

2. A desktop or laptop computer, probably running Windows or Macintosh.

3. Anywhere from 10 minutes to an hour.

A Hello World server

In this experiment we'll start up a server that you can access over the Internet that will display Hello World and the current time on the server. You can refresh the page as many times as you like.

From there, I will show you how to install the River2 aggregator.

AWS stands for Amazon Web Services. It's an umbrella term that stands for all the services Amazon provides, like EC2. In order to set up your own server, you need to sign up for AWS.

To do so, go here: http://aws.amazon.com/ -- in the upper-right corner you'll see a yellow widget that says Sign Up Now. Click it. It'll ask for your email address and Amazon password, as if you were buying a book or shirt.

Unfortunately I can't tell you about the next page, since I can't see it. I have already signed up for AWS. Hopefully it's not too confusing. If you have trouble, grab a screen shot and post it in a comment and we'll try to help out.

After signing up for AWS, you must also sign up for S3 (the storage system) and EC2 (to create servers).

1. To sign up for S3 go here.

2. For EC2 go here.

Create a key-pair

Now we're going into the cloud. This is where you start to get god-like powers.

1. Open the Amazon EC2 Console Dashboard. You might want to bookmark this in your browser.

2. In the upper right corner is a panel called "My Resources" that lists the number of Running Instances, EBS Volumes, Key Pairs, Elastic IPs, etc. These are fancy names for computer stuff that aren't actually that complicated.

3. Click on Key Pairs. You should see an empty list.

4. Click on the Create Key Pair button. A dialog pops up asking for the name. Enter the name of a place, like Tahoe. Click the Create button.

5. Now something interesting happens. It wants to save a file on your computer. This file is a key that unlocks all the web services. Without this file you get nothing. With it you get everything. So you obviously want to put this in a safe place and take good care of it.

Although your Key Pair file has a fancy .pem extension, it's really just a text file. You can store it anywhere on your system. Later in this tutorial, you'll open it with a text editor, such as Notepad on Windows or TextEdit on the Mac.

Create a security group

A security group is like the firewall in your operating system or router. It says what kinds of Internet messages can pass through to your server, and what kinds will be rejected.

1. In the left margin of the dashboard, click on the Security Groups link.

2. Click on Create Security Group near the top of the screen.

3. A dialog appears, asking for a group name and description. You can call it whatever you want, I called mine "default" and described it as "Server firewall settings."

4. Click Create in the dialog.

5. You should see your security group in the list. Click on it.

6. In the lower portion of the screen you should see information about your security group, with two panes: Details and Inbound. Click on Inbound.

7. You should see a popup menu that includes a list of Internet protocols. Choose HTTP. Click the Add Rule button. An item is added to the right side of the panel. Do the same for HTTPS and RDP; allowing your server to receive secure HTTP requests, control the server using Remote Desktop Connection on the Mac or Windows.

8. When you're done, click the Apply Rule Changes button.

Launch your server

Now it's time to create your virtual machine and launch it.

1. In the left margin of the dashboard, click on the AMIs link. You should see a popup menu in the upper left corner of the panel. Choose Public Images from the popup. It might take a few seconds for the list to appear (there are a lot of public images).

2. Type ec2ForPoets21 in the little search box near the top of the screen. A single item should show up in the list. Click on the checkbox next to its name.

3. Click the Launch button near the top of the screen.

4. A big dialog appears.

5. Set Number of Instances to 1 (it should be the default).

6. Change the Instance Type to Micro (t1.micro, 613 MB). Leave the Availability Zone choice on No Preference.

7. Click Continue to get to the next dialog. You can accept the defaults. Click Continue.

8. This is where you give your instance a name. It's like naming a cat. Pick something fun and memorable. (I named mine Einstein.)

9. Click Continue to get to the next dialog, where you choose the Key Pair. Click the radio button next to Choose from your existing Key Pairs, and then choose the one you created earlier in the popup menu. (We called our example Tahoe.)

10. Click Continue to get to the next dialog. Click the radio button next to Choose one or more of your existing Security Groups, and then select the default Security Group.

11. The final dialog in the sequence appears, summarizing the choices you made. If everything looks okay, click the Launch button at the bottom of the dialog. Pause for a moment to consider your accomplishment. You are now a Cloud Computing Expert.

12. In the left margin of the dashboard, click Instances. You should see a single entry whose status is "starting." We're now waiting for it to change to "running." This could take as much as 10 or 15 minutes, depending on how busy the angels and elves at Amazon are.

Notes...

Later if you want to go "shopping" for other AMIs, Amazon maintains a directory that's a bit like shopping for iPods or books, of popular AMIs. Too bad they don't offer the same reviewing options as they do for consumer products. (Consider this a feature request.)

In putting together the sample server for this tutorial, I started with Amazon's Basic Microsoft Windows Server 2003 AMI with none of the frills.

Find or download Remote Desktop Connection

Remote Desktop Connection, or RDC, is an application for Windows or Mac that allows you to control a remote server like the one you just launched.

If you're using a Windows machine, RDC is in the Start Menu, Programs, Accessories.

If you're using a Mac, you can download the app from Microsoft.

Update: An even better choice for Mac users is the open source CoRD app. It's better at managing more than one server at a time, and just plain better than the Microsoft app, which is showing clear signs of age. Plus you don't have to install Rosetta to use CoRD.

Wake the server up

Wait as much as 15 or 20 minutes for the status of the server in the Instances panel to change from "starting" to "running." Now we just have to kick it in the butt to get the server software launched. You do that by logging in the first time.

1. First, locate the Key-Pair file (mine is called Tahoe), open it with a text editor (Notepad on Windows, TextEdit on the Mac). Select-All. Copy. Close the file.

2. In the left margin of the dashboard, click on the Instances link.

3. Right-click on the instance you just launched. A popup menu appears. From the menu, choose Get Windows Password.

4. A big dialog appears explaining what it means to decrypt your password. Click in the "Private Key" box and paste the text you copied in Step 1. Click the Decrypt Password button. It takes a few seconds, then...

5. A dialog appears with the server password. Leave it open so you can refer to it in the next steps. (If you know how, you might want to take a screen shot of this dialog.)

6a. Launch the Remote Desktop Connection app from the previous section. Copy the address of the computer to the clipboard (in my screen shot it's ec2-72-44-33-187.compute-1.amazonaws.com). Paste it into the dialog that opens in the RDC app. Click on the wedge next to the word Options in the RDC window. Where it calls for the User name, enter Administrator. Then Copy the Decrypted Password from the dashboard, and paste it into the Password part of the RDC app. Click the big Connect button in the lower-right corner of the RDC window.

6b. If you're using CoRD on a Mac, instead of Remote Desktop Connection, the process is slightly different. In the lower left corner of the app window you'll see a wedge, the letter i and a big plus sign. Click on the plus sign. A dialog appears. Enter a name for the server, I like to name mine after football players or soft drinks, but you could just call it My First EC2 Server. Then enter the address as in step 6a. The username is Administrator, and the password comes from step 5. Click the Save password checkbox so you don't have to enter the password every time you connect. Below that, you can determine the size of the display, and some very nice advanced options that you can worry about later (accept the defaults for now). Close the Inspector and double-click on the name of the server in the left margin of the big window.

7. If all goes well, you should see your server running in a window on your desktop. But don't be fooled, this virtual computer exists in the cloud, not here or where you are. It's somewhere "up there."

A picture named poetsStartupWindowsSmall.gif

8. Three apps are visible: Firefox, the OPML Editor and the Windows Task Manager. OPML is, in addition to be an outliner, a web server. In the next section we'll access this server in a web browser on your desktop machine. You can close the RDC window, or leave it open, if you want to explore. It's just a Windows machine.

Find the admin password

We're almost there!

There's one more password to deal with, the one that allows you to access the controls of the OPML server software that's running on the machine.

1. You should see a folder on your server's desktop called OPML. Open it.

2. Inside the folder is a file called adminPassword.txt. Open it.

3. You should see a string of random-looking characters. Select. Copy. And paste it somewhere where you won't lose it.

You're going to need this password in the next step.

Visit your server

1. In the left margin of the dashboard, click on the Instances link.

2. Click on your instance. (There probably is just one in the list.)

3. Look in the lower portion of the window for a field named Public DNS. Copy it to the clipboard.

4. Open a web browser, paste into the address bar and press Return. You should see a sign-in screen that asks for a username and password. The username part is easy, it's admin. The password is easy too. It's the one you found in the previous section, from the adminPassword.txt file.

5. Once you enter the username and password you should see a screen that looks something like this.

The Hello World app

At the beginning of the tutorial I promised you would see the Hello World app running on your new server.

1. Add "helloWorld" at the end of the URL in the previous example and press Return.

2. You should see Hello World, today's date, the time, the IP address of the server, and a counter that starts at 0, and increments every time you refresh the page, which you should do a few times to prove that it's a dynamic page.

I know it sounded more exciting a few minutes ago. Because now you're a server dude or dudess, and this kind of stuff doesn't impress us really.

Comments

You now have a server running. It's a customized version of the simplest Windows image on Amazon. I added two open source programs: 1. The OPML Editor (web server). 2. Firefox (browser). It's a fully functional server, can run anything any of these programs can run.

When this package was put together the operating system and the included apps were all fully updated. If it's been months or years since it shipped, you may need to do some updating to get current. All three components will check automatically to see if they need updates.

Poke around, look in folders, see what's there. I tried to make it interesting and predictable. You can visit websites and install software. It's your server to do with as you please.

The OPML Editor comes with no apps pre-installed, but they are easy to install. In the next section we'll install River2 which is a feed reader that shows you the news in reverse-chronologic order.

Or it may make sense to shut off the server right away. The important thing is that now EC2 is not a mystery. Its purpose is to give you access to ordinary computers in a place where they're easy to get at through the Internet. The easier it gets, for more people, the more interesting things we'll be able to do, together.

Installing River2

River2 is an application that runs in the OPML server environment. It's a river-of-news aggregator. That means it reads a set of feeds that you tell it to, periodically, and shows you the new stuff in reverse-chronologic order. It's the application that inspired Twitter and Facebook, and it's still an incredibly useful way to get the news.

It's build on a standard format for distributing news that you might have heard of called RSS, which stands for Really Simple Syndication.

A lot of people use Google Reader to get the news from their favorite RSS feeds. River2 is another way to do that. And because it runs on your own server you completely control it. That's very different from the way Facebook works, for example. They decide what news you should see. Here, with River2, you do all the deciding. The software just does what you tell it to do. Exactly what you tell it, with no improvising and no advertising.

We've worked really hard to make it very simple to install River2. Here's how you do it.

1. On your server, in Firefox, go to this address: http://127.0.0.1/opmlEditor/catalog. This takes you to the Tool Catalog page, which lists all the apps that are available to run in the server environment.

2. Look for River2 and click the Install link in the right column.

3. A dialog should appear asking if you want to install it. Click OK.

4. Shortly another dialog confirms that it has been installed.

5. Click OK.

6. A web page appears, saying it worked, and gives you some ideas where to go next and what to do.

Accessing the River2 app from your desktop

From here, you probably want to use the app on your desktop computer. To do so...

1. On your desktop, enter the address of your server followed by /river2/. Here's what that address looks like on my machine.

2. You may get a login screen, if so, enter admin for the username, and enter the password found in the earlier part of the tutorial (it's in adminPassword.txt in the OPML folder on your server).

Where's the river?

There was a great commercial for Wendy's hamburgers a long time ago with a foxy grandma asking (about the competition) where's the beef? In this case, the question is different -- where is the river?

Here's where it is, on my server:

http://ec2-23-22-6-155.compute-1.amazonaws.com/rivers/admin/

It'll be in the same place on your server, except the name of the server will be different.

You don't need to log on to view that page. So you can give the address to your friends and colleagues at work, school, playground, senior center, etc. If you do that, you'll become their news service, and all you had to do was subscribe to some feeds.

There's a lot more to cover in River2-Land and more docs to write, for sure -- but that's the beginning. You now are one of the very lucky few who not only have your own server in the sky but you also have a river up there too! Be sure to carry your umbrella.

How to turn off the server

As Amazon makes very clear, until you shut off the server, they're going to charge $0.125 per hour. So if you're done, here's how to turn it off.

1. From the dashboard, click on Instances. You should see one item, the instance we just launched.

2. Click on the checkbox in the leftmost column.

3. Right-click it, and choose Stop from the popup menu. This causes the server to go to sleep, it's as if you closed the lid on a Windows laptop. You can turn it back on by choosing Start from the same menu. Both these operations are quite quick, they take a few seconds each.

4. If you know you never want to run this instance again, choose Terminate from the popup menu. This shuts down the server, and throws away the instance permanently. It's like taking your PC out to the garbage dump and driving away.

Support and community

Please keep comments here to support issues. I'm sure you can find other places to post your political and personal opinions.

We have a mail list for people who are running servers. It's for support and for discussing new features. The EC2 For Poets server is actually turning into a community. We're doing some very interesting stuff.

blog comments powered by Disqus

Last udpate: 4/14/2012; 10:49:36 AM.

Jan	MAY	Oct
	07
2011	2012	2013