Architecting AWS applications: A How To

After the EBS related outage last week, Amazon is proactively starting to educate their customers about architecting for the cloud. Great first presentation in a series.

Services are inherently fault tolerant
Ha systems: az, elastic ips, ebs
Budget vs. time to recovery

Aws hierarchy

  • Region
  • Availability zone
  • Ec2 instance

Storage

  • Ephemeral storage (instance attached. Lives/dies with instance)
  • Ebs (tied to s3)

Ec2

  • Amis (configured instance snapshots)
    • Can drive autoscaling groups
  • Autoscaling
    • Cloudwatch – react to changing conditions. Eg. # of requests per second
    • Or keep fixed # of instances alive (if one goes away, autoscaling will replace it)
  • Elastic load balancing
    • Spread traffic across instances
    • Remove unhealthy instances from rotation

Availability zones

  • Distinct locations
  • Recommend spread app across 2 or more zones
  • Each az should get a full stack
    • App continues to function if 1 az becomes unreachable

Design for failure

  • No single point of failure
  • Assume everything fails
  • Design a recovery process
  • High availability is expensive (as a business, decide how much you need)

Architecture

  • Use elastic ip
  • Multiple AZs
  • Replicate data across AZs
  • Realtime monitoring (eg. Cloudwatch)
  • Use ebs (take snapshots and send to s3)

Bootstrapping instances

  • Who am i and what is my role?  (asks a new server that spins up)

Enable dynamic configuration

  • Use autoscaling
  • Use elastic load balancing on each tier

System level abstractions (high level descriptions of roles in system)

  • Eg. App server: Nginx, Thin, Ruby 1.9.2  Machine
  • Use puppet or chef to prepare (cloudformation)
  • Build entire stack from template

Netflix chaos monkey  They test resilience of production system by randomly killing worker processes. What happens?

 

Author name:
Discussion:
No Comments

Quote

What I’ve learned about software engineering: 1) Listen to smart people 2) Prefer simple to clever 3) Have no ego 4) Shut the fuck up.

Ted Nyman

Author name:
Discussion:
No Comments

Hello, Android

Started my very first android project today. The platform docs so far has been great. (Lots to read!) Getting a dev environment setup setup was pretty straightforward too. Tomorrow I’ll go through the examples + sample code to get a feel for how application code is organized.

Exciting!

Author name:
Discussion:
2 Comments

Quote

Readability of code is now my first priority. It’s more important than being fast, almost as important as being correct, but I think being readable is actually the most likely way of making it correct.

- Doug Crockford, Coders at Work

Author name:
Discussion:
No Comments

Latency and the Web

Great post by Philip Tellis (link) on latency and how to minimize it in general. The mnemonic he uses is one I’ve not seen before but very much like… C-P-P. That is, latency is reduced by caching, parallelizing and predicting.

Caching moves the resources you’re serving closer to the clients requesting them which makes for shorter round trip times all things being equal. (Speed of light and such things.) Predicting means you’re speculatively pulling more data than you need  at any given moment in the hopes of avoiding future work. Parallelization is the somewhat controversial one.

It seems to me the movement that is page speed optimization has, for the last 2 years or so, been encouraging people to package and deliver things like js/css/images as single files to avoid making extra requests. And we have done. Philip’s research however suggests putting everything in a single file may actually hurt full page render times. In many cases, downloading multiple chunks actually bumps page speed. Parallelism.

I probably shouldn’t be too surprised. :)

Author name:
Discussion:
1 Comment

Quote

“Life is either a daring adventure or nothing. Security does not exist in nature, nor do the children of men as a whole experience it. Avoiding danger is no safer in the long run than exposure.” – Helen Keller

Author name:
Discussion:
No Comments

Best Youtube comment ever

As is usually the case on Youtube, wilfmartin was responding to an idiot…

ICrysisl: why didn’t those fools in the background shut up??

wilfmartin: “@lCrysisI because ICrysisl this was recorded at an outdoor concert hosted bt Luciano Pavarotti in Modena, Italy on 9th June 1998. There were 30,000 people there and the song had just got to No1 across Europe. I know this because I was there – it was my birthday! So I guess I was one of those fools in the background! The flautist incidentally is Andrea Corr of The Corrs.”

(video)

Author name:
Discussion:
No Comments

Quote

“If you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.” ~ Albert Einstein

Author name:
Discussion:
No Comments

Interview: Jeff Bezos with Charlie Rose

Highlights of the interview for me: the new kindle – love it, Amazon the company, philanthropy and the notion that lots of successful companies today are shifting gears towards a bigger focus on core values and designing+building kick-ass products.

Turns out that’s a big paradigm shift. It used to be that companies could – and would – do really well by making shoddy products, shouting really loudly about them, then selling them like gangbusters – 70% of operating budgets went to marketing, 30% to the actual product itself. (eg. Microsoft, Dell.) Not as easy to do anymore according to Jeff and you’d be hard pressed to disagree with him. Much more important now is building the best product you can and delighting customers with them. (eg. Apple, 37Signals, Zappos.)

Let happy customers get the word out for you. (Mostly.)

(video)

Author name:
Discussion:
No Comments

Last.fm robots.txt

User-Agent: *
Disallow: /music?
Disallow: /widgets/radio?
Disallow: /show_ads.php

Disallow: /affiliate/
Disallow: /affiliate_redirect.php
Disallow: /affiliate_sendto.php
Disallow: /affiliatelink.php
Disallow: /campaignlink.php
Disallow: /delivery.php

Disallow: /music/+noredirect/

Disallow: /harming/humans
Disallow: /ignoring/human/orders
Disallow: /harm/to/self

Allow: /
Author name:
Discussion:
No Comments
Newer entries
Older entries