Amazon Web Services: A Quick Reference Guide

Amazon Web Services (AWS) is a collection of managed web services (infrastructure building blocks) that together make up a cloud computing platform.  The most well known are Elastic Compute Cloud (EC2) and Simple Storage Services (S3).

Amazon has been steadily improving and innovating on multiple fronts – storage, compute, databases, management and orchestration. The innovation is outpacing the market ability to absorb in many cases. Corporate IT is way behind the startup world in its ability to comprehend, let alone leverage these Web Services.

The next decade is going to be the transformation of enterprise architecture to take advantage of these innovations. Just like e-commerce took almost 10+ years to become mainstream, i anticipate the same trajectory is going to happen with the adoption of cloud web services.

How can these web services help your business?

Amazon Elastic Compute Cloud (EC2)

The Amazon EC2 (Elastic Compute Cloud) is a Web service that allows users to provision new machines into Amazon’s virtualized infrastructure in a matter of minutes; using a publicly available API (application programming interface), it reduces the time required to obtain and boot a new server.

EC2 is very attractive to IT organizations that want to move fast.  The 5 core benefits are : (1) elastic capacity with ability to scale up and down in minutes; (2) quick and global deployment; (3) No CapEx, no initial large spend; (4) Pay as you go, for what you use; and (5) Automation and scaling.

Provisioning

Setting up an EC2 instance is quite easy. Once you create your AWS (Amazon Web service) account, you can use the on-line AWS console, or simply download the offline command line’s tools to start provisioning your instances.

EC2 instance is typically a virtual machine with a certain amount of RAM, CPU, and storage capacity. Users get full root access and can install almost any OS or application in their AMIs (Amazon Machine Images). Web services APIs allow users to reboot their instances remotely, scale capacity quickly, and use on-demand service when needed; by adding tens, or even hundreds, of machines.

There is no up-front hardware setup and there are no installation costs, because Amazon charges only for the capacity you actually use.

Pricing Models

Amazon EC2 provides its customers with three flexible purchasing models to make it easy for the cost optimization:

• 
  On-Demand instances, which allow you to pay a fixed rate by the hour with no commitment.
• Reserved instances, which allow you to pay a low, one-time fee and in turn receive a significant discount on the hourly usage charge for that instance. It ensures that any reserved instance you launch is guaranteed to succeed (provided that you have booked them in advance). This means that users of these instances should not be affected by any transient limitations in EC2 capacity.
• 
  Spot instances, which enable you to bid whatever price you want for instance capacity, providing for even greater savings, if your applications have flexible start and end times.
• Pay for servers “by the hour” (on-­‐demand); Pay for storage “per Gigabyte” per month; Pay for data transfer “per Gigabyte”;

AutoScaling and Provisioning

Amazon provides an excellent set of tools that help in provisioning service. For instance…it makes it easy to turn resources on/off (running  costs).

Amazon Auto Scaling is a set of command line tools that allows scaling Amazon EC2 capacity up or down automatically and according to the conditions the end user defines.

This feature ensures that the number of Amazon EC2 instances can scale up seamlessly during demand spikes to maintain performance and can scale down automatically when loads diminish and become less intensive to minimize the costs.

Auto Scaling service and CloudWatch (a monitoring service for AWS cloud resources and their utilization) help in exposing functionalities required for provisioning application services on Amazon EC2.

Amazon Elastic Load Balancer is another service that helps in building fault-tolerant applications by automatically provisioning incoming application workload across available Amazon EC2 instances and in multiple availability zones.

Notes and References

• AWS certifications that are critical for CIOs — SAS70 Type II Audit; ISO-27001 Certification; PCI Level 1 Service Provider; FISMA A&A; Sarbanes-Oxley (SOX) Compliant Platform
• See Amazon Web Services: A Quick Reference Guide

Related articles

• Pinterest: Leveraging the Amazon Cloud (cloudblueprint.wordpress.com)
• Amazon EC2 gets a spot market (news.cnet.com)
• Amazon Web Services simplifies creation of private clouds (infoworld.com)

Amazon Web Services (AWS)

AWS pioneered the introduction of IaaS clouds in 2006.  AWS mission “Enable businesses and developers to use web services (what people now call “the Cloud”) to build capable, sophisticated applications.”

AWS is essentially is a self-service infrastructure outsourcing business. It’s the third segment of Amazon.com’s business – consumer business, marketplace and IT infrastructure.

AWS business model is utility computing. Simply put this is a business model for on-demand delivery of computing power; consumers pay providers based on usage (“pay-as-you-go”), similar to the way in which we currently obtain services from traditional public utility services such as water, electricity, gas, and telephony.

Rapid Business Model Shift. In the IT infrastructure industry we are seeing the value migration to cloud (or utility computing) models. The concept is that computing is always there like electricity.  When plugging an electric appliance into an outlet, we care neither how electric power is generated nor how it gets to that outlet. This is possible because electricity is virtualized; that is, it is readily available from a wall socket that hides power generation stations and a huge distribution grid. When extended to IT, this concept means delivering useful functions while hiding how their internals work.

Pinterest: Leveraging the Amazon Cloud

Heard an interesting case study at the Amazon Web Services Summit, in New York.  Pinterest is an online pinboard where users can organize, share and explore things they love. Pinterest was named 2012′s Breakout Digital Trend at South by Southwest in March.

Pinterest is completely running in the cloud.  Pinterest has made heavy use of Amazon Web Services, in particular relying on Amazon EC2 and Amazon S3 for critical infrastructure.  What makes the case study interesting is the rapid growth, almost 18+M visitors, that Pinterest is able to support with a small infrastructure team.

According to Ryan Park, operations engineer for Pinterest, “The cloud has enabled us to be more efficient, to try out new experiments at a very low cost, and enabled us to grow the site very dramatically while maintaining a very small team of 2 people.” The advantages Pinterest is leveraging: (1) elastic capacity; (2) quick and global deployment; (3) No CapEx, no initial spend; (4) Pay as you go, for what you use; and (5) Automation and Reuse.

Understanding Personal Cloud Management: Apple iCloud, SkyDrive and AcerCloud

As people switch to mobile devices as their center of gravity, the ecosystem of tools need to support them and keep them productive changes.  Apple iCloud provide a glimpse into what is coming.

According to Steve Jobs  “it is a real hassle and very frustrating to keep all your information and content up-to-date across all your devices. iCloud keeps your important information and content up to date across all your devices. All of this happens automatically and wirelessly, and because it’s integrated into our apps you don’t even need to think about it,it all just works.”

Apple with its iCloud solution offering is attacking the consumer multi-device digital content access, backup and replication problem. Surprised Google has not responded with an equivalent offering whereas Microsoft came out with SkyDrive.

So, what’s the big deal?  Apple is addressing a widely faced content retrieval problem:   How to free up data trapped in “device and  application jails” in a user-friendly way. The “scan and match” concept   is quite applicable to large scale device specific repositories which suffer from data integrity issues as edge data capture and consumption devices proliferate.

What can enterprises learn from iCloud? Seamless data ingestion, governance and management is a huge problem facing large organizations.  Getting data out of and sync’ing data across laptops, desktops and multiple mobile devices of employees is a big challenge. Apple shows the next generation way of addressing the personal cloud management problem.

Cloud First Strategy of U.S Federal Agencies

The U.S. Federal Government is pushing a “Cloud First” policy in 2012.

This policy is intended to accelerate the pace at which the government will realize the value of multi-tenant, subscription-based  cloud computing by requiring agencies to evaluate safe, secure cloud computing options before making any new investments. The goal is shut down  1200 data centers by 2015.  The U.S government has 3200 data centers > than 500 sq ft.

The Dept of Defense (DoD) modernization cuts are going to accelerate the move to the cloud services.  DoD, Dept of Homeland Security (DHS) and the Intelligence Services are extremely large buyers of computing.  If they start moving services to the cloud, I believe the cloud trend is going to accelerate creating a talent shortage in the government contractors.

The increasing adoption of cloud computing and penetration of mobile devices are the two inter-related trends transforming the IT industry today.  Government is no exception and in some areas leading the charge.

We are clearly in the middle of a once-in-a-decade transformation.  In the consumer and retail world, the movement to Web based cloud services, is being accelerated by the move to build social apps, mobile apps, location aware, real-time applications. It’s increasingly obvious that old applications and technologies need to adapted —  Four different delivery models are emerging differentiated by service level agreements (SLAs).

But, how do you explain “cloud services” to someone new to this space  especially in Government?  There are multiple ways of describing cloud services.

In this page I highlight some useful taxonomies from various sources that I found useful in explaining Cloud Computing and Cloud Services. Depending on who you are talking to pick the relevant one.

• Describing the Cloud to an Enterprise Audience
• Describing the Cloud to a Technical Audience
• Describing the Cloud as Outsourcing 3.0
• Describing the Innovation Roadmap unleashed by the Cloud
• NIST Cloud Taxonomy

Zynga’s Z-Cloud Infrastructure

Cloud Computing when done right can be a source of significant competitive advantage.

Case in point Zynga, the social gaming company.

Zynga, founded in 2007, is the world’s largest social game developer with more than 250 million monthly active users playing their games, which include CityVille, FarmVille, FrontierVille, Words With Friends, Mafia Wars, Zynga Poker, Café World and Treasure Isle.

Zynga games are available on a number of global platforms including Facebook, MySpace, Yahoo, the iPad and iPhone, and Android devices.

Zynga makes most of its revenue by serving the 800m audience on Facebook.

Social games powered by Facebook represent a new form of entertainment that will capture an increasing proportion of consumer leisure time.  The potential is huge.  One data point… my 72 year old mother-in law spends hours playing games on her iPad.

Social games are the most popular applications on Facebook and will be, a key driver of engagement on social networks, and increasingly on mobile platforms.

Games have become the second most popular internet activity based on time spent, and have even surpassed email.

To support this growth, Zynga  created a scalable hybrid cloud-based server and network infrastructure that enables it to deliver games to millions of players simultaneously with high levels of performance and reliability.

Investing in SaaS – Fidelity Investments

Fidelity Investments put out an interesting macro analysis on  Software-as-a-Service (SaaS) as an Investment Themes for clients.  I reproduced the article here with additional commentary.

Key Takeaways

• As virtualization and other technology developments break the link between applications and the hardware on which they run, software can be offered as a service rather than a product.
• The shift from servers to services can provide tremendous cost savings in application development and maintenance to a variety of businesses.
• Properly executed… SaaS users will benefit initially from lower start-up costs, and developers of popular applications can generate significant subscriber revenue on an ongoing basis.
• However, figuring out which applications to bring into the cloud and which to retain in the traditional data center can be a daunting task for any IT organization.

Investment Theme – Software as a Service

Accessing software as a service (SaaS) is a direct result of cloud computing.

Under the traditional model, even a small- or medium-sized enterprise (SME) would incur significant start-up costs of software application licensing and installation, as well as ongoing expenses for maintenance, renewals, upgrades, and professional services such as IT consultants. Similarly, consumers had to purchase and support software on their own computers.

Investing in Cloud Computing – Fidelity Investments

Fidelity Investments put out an interesting macro analysis on Cloud Computing  as an Investment Themes for clients.

Driving the interest in Cloud is executive dissatistfaction with IT in most corporations. Typical reasons:   IT assets are underutilized? IT costs are too high? IT is too slow and not current (e.g., tablets, etc.)?  Not getting the bang from IT projects or investments?

Basically,  the Cloud is a shift from “buy-and-own” to “pay-as-you-go.”  This (IT-as-a-service) has broad implications for activities such as procurement and staffing— and it could lead to a new role for the IT department.

Key Takeways — The Cloud’s “Snowball Effect”

Massive new wave of IT infrastructure upgrade is underway.  Check out this causal chain.

• Maturation of Virtualization Technology driven by VMware has happened (2007 – present)
• Server Virtualization enables Compute Clouds
• Compute Clouds create demand for Storage Clouds
• Storage + Compute Clouds create Cloud Infrastructure
• Desktop virtualization creates demand for Network upgrades
• Cloud Infrastructure enables Cloud Platforms & Applications
• Multiple Cloud types lead to Cloud Aggregators
• Niche requirements enable Cloud Extenders

Investment Theme – Cloud Computing

Every decade or so, there is a major architectural shift in technology where a dominant ecosystem emerges and results in massive wealth creation. Cloud computing—using data centers pooling compute, storage, and network resources to manage data and run software—represents such a shift today.

Cloud Services Brokers – New Wave in Systems Integration

Multi-tenant, subscription based Software-as-a-Service is entering a new phase.  SAP announced in December 2011 that it’s acquiring Success Factors for $3.4 billion. The acquisition of Success Factors is aimed at making SAP AG  a leader in “cloud computing”.  Seems like a catchup play. The good news is that this acquisition legitimizes the Cloud and makes it mainstream.

Every customer’s reality is that they are going to have multiple datacenters.  Inside the private cloud, inside the public cloud, they will use different service providers, and we’re going to live in this world of hybrid IT for all time to come.  And you really need a distributed computing fabric that brings things together on behalf of enterprise IT as well as the developers writing native cloud applications.  And these commonalities of identity, virtualization, management, and application development is what makes the next generation systems integration challenge very unique.

As SaaS, IaaS and PaaS becomes mainstream, we are seeing a new demand for Systems Integration capability.  According to Forrester Research: “…developers are bypassing IT and putting applications onto public clouds at a rate five times greater than IT thinks.” 

Companies need to hook up [users, systems, databases, applications, and web services] with more than a dozen different cloud services providers.  Instead of doing a point to point integration they are looking to hook into a Cloud Service Brokers, who would serve as intermediaries, offering such services as customization, integration, security, and aggregation.

The figure below shows a simplistic view of the service broker model.

A new category of players is emerging in the Systems Integration market: Cloud Service Brokers (CSB). Because CSB is not a technology, it is necessary to understand describe the business model, providers, functions and enablers.