I hope this will give an insight into the way in which Service Providers need to be thinking in order to create an efficient operating model for storage services and provide the breadth of different services that are being required to support today’s application architectures.
I’ve got quite a bit of content to cover so I’m going to break this up into a series of three posts to make this more digestible.
Firstly, here’s a chart that shows the typical service offerings that a service provider might want to offer and their characteristics.
Object storage presented via a web-services API is the immediate assumption of the service offer that purists will assume is at the core of any ‘Cloud-storage’ offer, but what’s the reason for this? Two factors that are driving this perception in my view. Firstly, Amazon’s S3 service is often regarded as the ‘flagship’ storage a service offer within the AWS portfolio. Secondly, Amazon’s position as the largest ‘over-the-top’ (OTT) or internet-based cloud storage service provider gives them the opportunity to act as an anchor for customer’s expectations of the services and pricing that other SPs need to have in their portfolios. The first one of these points isn’t strictly true as Elastic Block Storage (EBS), or something with similar characteristics, is also a critically important component of AWS or any other cloud storage offer, but I’ll come back to this when I cover block in a subsequent post.
The key thing to remember about Object Storage Services is that they are accessed via a programmatic web-services interface over an IP network and that in most cases these APIs are directly accessible via the internet. The use of a web-services API for access means that they cannot be used with most existing enterprise (or legacy, if you prefer) applications that are expecting to deal with directories and files via an interface provided by the operating system or even directly with raw blocks of storage capacity in some cases. Whilst some gateway devices are appearing now that allow you bridge between operating system and cloud storage, these are very new and the performance characteristics of Object Storage might result in unexpected or unpredictable results with your applications.
There’s obviously an upside to the API presentation, which is that you can post metadata whenever you store an object via the API. This metadata can then be used to apply policies to the object whilst it is held in the platform. These policies might control features such as the expiration and deletion date of the object, the number of copies of the object to hold, and where to hold them, or the security policy applied to requests for access to the object. It doesn’t take much imagination to quickly realise how this might make managing extremely large numbers of objects and owned by very large numbers of tenants much less resource intensive that traditional storage management techniques – as long as you get the policy and metadata framework right in the first place of course!
Just one thing about costs and pricing for these services – if you plan to retain data for a significant length of time, say in excess of 12 months, or it access it frequently, you need to look carefully at the costs of using S3. And remember that IOs and data transfer from the Amazon service are charged in addition to the baseline data storage charges. It is significantly more cost efficient to build a platform using either a software and commodity hardware platform, or an appliance-based platform like EMC’s Atmos, than it is lease capacity on Amazon S3. When you’re operating at scale, like most service providers do of course, you can achieve a sell price to your customers below that of Amazon’s and still gnerate significant margin from these services.
Service Provider selection criteria for object storage platforms? You need to be thinking about operations at scale from day one, so a robust multi-tenancy model and policy based management framework are absolutely critical. Also, look for simplicity and ease of scaling when adding nodes to the environment, integration with rating and billing systems and of course acquisition and operating costs.The ultimate objective for a Service Provider launching this type of service is generating the demand to ensure rapid adoption and scaling of the platform. This will in turn result in lower per-unit operating costs and improved profitability and the most successful SPs are those that have developed a portfolio of solutions that drive content onto the Object Storage platform – good examples are remote data archiving and backup services, sync and share applications like Oxygen Cloud or VMware’s Project Octopus and of course other software solutions from software developers that might be persuaded or encouraged to use an SP provided Object Storage Service – for example mobile applications, medical imaging applications and pretty much anything else you or more likely a 21 year old computer science student can think of
Some Useful Links
The OpenStack Object Storage Project
Case Study of an Object Storage (Atmos) Implementation within an Enterprise
That’s a quick explanation of Object storage services and their increasingly important role in an overall storage architecture. I will cover scale out File and scale out Block in similar detail in a couple of subsequent posts over the next week or so.