Cloud Gazing – Autumn 2010

13 Oct, 2010 in Eduserv Research by Matt Johnson

This summer has been an exciting one for software offering Infrastructure-as-a-Serivce (IaaS) functionality.

Quota Management in Eucalyptus 2.0

Quota Management in Eucalyptus

One of the best known of the early cloud software providers, Eucalyptus, has recently released version 2.0 of their application, bringing numerous benefits, particularly around storage optimisation, quota management and the support for Microsoft O/Ss in guest instances. Unfortunately for open-source die-hards Eucalyptus operate an “Open Core” model, which means that some of the most attractive new features are only available in the Enterprise (commercial) version. Nonetheless it is a solid release, and is very much needed for the organisation, since in the last few months Eucalyptus has gained some major new competition.

RedHat Cloud Foundation

RedHat Cloud Foundation

Recently RedHat have announced their formal entry to the Cloud market with the RedHat Cloud Foundation, an attempt to deliver an open-standards reference architecture for delivering cloud-based services. Unfortunately whilst the technology and philosophy is sound some of the key components (such as a self-service portal) are missing, and the messaging coming from RedHat is confused and badly presented to say the least. This is not helped by a licensing model (yep, to use the “Official” RedHat software you need to buy an annual subscription) that is poorly designed and rooted in traditional physical environments, rather than cloud-based services. Updates to this model are promised in the coming months which should help clarify the situation, but it is still not clear if a RedHat supported infrastructure will be cost effective for non-enterprise organisations.

OpenStack Logo

OpenStack

Perhaps of more significance is the announcement of OpenStack, an open-source collaboration between a number of organisations coordinated by NASA. OpenStack offers a cloud architecture (codenamed Nova) which has the potential for carrier-scale (1m+ hosts, 10m+ VMs) services. Complementing OpenStack is “Swift“, a cloud storage software that was initially developed by Rackspace and is used to power its Cloud Files service, which is also being open-sourced.

Although only in the stages (the software is still in beta at the moment, and there is a glaring lack of management tools), the potential of OpenStack, and the people backing it, means it is sure to be a big player in the coming months. From a personal perspective, I would be surprised if it didn’t form a key platform of the UK’s planned G-Cloud pubic sector infrastructure [PDF] (see more of my thoughts here).

The more traditional players in the market are responding to the threat of open-source solutions by making their core hypervisors free of charge: VMware (vSphere ESXi), Microsoft (Hyper-V Server 2008) and Citrix (Xen Server) all have base products that are free to use. These companies generate their revenue from the management services that offer advanced functionality (such as vMotion, load-balancing, etc), but even this could be threatened in the coming months by 3rd party solutions that integrate with these services – for example, Eucalyptus can use ESXi as a base hypervisor.

In order to keep ahead of the game, VMware has released vSphere Cloud Director, designed to offer Cloud-style functionality (self-service, metered usage, etc) on top of its more traditional virtualisation business. Microsoft are slightly behind the leading edge; their Dynamic Data Centre (DDC) initiative is more of a stop-gap collection of scripts and technologies to offer automated provisioning, and I would be highly surprised if Microsoft didn’t offer a more cohesive solution in the next 12 months.

Amazon Web Services

Amazon Web Services

In the hosted IaaS arena the leader is clearly still Amazon with its Amazon Web Services (AWS) offers. Highlights so far this year include a very impressive set of MySQL architecture options, in which it becomes ridiculously easy to offer clustered, replicated MySQL services with almost no technical knowledge. In response to demand from customers they have also released a low-cost “Micro” instance designed for small-scale web services, and have introduced their own Linux spin, with AWS tools optimised and tightly integrated into the core O/S.

FleSSR Conceptual Diagram

FleSSR Conceptual Diagram

From Eduserv’s perspective, we have just installed and configured the “public-cloud” segment of the FleSSR project, a 12-month proof-of-concept service developed by a consortia of universities and commercial organisations which aims to demonstrate how private-public cloud interoperability within higher education (and potentially the wider public sector) might work. Our contribution is a cluster of Ubuntu compute nodes backed by a multi-terabyte SAN(provisioned in conjunction with the generous support of NetApp and Q Associates), hosted from our new Swindon Datacentre facility. This will be used by the participating organisations to offload compute or storage capabilities from their own private cloud infrastructure through a policy-driven management service.

Eduserv will also be exploring the benefits, drawbacks and financial/business models of a Cloud-based IaaS services such as FleSSR, particularly in comparison to infrastructure like our own Government Ready Platform, a multi-site replicated vSphere managed-service environment designed specifically for central government. We expect to make these findings available publicly as part of the FleSSR project deliverables, and we anticipate that in addition to contributing to our own thinking around our cloud strategy, this type of information will also be highly valuable to the G-Cloud steering group.

0 Comments
tagged , , , , , ,

OpenStack – the future of G-Cloud?

19 Jul, 2010 in Eduserv Research by Matt Johnson

For the past 18 months the UK Cabinet Office have been working on an ICT Strategy (which, interestingly, is now only available from the National Archive site), and one of the key underlying principles has been the concept of the G-Cloud: Infrastructure-as-a-Service delivered to a UK government specification. Whilst there has been a lot of talk of the potential benefits and savings that could be achieved from this approach, very little has been said on the technology that will be used; hardly surprising, when you consider both the number of stakeholders and the range of cloud services already available. The cloud choices available to UK.Gov include (but are not limited to):

  • Commercial providers of Cloud services
    • Amazon AWS
    • Rackspace Cloud Servers
    • GoGrid
    • Flexiant
  • Commercial providers of Cloud software
    • VMware vSphere
    • Microsoft Dynamic Datacentre
    • Citrix XenServer
    • Eucalyptus Enterprise
  • Open-source providers of Cloud software
    • Eucalyptus Open
    • Canonical UEC

The variety of software (and associated “standards”) presents a big challenge; which one(s) do you choose to support as part of a G-Cloud service? Working with just 1 or 2 simplifies your integration work, but leaves the others understandably upset; go with all of them, and you have a nightmare situation in terms of interconnectivity. An alternative approach is to adopt an open specification, such as the one offered by the OGF Open Cloud Computing Interface working group, and encourage (or force) any supplier wishing to join to support that standard. The current website, however, doesn’t inspire much confidence in level of adoption of the technology.

This may all change with the recent announcement by NASA/Rackspace of OpenStack, an open-source software/storage stack for managing compute and storage clouds, but from NASA’s Nebula compute platform. Although not fully OGF OCCI compliant, it aims to implement the Amazon AWS API, which is a de facto standard within Cloud Computing. Other providers, notably Eucalyptus, already offer this, however the OpenStack system promises better scalability and more functionality within the GPL licenced software.

OpenStack is of great interest to those of us providing hosting services to the UK public sector; personally I would be very surprised to see the Cabinet Office adopt anything else, because:

  • The platform is open source in its entirety;
  • The standards it will work under are well known and published;
  • It is based on the KVM hypervisor, which supports not just Linux but also Windows guest platforms;
  • There is already a large pool of people who understand the AWS API specification;
  • The platform’s development is being funded by others (primarily the US Government);

In essence, NASA (and its commercial partners) have effectively solved the technical specification of the G-Cloud – now all that remains is the implementation…

1 Comment
tagged

Using Dropbox as a data source

18 Jun, 2010 in Desktop Tech by Mike Ellis

I’ve been building a site in my spare time for a friend – it is called The Last Survivors, and focuses on two particular species that are facing extinction.

One of the requirements was for the editors of the site to be able to add results from their surveys of these species quickly to a Google Map for embedding in the website. Nothing particularly new in the build there, but I wanted to make sure that the workflow was as seamless as possible and at the same time keep the dev work to a minimum.

We started off collaborating using a bunch of Google Docs – they held latlong and description data in a Google Spreadsheet with an associated form for inputting data – and I came across this simple service to create a map from a Google Spreadsheet source.

It’s pretty good and worked very well for the initial stages. Pretty soon, though, it became clear that this wasn’t going to cut it longer term – we needed to do things like add different colour markers depending on the type of species, change the description field based on various input fields and so on.

I wanted to avoid building yet another interface for the input of data, so started off by looking at ways in which I could continue to use a spreadsheet as a data source, but then pipe the data (as CSV) into a custom script which did the actual building of the map.

Firstly, I looked at Google Spreadsheets, which let you publish your stuff in a whole bunch of different ways. This makes it a pretty powerful platform for holding data; The Guardian is innovating in this space – here, for instance is an article on UK pubs and an associated, downloadable and queryable spreadsheet with the data.

Then I started thinking about how I could improve the flow of the user experience. The research team already hold their map data in Excel, and also use Dropbox, and before long the idea of using this as a data source floated into my mind.

Dropbox provide a very powerful feature which they don’t seem to shout about that much – every user has a “public” folder – whatever you put in here has a public URL which anyone can see. I use this all the time when sharing screengrabs, snippets of html and so on: of course I could FTP, but when you’re working in a desktop environment the usability hit of simply copy/pasting into a folder is lower than booting an FTP client. And it’s not like my clients want to be bothered installing a client, learning how to navigate the backend of their website, and so on.

So that’s it. I gave my script the public URL of the endangered species CSV data source – it heads off whenever prompted, grabs the data, builds a map and then creates a KML file which is consumed by the map page. The guys who add the data don’t need to know any of that – all they do is open up the CSV in Excel, edit the data, save it. Job done.

None of this is revolutionary, but it’s been an interesting experiment – from a technical perspective, yes, but more because the team have really appreciated a simple approach which has zero impact on their flow of working.

2 Comments
tagged , , ,

Designing a Ubuntu Enterprise Cloud

25 May, 2010 in UEC Cloud Platform by Matt Johnson

Over the coming months Eduserv will be building a proof-of-concept implementation of a Ubuntu Enterprise Cloud (UEC) platform, to assess how (or if) it can meet the needs of the UK public sector in general, and more specifically the higher education community.

The concepts behind building a UEC infrastructure are simple enough, but they do require you to “unlearn” common assumptions and best practice that comes from running more traditional managed hosting services. This is particularly true when it comes to balancing infrastructure costs against high-availability designs; in most cases the assumption is that cloud services should be cheap. One of the key aims of this project is to understand if this is indeed the case, or if there is a need for “highly-available” cloud services with appropriate SLA guarantees.

Continue Reading »

0 Comments
tagged , , ,

QR tag demo at the Eduserv Symposium 2010

12 May, 2010 in QR Demo by Mike Ellis

The Eduserv Symposium (The Mobile University) is tomorrow. We’ll be discussing all things mobile and trying to tease out what these new approaches might mean in the education space.

As part of this, I thought it’d be interesting to put together a re-hash of the prototype I built for the Museums Computer Group conference in 2009: a simple and lightweight system which uses QR tags to help people connect with each other in a conference environment.

I wrote up the MCG demo on my personal blog, but briefly the system works as follows:

  1. Every delegate has a personalised badge which has the normal things you’d expect (like name and institution), but also a QR tag which contains a URL to a unique web page on our system, and a unique PIN
  2. When person A meets person B, they can use their mobile to scan that person’s badge. If this is the first time they’ve used the system, they are asked to login by using their own PIN
  3. When they scan the badge, the system directs them to a unique URL, and then displays the details for the person they just tagged.
  4. More usefully, the system also emails them a vCard of that person which they can then add to their contacts if needed

The system is incredibly simple, both from a technical and – hopefully – from a usability perspective, but I hope that there is enough of a value-add for people to try it out.

From a usability perspective, there are probably several key factors which will determine how much this kind of a system might be used. The most important of these lie not around the technology but the awkwardness of the inter-personal interaction. Walking up to someone you’ve never met before and asking them to stand still long enough for you to get out your mobile, boot up a QR app, scan (“hold still, left a bit….” etc) is likely to be a fair-sized barrier; a bigger barrier than simply asking them for a card! Pretty much any interaction of this kind triggers a kind of subconscious cost/benefit analysis in the user: “how much gain do I get from doing this interaction this way, or is it easier to do X?” – what is interesting is watching where the walk away threshold is in any given scenario.

Secondly, of course, not everyone – in fact, probably a minority (even at a conference about mobile, where we’ve primed reasonably tech-savvy delegates to download a QR reader) – are going to have a mobile device which is internet enabled.

I gave a talk on mobile use in cultural heritage institutions recently in which one of the key messages was that in building for mobile we need more than ever to make decisions about balancing capability and ubiquity. Here is the current reckoning on the current state of the mobile ecosystem in terms of capability:

  • 120% of UK citizens have a mobile (yes, some of us have more than one..)
  • 100% of all mobiles support SMS
  • 95% have *some kind* of browser (including WAP)
  • 73% have a camera
  • 71% have a “modern” browser

The important one from our perspective (the 95%/71% browser stat) is actually pretty optimistic in some senses: it is making an assumption based on hardware which may not be backed up either by the user knowing how to use their browser OR having a data plan which supports browsing.

The prototype – as I’ve already indicated – has a 4-digit PIN for each delegate: this removes a fair amount of the barrier (you don’t actually need a QR code reader to take part in the demo), but you do need a mobile and data plan which support web browsing.

I did some analysis for the museums conference looking at who tagged who and when, which showed some interesting things (for instance, a large number of people tagged themselves…!). I’ll do the same following the Symposium and we’ll see what similarities and differences there are.

2 Comments
tagged , , , , , ,