Category Archives: Cloud

First look at Azure Container Service

In Episode 115 of the Microsoft Cloud Show we interviewed Ross Gardler from Microsoft about their new Azure Container Service which is currently in preview. I finally got some time a few weeks ago to play with ACS and thought I would share my first experiences here.  This is my 0 to first container experience.

Currently ACS allows you to provision 2 types of container service. Either a Mesos based deployment or a Swarm one. I hadn’t played with Meso much so opted to try that out.

Getting started

Note: I followed the getting started guide for deploying a new container service on the Azure website.

To get started its as simple as clicking a “Deploy to Azure” button on the pre-canned Azure template.   This will take you to your Azure management console where you can configure the various parameters for this template as shown below.

2016-02-20_16-10-37

You need to name your cluster, pick the VM size for the nodes you want to run and set authentication details.  The toughest part of this for most people will be generating the SSH keys as this is pretty foreign to many Windows folks.  But they provide a fairly simple walkthrough for you to create a key pair.

When complete you hit OK and go get a coffee while your cluster is deployed 🙂  It can take a while as it spins up a few machines and configures everything.

deploy

Note: I got an error “\”The subscription is not registered to use namespace ‘Microsoft.Compute’.” during deployment the first time.  I was deploying into a new MSDN Azure Subscription with free credit on it.  Turns out I needed to manually create a VM in this subscription first (any VM will do) before deployment of a template would work.  Once I had done this the template deployed fine.

I deployed a pretty simple cluster with 2 agent nodes and a mesos master node.  In Azure you can see all the resources the template created in a new resource group such as the VMs, networks and security groups etc…

2016-02-20_16-48-48

Now I had a cluster up and running I could log into Mesos.  To find the URL click “Succeeded” on the resource groups deployment status and click “Microsoft.Template”.  You should see a couple of fully qualified domain names.

2016-03-14_11-31-39

To actually hit Meso you need to create an SSH tunnel from your box into the cluster.  There is a decent write up on how to do this here.

Once you have your SSH tunnel running you can hit the Mesos web interface on http://localhost/mesos/  (this is redirected over the SSH tunnel to your meso box running in Azure).

2016-02-21_10-22-35

Now you are ready to start running things!  Hit http://localhost/marathon/ to open the Marathon web UI which makes it pretty simple to run jobs on your cluster.

Click create and give it a name, 256MB and 1 instance.  Open the Docker container settings and specify “yeasy\simple-web” as the image name.  Then in the Optional Settings area set Port = 80.  This will map port 80 in the docker container to port 80 on the host. Create the app and let it spin up.  You should see it in the UI similar to this:

2016-02-21_10-25-17

Grab your load balancers fully qualified domain name from the Azure portal.  It’s the AGENTFQDN url in the deployment details you found earlier.

You should be able to hit that URL and see your simple website running!

Summary

This is obviously only the most basic thing you can do with a Meso based cluster running in Azure, but was my attempt and seeing how Azure are approaching the setup.  All in all it was surprisingly painless.

The goal of ACS right now so it make it simple to run a docker cluster in Azure using either Mesos or Swarm.  It doesn’t take away the need to manage that cluster in Azure once its deployed, so you will need people who know how to run a Mesos cluster and feed and water it appropriately.  Deployment is step one, but running it is a different beast all together from what I understand.  I am no expert in this area and so you will want to tread carefully and make sure you have the appropriate skills on staff to do this.

I for one would LOVE to see Azure also add as Container as a Service (CaaS) offering where you just specify how much compute you want, how much memory etc… and then have Azure spin up and manage a Docker cluster for you with the infrastructure being invisible.  This way you don’t need to be a Mesos master and you can let the pros run it for you.

I think CaaS is the final destination for Docker … just prior to everyone starting to espouse the virtues of true Platform as a Service (PaaS) and ditching this whole concept of apps running in containers and being aware of the OS at all.

When true CaaS comes to fruition, like it think it will in time, maybe Ray Ozzie (inventor of Azure, codename Red Dog) can all say “told ya so” about his vision of Platform as a Service being the ultimate destination for cloud computing (but being about 10 years too early).

Running apps using Docker Cloud (aka Tutum)

Anyone who has listen to me rant on about how interesting Docker is on the Microsoft Cloud Show may have caught me talking about Tutum.  

imageThe short story on Tutum is that it provides an easy to use management application over Virtual Machines that you want to run your apps on with Docker.  It is (sort of) cloud provider agnostic in that it supports Amazon Web Services, Microsoft Azure and Digital Ocean among others.

It was bought by Docker late last year and recently was recently re-released as Docker Cloud

What does it provide?

At a high level you still pay for your VMs wherever you host them, but Docker Cloud provides you management of them for 2c an hour (after your first free node) no matter how big or small they are.   You write your code, package it in a Docker Image as per usual and then use Docker Cloud to deploy containers based on those docker images to your Docker nodes. You can do this manually or have it triggered when you push your image to somewhere like Docker Hub as part of a continuous integration set up.

Once you have deployed your app (“Services” in Docker Cloud terminology) you can use it to monitor them, scale them, check logs, redeploy a newer version or turn them off etc…  They provide an easy to use Web App, REST APIs and a Command Line Interface (CLI).

So how easy is it really?

Getting going …

The first thing you have to do is connect to your cloud provider like Azure.  For Azure this means downloading a certificate from Docker Cloud and uploading it into your Azure subscription.  This lets Docker Cloud use the Azure APIs to manage things in your subscription for you. (details here)

Once you have done that you can start deploying Virtual Machines, “Nodes” in Docker Cloud terminology.  Below I’m creating a 2 node cluster of A2 size in the West US region of Azure. 

image

That’s it.  Click “Launch node cluster” wait a few mins (ok quite a few) and you have a functional Docker cluster up and running in Azure.

image

In Azure you can take a look at what Docker Cloud has created for you.  Note that as of the time of writing that Docker Cloud is provisioning “Classic” style VMs in Azure and not using the newer ARM model.  They also deploy different VMs into their own cloud services and resource groups which isn’t good for production.  That said, Docker Cloud let you Bring Your Own Node (BYON) which lets you provision the VMs however you like, install the Docker Cloud agent on them and then register them in Docker Cloud.  Using this you can deploy your VMs using ARM in Azure and configure them however you like.

image

Deploy stuff …

Now you have a node or two ready you can start deploying your apps to them!  Before you do this you obviously need to write your app … or use something simple like a pre-canned demo Docker Image to test things out.

Docker Cloud makes this really simple through “Services”.  You create a new service, tell it where it should pull the Docker Image from and a few other configuration options like Ports to map etc… Then Create and your containers will be deployed to your nodes.

Try this once you have your nodes up and running.  Click Services in the top nav,  then Create Service. Under Jumpstarts & Miscellaneous category you should see the “dockercloud/hello-world” image. Select it and then set it up like this:

image

There are only a two things I changed from the default setup.

  1. I moved the slider to 2 in order to deploy 2 containers
  2. Mapped Port 80 of the container to Port 80 of the node and clicked Published.  This maps port 80 of the VM to port 80 of the container running on it so that we can hit it with a web browser.
  3. High availability in the deployment strategy.  This will ensure that the containers are spread across available nodes vs. both on one.

Click “Create and deploy” and you should see your containers starting up.   Pretty simple huh!

image

Note: There is obviously a lot more available via configuration for things like environment variables and volume management for data that you will eventually need to learn about as you develop and deploy apps using Docker.

Once your containers are deployed you will see them move to the running state:

image

Now you have two hello world containers running on your nodes.  If you go back to your list of Nodes you should see 1 container running on each:

image

I want to see the good man!

You can test your hello world app out by hitting its endpoint.  You can find out what that is under the Service you created in the Endpoints information.

image

  1. This is the service endpoint.  It will use DNS round robin to direct requests between your two running containers.
  2. These are the individual endpoints for each container.  You can hit each one independently.

Try it out!  Open the URL provided in a browser and you should see something like this:

image

Note that #1 will indicate what container you are hitting.

Want more containers?  Go into your Service and move the slider and hit apply.

image

You will get an error like this:

image

This is because we mapped port 80 of the Node to the Container and you can only do that mapping once per Node/VM. i.e. two containers cant both be listening on port 80.  So unless you use a HAProxy or similar to load balance your containers you will be limited to one container on each node mapped to port 80.  I might write up another post about how to do this better using HAProxy.

Automate all the things …

We are a small company and we want to automate things as much as possible to reduce the manual effort required for mundane tasks.  We have opted to use Docker Cloud for helping us deploy containers to Azure as part of our continuous integration and continuous deployment pipeline.

In a nutshell when a developer commits code it goes through the following pipeline and automatically is deployed to our staging environment:

  1. Code is committed to GitHub
  2. Travis-CI.com is notified and it pulls the code and builds it.  Once built it creates a Docker Image and pushes it to Docker Hub.
  3. Docker Cloud is triggered by a new image.  It picks it up and redeploys that Service using the new version of the image.

This way a few minutes after a developers commits code the app has been built and deployed seamlessly into Azure for us.  We have a Big Dev Ops Flashing Thing hanging on the wall telling us how the build is going.

Cool … what else …

At Hyperfish we have been using Tutum for a while during its preview period with what I think is great success.  Sure, there have been issues during the preview, but on the whole I think it has saved us a TON of time and effort setting up and configuring docker environments.  Hell I am a developer kinda guy, not much of a infrastructure one and I managed to get it working easily which I think is really saying something 🙂

Is this how you will run production?

Not 100% sure to be honest.  It is certainly a fantastic tool that helps you run your apps easily and quickly.  But there is a nagging sensation in the back of my head that it is yet another service dependency that will have its share of downtime and issues and that might complicate things.  But I guess you could say that about any additional bit of technology you introduce and take a dependency on. That said, traffic to and from your apps is not going through Docker Cloud, traffic goes direct to your nodes in Azure so if they have brief downtime your app should continue to run just fine.

I have said that for the size we currently are and with the team focusing on building product that we might consider something else only once we can do a better job that it does for us.

We might consider something else only once we can do a better job that it does for us.

All in all I think Docker Cloud has a lot of great things to offer.   It will be interesting to compare and contrast these with the likes of Azure’s new service, Azure Container Service (ACS) as it matures and approaches General Availability.  It’s definitely something we will look at also.

-CJ

Sandbox code is “deprecated”, long live the Sandbox

we have deprecated the use of custom managed code within the sandboxed solution – Brian Jones, Principal Program Manager, Apps for SharePoint

It’s been a long time coming and widely anticipated that this announcement would come at some point.  It’s great to see the announcement and the clarity many have been asking for.

I feel like I’m in a good position to comment on this and give some background about why I think deprecating code based sandbox solutions is good idea. I was on the SharePoint engineering team when the sandbox was being built and for a period of time I was the Program Manager for the feature.

Background:  Sandbox solutions in SharePoint were introduced in SharePoint 2010.  They allowed a packaged set of assets and code to the uploaded to a SharePoint site.  That can consist of declarative components like XML for adding things like List Templates, as well as compiled code for things like Web Parts or Event Receivers.

When the Sandbox was being designed and built it was about 2 – 3 years prior to SharePoint 2010 being released. Azure, and cloud computing in general, either didn’t exist or was in its infancy. SharePoint needed a way to upload customizations/components to SharePoint sites where the administrators were not comfortable with installing Farm Solutions aka. Full trust solutions. Microsoft itself was a perfect example of this.  If I built a web part I, as a Microsoft employee at the time, couldn’t load that onto our SharePoint sites that MS IT ran.  No 3rd party web parts or products. This was a common problem in many large organizations and we heard about this time and time again with customers.

Sandbox Solutions were the answer. They allowed users to upload a solution and have SharePoint run it while it being controlled, secured and run in a sandboxed process. The main thing this gave SharePoint was the ability to isolate the code in that solution and ensuring that if it crashed or was badly behaved that it didn’t break the rest of the SharePoint environment.

The problem was that Sandbox Solutions was a feature added in Windows SharePoint Services (WSS) and that was a different product from SharePoint Portal Server (SPS) that built on top of WSS.  The API set in the Sandbox that was available was limited to WSS APIs and even then only a subset of them. There were good reasons for this, but at the end of the day it was very limiting for people.  Ideally it would have been great to have lots of SPS APIs available too. But that didn’t happen (different story).

So that is the background about how/why Sandbox came about.

… now fast forward to today …

In a nutshell the Sandbox was a good solution to the problem faced when it was designed. However, it’s not a great solution to the problem given the technology we have today.

Why is it no good today you ask?

A lot happened after the Sandbox was designed and built.  Cloud computing took off, new advances in code in all sorts of places got easier e.g. isolated apps on a phone.  A lot was learnt. 

In short its my belief that SharePoint shouldn’t have been trying to replicate an isolated code hosting environment.  That is reinventing the wheel and there are other teams at MS who build products to do this extremely well already.  Namely IIS and Azure.

Think about that for a second.  Imagine being given some arbitrary code and told to run it, but doing it in a way that was safe, secure, manageable and fault tolerant.  It’s actually quite a tough challenge. If you say it’s easy then you should try doing it instead of talking about it 🙂 (tip with wider life applicability too)

So today MS clarified that SharePoint was getting out of the code hosting game.  Why?  Because it was limited and there are better solutions to this problem today. 

The new SharePoint app model is designed to solve this by moving “sandbox” code to an alternative host e.g. IIS or Azure or <insert thing that runs code here>.

Sandbox code might be “dead”, but the new app model IS the new Sandbox!

I see the reasons why the sandbox came about the same as I do for the new app model.  They are solving the same problem.  How do you allow someone to customize, extend and build new things on SharePoint without compromising the integrity of SharePoint itself?  That is the goal.

Today in SharePoint 2013 and Office 365 we have the ability to build solutions that use this new app model.  Sure, it’s not perfect in the APIs it provides and there is plenty of scope for adding things.  I am certain it will evolve to cater for more of these things over time.  That said, it is MUCH better suited for the long term.  I for one am loving the ability to use all the latest dev tools and technologies in Azure that were not possible in SharePoint previously.

The app model may not be applicable or possible to be used for everyone today and that is fine. It’s going to develop and that will change over time would be what I bet on.  But it is the right path moving forward.  This might cause some pain for people in the short term and I understand the frustration people have with changes like this. But I would MUCH rather have to deal with this change than be limited to a inferior set of capabilities in the longer term.  This is the right move long term (in my humble opinion).  Short term pain, long term gain.

Getting SharePoint out of the code hosting solution was the right things to do and I applaud the team for clarifying there position on this.

I look forward to the SharePoint Conference in March where hopefully (fingers crossed) we will hear more about the future of the new app model and how it will address its shortfalls today.

-CJ

Ep 10 of the Microsoft Cloud Show – News on Azure, Google Compute Engine and Amazon and more

I can’t believe we just hit episode 10 of the Microsoft Cloud Show!  It feels like a mini milestone.

Episode 10 is jam packed with news and updates from the Amazon re:Invent conference, as well as news on the newly released Google Compute Engine & , of course, lots of Azure goodies too.

Go get it and let AC and I assault your ear buds.

Episode 010 – Latest news in the cloud from Microsoft, Amazon and Google

Thanks for the amazing support so far with the podcast.  We had > 7000 downloads over the past month or so which is astounding!

-CJ

Managing your Azure cloud costs with Kerrb

One of the big problems developers and organizations have using cloud services like Azure is the potential for the costs to go crazy if you don’t shut your dev, test or temporary Virtual Machines off. Sometime back Andrew Connell and I got talking about and had an idea for an online service that would help you manage those costs.  We talked with some people and found found loads of people that were concerned with using Azure and Amazon Web Services because of these cost overrun type of issues.

KerrbSo we decided to fix it …  Introducing Kerrb.

Kerrb is a SaaS product designed to save you money by automatically turning off Azure VMs that you forget about.  If you forget to turn off a virtual machine Kerrb will make sure it’s turned off on a schedule that you decide on.

Kerrb is still being built, but you can sign up for the launch list and be one of the first to get access when it is ready.  We will send you updates on how development is progressing and finally give those on the launch list the opportunity to sign up and test out the system when it’ ready. Also as an added bonus, if you are on the launch list then we will honor the pricing we have up on the site, even if we decide to tweak it prior to launch.

Kerrb will start small and evolve quickly as demand and feedback drives the product development. The high priority “Pri 0” [1]  feature is to turn off Virtual Machines in Azure if you forget, but we have a lot of other great features on the roadmap including adding Amazon Web Services as well as support for other leading cloud providers.

Keep up to date with developments and help us get the word out by:

  1. Signing up for the launch list
  2. Liking Kerrb on FaceBook
  3. Keeping an eye on the Blog for updates and news
  4. Follow @KerrbApp on Twitter

Have a read of a blog post Andrew wrote on the Kerrb blog here: Using VMs for Dev, Test & Show – Perspectives from an Indie Consultant, Trainer and Presenter

And something I wrote about Managing cloud spend in a development organization

We look forward to hearing your comments and feedback!

-CJ

[1] Pri 0 – Microsoft speak for the highest priority features in product development. You have to have all the Pri 0’s.

SharePoint Saturday Chicago #SPSChicago

Today I presented a 101 session on getting started with the new SharePoint App Model. It’s a lot to get through in 75mins, but hopefully it gave people enough to get started and try out building an app for Office 365 or SharePoint on premises.

Thanks to everyone that came along!  #SPSChicago is a great free event for people with a great lineup of speakers.

Here are the slides:

Introducing the Microsoft Cloud Show podcast

Microsoft Cloud ShowThe only place to stay up to date on everything going on in the Microsoft cloud world including Azure and Office 365.

Quite some time ago I bugged pestered asked my good friend Andrew Connell (AC) if he would be interested in starting a SharePoint podcast.  Given he is a busy guy with a lot on his plate he wasn’t so sure it was a good idea to begin with.  After all, we both said that if we were going to embark on something like this that we wanted to do it right.

We ultimately decided to broaden the shows scope to include not only SharePoint … but rather take on talking about the whole Microsoft cloud story.  After all, Microsoft are one of the largest Enterprise players in the market and there is A LOT going on in their cloud offerings.

The Microsoft Cloud Show was born.

Our aim with the show is to bring you news, information and commentary about all things going on in the Microsoft cloud world. We want to invite the listener into the show via way of audio and email questions sent in. We want to keep a consistent delivery of shows that you can count on.  Most of all we want it to be easily digestible and will try to stick to 30mins per show.

We are not pretending to be professional podcasters here and we will likely learn a lot along the way.  But we hope you will join us for the journey.

We are launching the podcast in iTunes (MS marketplace coming soon) with 3 episodes. These are just introductory shows and talk about our motivations for the show & then a background show on each host. We will be getting into the meat and potatoes in Episode 4 which is coming soon.  Ideally we would love to settle into a fortnightly show.

We would love to hear your feedback! We would love to hear about topics that interest you and that you would like to hear us address on the show.

In the meantime … please enjoy this screenshot from us recording our first show.

image

-CJ

Keeping up with cloud releases, Yammer vs. 365

A colleague Brendon Ford , and Office 365 MVP I would add, pointed me at a great resource today for keeping up with what Yammer are up to in terms of new stuff coming down the line.

http://success.yammer.com/product/releases/

This gives a reasonable amount of detail on the things they are working on and at what stage they are at.  I say reasonable because they don’t give hard dates on when things will release, but they do give some detail on what is coming.

Here are some examples that specifically relate to Office 365.

Better search integration…

image

Could this be the single sign on we are all waiting for?  Maybe not, but it seems to be getting better…

image

Office web app integration in Yammer for viewing documents etc…

image

This is pretty cool insight into what’s coming.

The question I have is:

When are we going to get this level of visibility into the core Office 365 products? Exchange, Lync and SharePoint?

One of the big issues we face with customers in Office 365 is having zero visibility into when things change.  We are not alone, Jeremy has a good write up about some issues they have faced with APIs changed unexpectedly.

If I were at Microsoft still here is a sampling of things I would be pushing hard for:

  • Release and Update schedule.  Notes on exactly what is coming and when (per tenant).
  • Dev sandbox.  Ability to ask for a temporary test tenancy on a particular “version” of the service/product so that customers and partners can test their other systems that integrate with Office 365.
  • Postponement.  The ability to ask for a postponement of an update for up to 30 days. This would give customers/partners time to test and fix up any issues.
  • Open dialog with trusted advisors.  Microsoft was built on the back of strong partnerships.  Having a group of trusted advisors with a Bat Phone to someone who cares in engineering would go a long way. This group wouldn’t need to be big, but would be of good people who get it. People who want to help MS get better, not moan about it publically.

Some might say:

“Chris that is all great, but surely they should just get their cloud model to work and not break stuff randomly!”

Yeah that is great if you think you can reach Nirvana.  However, that is an impossible goal. Especially with something as complicated as Office 365 and all of its constituent parts.  Don’t let some chump from the valley with bubble gum SaaS product tell you otherwise 🙂

I think with some of the above changes proposed that things would get A LOT better really quickly.

-CJ

Extending your Azure AD tenant to include Office 365 services

I learnt something today that I thought would be interesting to share in the hope someone else won’t need to do the research.

Say you already have Windows Intune or Azure AD already up and running and now you are ready to give Office 365 a go.

You have a couple of choices:

  1. Create a new Office 365 tenant
  2. Extend your existing Azure AD tenant and add Office 365 services.

The correct way to do things is to Extend your existing tenant and add Office 365 services.  If you have Azure AD already you are likely DirSync to push all your user accounts from your on-prem AD to Azure AD.  It makes sense that those are the same users you want to access Office 365 no doubt.

If you try and create a new tenant and then do DirSync to that tenant you will most likely hit issues with trying to push the same users to two different Azure AD tenants.

Extending is the way to go.

If you sign into the Office 365 management portal using your current credentials you use for Azure AD/Intune you will see a page like this:

image

You will notice it is saying that you are not currently subscribed to any Office 365 services.

So how do you go about adding those?

Jump over to the “purchase services” tab in the left navigation and you will get a selection of the various plans (aka SKUs) available.  In my case I picked the E3 –Trial.

image

This will then add the services included to your tenant. Once provisioning is complete you can carry on with the other tasks you might like to do like setting up Identity Federation (ADFS) etc…

It seems blatantly obvious now I have tried this and this is possibly hardly worth a blog post, but until now I had always started from the Office 365 side of things and had never looked at starting with Azure AD and adding Office 365.

Turns out to be dead simple 🙂

-Chris.