Controlling cloud costs

It is almost a cliché by now – whatever you think your run costs are going to be in the cloud – you are wrong.

Here’s some practical tools

• Cost modelling should forecast out to a rolling three years.  The estimate out at that horizon will be wrong, because circumstances will change, but this gives you the discipline to adjust the forecast every time those circumstances change.
  

• Tie your engineering designs to your cost model to your invoicing.  Again it is almost a cliché: Our design documents show that we should have 50 servers in the cloud. Our cost model assumes we have 100 servers and we are being invoiced for 200!
  
  Well your design was done during the implementation, wasn’t updated to AS-BUILT and hasn’t been kept up to date with changes since. Your cost model was based on the design but doubled to allow for the staging environment during migration. Your actual environment includes dev and test environments, and that training environment that got spun up and never spun down again, and those extra servers had to get added after the recent upgrade to deal with performance.
  
  Firstly the billing gives you an opportunity to get you AS-IS documentation up to date. And if you trust your design document it is much easier to analyse where the costs are going and negotiate with the stakeholders if you are trying to save costs “you can’t spin down the training environment – we use that to replicate support issues!”. 

• Factor in organic growth, If the growth department says that you will be growing members by 10 percent a year then reflect that (and the associated costs) into the cost model.  If actual growth is 20% then everyone will forgive you and wear the unexpected cost. 

• Factor in transactional growth. Even if you don’t grow your membership you may well have continuous transactions to factor in. If your system has been running for 5 years and the core database is at 10 terabytes, then you should assume it I growing at 2 terabytes per year until you have better analysis.  Most databases don’t have mechanisms to purge, consolidate or archive. They often rely on the entire history being available. This growth compounds, so after 3 years you are looking at 75% storage growth

• If you have your designs up to date, and you are modelling growth, then you can meaningfully assign new costs to projects. 
  
  If your analytics team wants to scoop up a bunch of data for some new insights then you can assign a meaningful cost to that.  We want people making use of the data, but we also want well informed decisions. Being able to assign a meaningful running cost to an initiative isn’t going to reduce the quality of the go/no go decision.

• Model your backups closely!
  
  It is not uncommon that the backup practices are a lot more liberal than the information security policies require.  The policies might make different statements about DR, document restoration, retention periods for different information asset classes etc – in practice however it was simpler to just “backup everything all the time” and be confident all the cases were covered.
  
  In cloud the cost of backup storage becomes a running cost. Also the nature of different ‘as a Service’ platforms change the equations – for instance if your document management systems (like SharePoint) is retaining all versions then what is your actual requirement for separate document retention?

• Beware of immutable backups.  I love immutable backups, they are the ultimate in ransomware protection. But be very careful in how you configure them in the backup regime. Backups take up a lot of space and if you later decide that you want to change the schedule – you may have to wait for existing inventory of backups to age out. Let’s say you decide to backup every 2 hours, a perfectly reasonable decision, and you decide to keep these for a year, probably excessive, but the sort of quick decision that can be made during an implementation and refined later. 6 months on and realise that your backup storage has blown out. You look at the schedule and realise that most of these backups are only required to be held for a week. But then you realise that the ‘do not erase’ flag was ticked.  You have 2190 of these backups built up over the last six months. You are going to have to wait for 12 months before they are flushed out of your invoicing.  Sucks to be you.

These tips won’t solve all your cloud costing headaches, but they will put you on the right path!