The allure of the cloud for faster deployments of new capabilities with potentially lower equipment and labor costs are hard to turn away from. For small businesses that either don’t have the capacity or don’t wish to grow their data center, the cloud can be the way to go. I’ll first focus on evaluating software as a service (SaaS) solutions. The assumption here is that you will, at a minimum, evaluate SaaS against a local deployment of the solution. In the near future, I’ll focus on how you might evaluate using Infrastructure as a Service (IaaS) - using the cloud like a data center.

My example evaluation below will use a simple table in a spreadsheet. The table can contain as many columns as you have solution options. I’ll break up the evaluation into two sections, one for items that we can put a dollar figure on, one that will contain a rating with a possible multiplier. The multiplier services as a weighting mechanism where the higher the multiplier, the more important the feature or capability is. When using a rating (say 1 to 5), larger is better. I’ll draw from my earlier posts, “Benefits of Cloud Computing” and “Challenges and Risks of Cloud Computing” to help come up with the rating criteria.

The final decision is rarely completely encapsulated within an evaluation but there may be more benefits than you expect. The evaluation sheet supports your recommendation to management and by sharing your approach, and what you consider important, you will foster discussion, and, hopefully, agreement within your team and business partners about what is truly important. It will also serve as a tool to gain alignment of the entire decision making team and others that are impacted by your decision as you begin change.

When entering costs in the first set of criteria, you may want to calculate the total cost over a period of time. Many companies capitalize their equipment (and possibly software) over five years and what might look like a advantage for SaaS is really due to the short term benefit of not having the initial hardware investment. The half-life of many business systems is five years so going beyond five would be pushing this model pretty hard considering the assets are fully depreciated by then. You might also want to break down the costs for various factors to give visibility and confidence in the numbers you come up with (separate line items for each, like computers, racking, etc.).

For factors that receive a “score” rather than a cost figure, use a scoring system from 1 to 5 (a larger range beings to lose meaning).

Add calculated columns to the example to give you the weighted score (wt X Option score). It’s assumed that you will also add rows that help you score feature functionality for each option that you are looking at. My table below contains the baseline items used in any any system selection and it’s expected that feature/capability will be added to this list depending on the tool being evaluated.

Cost Evaluation

Factor

Notes

Opt A

$$$

Opt B

$$$

Hardware Investment costs
What is the cost to deploy this application service within your own data center? If you capitalize your hardware, the annual cost can vary. This value should include things like computers, racking, network, adding air conditioning capacity, upgrading power, storage and SAN upgrades, backup hardware and software. You’ll need to multiply some of this if you plan to deploy a highly available solution that has load-balancing or failover. Also include any data center improvements needed. If your infrastructure is virtualized, and you are going to be sharing the app with other virtual machines, split the cost accordingly. Don’t forget the cost to install all of this along with virtualization software.

For SaaS, the value is can be zero but it’s possible that you’ll need to increase bandwidth of your network and reliability since SaaS could put more strain on this part of your infrastructure. If SaaS allows you to lower costs on your data center, add a return column so that it’s reflected in the score for SaaS option by using negative values.

Annual hardware maintenance costs
Include the cost for hardware maintenance for all of the items listed above. If SaaS allows you to lower costs elsewhere in your data center, reflect that here with a negative value.

Data center operations cost
For on-prem implementations, include the annual dollars to operate the equipment used to drive this app. Include electricity, air conditioning costs (power and maintenance). Many of these costs increase over time (even power) so account for this in estimates over time. Also, add the cost to backup and manage the backups, administer the systems and manage the network.

For SaaS, the value is normally zero but if you need to build out a faster more reliable network for your SaaS implementation you should reflect that incremental cost.

Software Costs
For on-prem solutions, include the acquisition cost of the application software. This may be annual value if you can capitalize it. SaaS might also have an acquisition cost.

Software maintenance
Most software deployed on-prem includes a maintenance contract that provides for upgrades, security patches. If not included in Data center operations cost, include the annual maintenance costs on network software, backup software. Include the cost to to perform upgrades on hardware and software by local or contract employees.

Support costs
For on-prem solutions, include the annual costs to support your environment including project managers, technical and management.

Don’t be fooled: SaaS could require on-prem support as well, including staff to manage the app and coordinate upgrades, downtime, etc.

Usage costs
For on-prem solutions, higher usage doesn’t normally drive higher costs unless it requires more hardware for compute and storage. (for example invoices per month). Another example might be network costs if it’s a mobile apps.

Depending on your SaaS contract, volume could impact costs. Look closely at the contract for any variable costs due to volume. For example not only could there be transaction cost, but also networking costs if your usage exceeds certain thresholds. Depending on your application, storage could be another pay as you go cost. If you are integrating your SaaS solution with on-prem applications, network usage and API calls could add up. Some SaaS vendors lowball their solution so they can make money on usage fees. Look for this and if needed, ask your procurement staff to push for a predictable annual cost since CFO’s dislike unpredictability.

Business Continuity
Evaluate costs associated with delivering high availability that meets your needs. Compare what it would cost to deploy a local solution and with each SaaS provider. Some SaaS solutions include this as part of the cost.

Training Costs
Include the costs for training. Many SaaS apps have an advantage because they’ve designed for usability. SaaS vendors also prepare training to be delivered virtually, further lowering costs.

Attribute Scoring

Factor

Notes

Opt A
Score

Opt B Score

Business Agility
Look at the number of days or weeks to implement the solution and then rank the results with all the options. Don’t forget to include procurement time for all options. Obviously lower is normally better. Business Agility is one of the benefits of cloud computing but not always.
20

Interoperability
How interoperable is the proposed solution with other systems in your technology architecture. How much work will it take to make it interoperable? You may need to add to the cost section of the evaluation for making the solution interoperable.
20

Mobility and Collaboration
Mobility and collaboration is a key enabler for companies. Score your solution on how well it performs here.

Many SaaS solutions will have an edge here because a) they are new, and b) they are normally accessed over public networks enabling easier mobile access and allowing partners to easily access it.
20

Business Continuity
When considering on-prem solution as well as SaaS, evaluate your ability to deliver high availability. Depending on the criticality your weighting will change. Also, include costs for downtime multiplied by expected recovery time. Recovery can vary depending on how system failures are handled by each solution.
Take a look at the Business Continuity section of my risks article.
10

Security
Evaluate each solution’s security. This may force an uncomfortable review of internal security. Smaller companies can’t afford to have around the clock perimeter security, monitoring, reaction teams, and secured data centers. If needed, look at the solutions’ ability to support two-factor authentication.

This is just the tip of the iceberg though. Do some research applicable to your requirements and government regulation for the type of data that will be stored. Each solution will vary case by case. You may require third party certifications such as ISO 27001 and FIPS 400 or SAS 70. If you are a multinational corporation, there may be other legal requirements that you should consider.
Read the Security section of my risks article. Also see “Legal Implications” section of my risks article.
20

Data Interchange
Score each solution as to how easily it is to get data out of it to use in other solutions, or migrate to a new solution. Some consider this “lock in” but many on-premise solutions can actually be worse in terms of “lock in” than SaaS. Don’t focus on “lock-in” but rather how open to data access each solution is.

(Investing lots of money in on-premise hardware can be a big “lock in” in itself.)
30

Application Flexibility
Score how flexible each solution is. Some on-premise applications allow you access to the code or APIs to modify the application to suit your business needs. SaaS vendors may also offer APIs and complete development environments where you can add onto or integrate with their applictions (SalesForce.com is a good example).
20

-- Chris Claborne

Cyberthoughts

Sunday, June 10, 2012

Evaluating Software as a Service

References & Related

No comments:

Post a Comment