Webcast of a keynote I just delivered at the Open Source Business Conference (OSBC) can be found here. I did have a fair amount of coffee before the speech, which someone in the audience suggested was evident in the pace at which the speech was delivered. Listen for yourself.
There were quite a few startups in the audience – and given that we’re beginning to turn our focus to recruiting those startups back to Sun, I thought I’d live up to my (now delinquent) commitment to publish an email I received from Marc Andreesen about his startup‘s experience with “Linux on Intel” vs. Solaris. Here you go:
——– Original Message ——–
Subject: followup
Date: Thu, 02 Feb 2006 15:51:28 -0800
From: Marc Andreessen
Organization: Ning Inc.
To: Jonathan Schwartz , Anil Gadre
Hi guys — below is the writeup of the data I spoke about on stage yesterday.
We (Ning) are fine with you all using this however you want.
Marc
Ning Server Platform Analysis
The following data is based on an analysis we did in support of the
implementation of our production environment. Ning had deployed a beta system
based on a commercial Linux distribution running on whitebox (aka local system
integrator PC clone) AMD Opteron servers. This infrastructure is hosted in our
own cage at a commercial co-location facility. We pay a monthly fee based on
the total sqft of the cage as well as the total power delivered to the cage
(measured in amps).
The space and power costs are blended averages for our area (San Jose, CA):
Monthly space cost: $27.00 per sqft
Monthly power cost: $17.00 per amp
A 4-post rack (or cabinet) occupies approximately 20sqft. Each rack/cabinet
has approximately 40 rack units (RU; 1RU = 1.75″) of useable space. Thus:
Monthly cost per rack: 20 sqft * $27/sqft = $540
Monthly cost per RU: $540/40 = $13.50
For purposes of our analysis, let’s assume all servers are of equal cost:
Server price: $3,000.00
Next we need to consider the operating system for the server, so we need to
include the purchase price plus any maintenance fees. We compared a commercial
Linux distribution (enterprise class) vs Solaris:
Commercial Linux annual subscription fee: $900.00
Commercial Linux per-incident charge: $380.00
Total: $1,280.00
Sun Solaris subscription fee: $120.00
Sun Solaris per-incident fee: $0.00
Total: $120.00
Given that we are in a hosted environment which imposes certain constraints
(total space and power available), the acquisition cost is only a part of the
equation. As Paul Harvey would say “this is the rest of the story”.
Our co-location provider imposes a per-rack limit as to how much power they
will provide. In our case this is 60A. The rule of thumb is to only load a
circuit to 80% of the total capacity. Thus:
60A * 80% = 48A useable per rack
Now we need to determine how many systems we can fit in a rack. We know we
have 40RU available and 48A of power available per rack. But how much power do
the systems require? Let’s see:
Sun x2100 (model 175; dual-core Opteron): 1A
Whitebox AMD (2x AMD Opteron 248): 2A
Intel dual Xeon: >3A
To determine the maximum number of systems we can deploy per rack:
Intel: 48A per rack / 3A per system = 16 systems per rack
Whitebox AMD: 48A per rack / 2A per system = 24 systems per rack
Sun x2100: 48A per rack / 1A per system = 48 systems per rack.
Given there are only 40 RU in a rack, the real limit for the Sun systems is 40
systems per rack. Next we need to determine the monthly cost of operating the
systems:
60A per rack * $17 per A per month = $1,020.00 per month
Monthly rack cost: $540.00
Intel (@ 16 systems per rack):
power: $1,020 / 16 = $63.75 per month
space: $540 / 16 = $33.75 per month
total: $97.50 per month
whitebox AMD (@ 24 systems per rack):
power: $1,020 / 24 = $42.50 per month
space: $540 / 24 = $22.50 per month
total: $65.00 per month
Sun AMD:
power: $1,020 / 40 = $25.50 per month
space: $540 / 40 = $13.50 per month
total: $39.00 per month
Looking at a straight line analysis of the combined costs of server + OS +
space + power over 36 months we end up with:
Intel + commercial Linux:
$3,000 + (3 * $1,280) + (36 * $97.50) = $10,350.00 over 3 years
whitebox AMD + commercial Linux:
$3,000 + (3 * $1,280) + (36 * $65.00) = $9,180.00 over 3 years
whitebox AMD + Solaris:
$3,000 + (3 * $120) + (36 * $65.00) = $5,700.00
Sun AMD + Solaris:
$3,000 + (3 * $120) + (36 * $39.00) = $4,764.00
Based on this analysis the Sun solution is less than half the cost of running
Linux on Intel hardware.
A key consideration: data center space is a limited commodity over which we
have minimal control. We can “reserve” space for expansion, but this requires
additional expenses. A solution that allows us to maximize the space we
purchase is a significant advantage, as this allows for much more cost
effective growth.
——————
ps. note comments have been turned on.
pps. And to that point, The Register just published an excellent analysis – and given the increasing scrutiny under which Gartner and IDC now find themselves, it’d probably help their credibility if they, like the financial analyst community, started disclosing revenues they receive from the vendors they cover. At their scale, vs. the smaller boutiques, I think it’s going to become an imperative.
And personally, it may end up being a determinant of whether we’re willing to do business with such firms. Transparency’s a good thing.
What I Couldn't Say... 
There’s one error in Marc’s comparison – it’s not like-for-like on the OS costs.
The $120 Solaris contract does not provide support, only Patches/updates. The Linux cost specifically includes the cost for a support call.
In order to turn this into a true like-for-like, the Solaris contract would be $240 per systems (yes, the entire charge is less than the “per-incident charge” for Linux – go figure!)
The end result is the same – Solaris still comes out a long way in front.
Scott.
I manage the installation of server platforms for a living. I have been hitting the space/ power/ cooling barrier for about 8 months now. As a floor of IDC space fills up there is quite a high up front cost to build extra room at IDC standard. Your example is colo, for a company doing it’s own hosting, the costs of increasing chiller plant is quite high as well.
So, basically, the biggest issue is the cost of commercial linux with it’s (admittedly steep) annual support fee. Including the different hardware options in the equation, albeit valid, confuses things to the negative side for all solutions except Sun AMD/Solaris. It’s just as valid to run the commercial linux on Sun AMD as it is on whitebox AMD, hence it would come out as: 3000+(3*1280)+(36*39) = $8244.
Still steep, but a ~$1000 save over whitebox AMD with Linux…
Doing a little bit of research, I see “a commercial Linux” actually has two versions. ES and AS…and of course to favour the comparison to Sun/Solaris an expensive version of RedHat (standard, AS) is put against the most basic form of support for Solaris. That is indeed a factor 10 difference. But is AS really needed? I’d wager not given the rest of the story, so ES will do as well with a “basic” support option of “only” $349/yr. Different numbers indeed. So lets pick that, and then see what “commercial linux” on Sun/AMD would do:
3000+(3*349)+(36*39) = $5451..this is “only” about $1000 more expensive than Solaris on initial purchase. Although Sun AMD/Solaris still comes out cheaper, it has turned into a much fairer comparison all of a sudden…but lets analyze the actual support costs of Solaris a bit more. Support for Solaris is indeed $120/yr for the basic option, but that’s per socket…and we’re working with dual socket systems. So Solaris support is $240/yr all of a sudden. Oops, slight oversight in the calculation. So lets work that into it too, so the Sun AMD/Solaris solution comes at:
3000+(3*240)+(36*39) = $5124…they’re closing in on eachother…especially if you have a quick look at what those “basic” support options actually are. But that’s just assessment of what you need. It’s up to the customer to decide if the difference in the offered support is sufficient to save $100/yr between Solaris and RHES.
Anyway, IMO you two industry buffs are confusing the issue. You claim “Solaris is cheaper than Linux”, while it isn’t. Solaris _can_ be cheaper than a random commercial Linux distribution…but only if you pick the wrong version for your needs, and only if you settle for the most basic support options.
But that’s too long for a blog title, indeed.
Jonathan, comments are IMHO crucial for a blogger to have a real conversation and it is great you did that! Now I wish all Sun’s bloggers could follow your example. There are some folks who seem to be afraid to have a real confrontation. For me getting opinions of other people back through the comments was a very good experience, it gives you other views on the topic you’re discussing. It also increases the relevance of the blog, because it shows that you stand behind your opinions. Great news with the cost analysis btw 😉
And the list prices diverge even more if you want 24×7 phone support on the OS. For a single-CPU server, over 3 years, the advertised OS support costs are: Solaris – $1,080 vs Commercial Linux – $7,497. This takes the 3-year total running costs to: Sun/AMD – $5,485 vs Linux/Intel – $14,007.
Solaris is indeed a smarter server solution than Linux.
That notwithstanding, Ning is entirely useless.
hi jon,
Good info — Thanks. 1 Question.
What I don’t get about the server price is the identical price $3000/- . I believe that whitebox AMD(which can ship dual core Opteron now/nearFuture) can be less expensive(out of the box) than sun.
Also, it will be nice if you give a very brief pricing scheme for Sun Opteron Servers so startups can quickly decide for themselves.
Thanks again for turning the comments on,
BR,
~A
I just started a new job in 2006. We are running Web environment on Linux, Solaris and Windows platform. Each server has been replace in the last 6 months so we have brand new machines: HP DL380G4 and Sun V440.
As a Unix system administrator, I have my preferences for Solaris over Linux for enterprise management. The number of different platform we have to manage is not easy, because I have to deal with different, OS and packages.
Basically, the Linux servers are for Apache, the Sun boxes for Tomcat & a proprietary app server and le Windows boxes are SQL Server databases.
Linux was chosen because Apache was already available and configured with mod_jk for Tomcat. They were having problem doing it on Solaris in the past (because there was no realy Unix administrator).
On the development side, we have a couple of old Sun servers and we have started using VMware ESX for Linux VM. The problem is that Solaris x86 is not supported on VMware ESX for the moment.
Because all servers are really new, it is not the time to switch and buy new hardware, but I would like to find a way to normalize my platform to Solaris (at least get rid of Red Hat Linux).
What could be the best way to convince my bosses?
While this is good and reflects the significant progress made on the software side of SUN the hardware side of the business needs to get beyond the data center mindset. The ivory tower data center culture is what is holding back the next level of commodity computing.
Cool calculation. I think this shows especially how ridicoulus current Linux Support plans are.
BTW: Debian Linux on Sun Hardware would have an even better pricing. 🙂
I can only concratulate, slowly the Solaris technology catched up with the user friendlyness of Linux, and the price is more than right, with the interesting support plans from Sun.
Gruss
Bernd
Re Gartner: check out RMH’s view from Inside Bourton Group: http://rmh.blogs.com/weblog/2006/02/daytoday_inside.html
The $$$ numbers are good, the technology is great – so how do I go about renting a Sun box? I don’t see any hosting companies out there with your T or X series servers. And I don’t see a public Sun Grid either? Please do that “network is the computer” thing and sell me computing, not just the computers. PS Love the enabled comments too – fortune favors the brave 🙂
This comment is about the Itanium sales forecasts issue that Mr. Schwartz noted at the end of his post. I’m active in advertising business and viewing the Itanium issue from a different perspective than IT experts.
IMHO by introducing IA-64 and hoping that their massive existing installed base will see it as the next big thing, Itanium gurus basically ignore the simple lesson from their neighboring x86 product line that the volume is more important than architecture.Looking back at 90’s ,Intel marketing should now learned with so much pain that IA-64 deal is different “upgrade” deal than moving from 486 to 586 and name it Pentium, or selling Xeon or limiting the competitive field by creating too many brand names and products that are slightly different from each other from technical point of view but fit well into a specific market sector.
Regarding Itanium, I think they should have taken the death of HP Itanium workstation line as a serious alarm, showing the lack of interest of technology leaders.
As just a mere observer, it’s very strange to me to what extent an intelligent company can be unaware of the secret of its own success, the way Intel was. Or maybe they were thinking so big that it’s incomprehensible for someone like me.
Maybe Intel saw people buy Pentiums and Xeons and take it by mistake that’s the result of their successful branding or maybe even because they have advantages in price/performance (Which they didn’t have several times in their history, see Alpha, Sparc and PPC product/timelines) alas the fact was at those times people needed a CPU that run their legacy win32 applications and each time they find better alternatives to run win32, they begin moving toward them. It seems users –for the sake of integration- are following Windows on every chip it chooses to reside, even times the target platform is not a conventional PC or server; the latest example of this can be studied by comparing sales of the latest Palm smart phone that runs windows mobile with its Palm OS based counterpart in future.
However, my main intention of writing this comment is putting emphasis on a less direct consequence of Itanium failure in market. Most of the companies that counted on Itanium were Microsoft’s close partners. Remembering the fact that most server or PC makers are under indirect pressure from Microsoft business model to choose Windows for their servers or desktops (Remembering how Be OS died because of this strategy), Microsoft will feel a lot lonelier in the server market if Unisys or Bull fail, or HP loose more PA-RISC market share to SUN or IBM. Is it true? Will it affect Windows server market? Many thanks.
Hello –
It’s great that comments have been activated.
About the price of the AMD Whitebox and the Sun server, it first seems really strange, but Sun has very affordable entry-level AMD servers:
http://store.sun.com/CMTemplate/CEServlet?process=SunStore&cmdViewProduct_CP&catid=48589
Also, I found that comparing the lowest Sun solution to the most expensive Linux isn’t really fair.
Customers prefer real comparisons than abusives one, even if it seems more expansive 🙂
-Thomas
PS: sorry for the bad english
Ferdinand O. Tempel, the X2100 is a single socket system (see Opteron 175 at AMD’s website or somewhere on sun.com). The $120/year is absolutely valid!
Nice to see comments at last 🙂 Nice job!
Interesting overview on the costs of implementing different solutions in a datacentre – was pleasantly suprised to see Sun and Solaris come out in front! What would be an interesting comparison would be a TCO analysis of not just whitebox vs. Sun and Linux vs. Solaris – but Sun vs. Sun! How do the offerings from Sun running on Opteron processors compare in power usage, performance and heat output compared to other offerings from Sun running on UltraSPARC, for example the T1000? Factor in the purchase cost, and which is a lower total cost?
Jonathan, a couple quick points:
1. I would recommend that you keep your commenting functionality activated on your blog. Right now you are simply Jonathan telling the world about Sun, not Jonathan actively engaging in discussion with the world about where they want/need Sun to go.
2. My company is in stealth and is considering using your Niagara servers to power our platform. Who do we contact about discussing a partnership? The partnership would not only involve servers, but also a co-branding initiative that is pretty damn interesting for everyone involved.
3. Why does SUN not have a hosting alternative available? I checked into your grid computing, but most of our work, and the majority of the tasks on the web today, require hosting far more than offloading processing. Frankly, your server architecture has been built leveraging similar assumptions as it is optimized for threading. It just does not make sense to me. It would be phenomenal to be able to have the option to buy hosting on Niagara servers from a trusted provider such as SUN and, as we grew as a company, potentially using other services from SUN such as SUN Grid. I have talked to some folks in your organization about this and confirmed that it is not the craziest idea. Thoughts?
you better watch out. people might call this a “real blog”… now it has comments.
“Ferdinand O. Tempel, the X2100 is a single socket system (see Opteron 175 at AMD’s website or somewhere on sun.com). The $120/year is absolutely valid!”
Yep, I noticed. Which brings up the question why a comparison is made between dual socket/single core systems and a single socket/dual core system. Only to show that a single socket opteron draws less power than a dual socket box? Not really a surprising result, is it? Apples to oranges, etc.
All of the comments seem to be addressed to the issue of server cost of ownership. What about the issues raised in your keynote? I must be a little slow, because I can’t get my head around the concept of Sun becoming a profitably growing company again by open sourcing software *and* hardware. I admit I’ve struggled with the concept of the profitability of open source software, but it becomes all the more untenable, IMO, when you extend the concept to hardware. I know there are some shining counter examples, but I believe they are simply the exceptions to validate the rule. That being said, I’m open to the counter argument and would love to see some business models – even trivial ones – that demonstrate continued growth and profitabilty.
Colin: How about if, rather than seeing “open sourcing” as “giving away free” it’s seen as “payment at the point of value” instead of “payment at the point of acquisition”? Successfully collecting that payment is of course an issue but seen in that light the various open source business models seem less extreme.
Given your advocacy for more transparency, “Transparency’s a good thing,” any chance you can help persuade Anil Gadre’s product managers for Java Desktop System to provide their roadmap?
There’s been some excellent work identifying opportunities for improvement in regard to customers’ and developers’ Solaris desktop experience, but I’ve not seen a corresponding roadmap which would indicate with “high, medium, and low confidence” what features might be added by Sun and when and what features will be left to the ISV and open source communities to address.
See Darren Kenny’s “Solaris Desktop Gap Analysis”
http://www.opensolaris.org/os/community/desktop/communities/jds/roadmap/solaris_gaps/
Thanks,
John
groenveld@acm.org