Saving a Fortune in Data Warehousing

UPDATE at bottom.


I just wanted to extend my congratulations to the team at Greenplum, and our joint customers at Fox Interactive Media – the folks behind MySpace, Photobucket, IGN, FOXSports.com, and a whole series of web properties that together represent one of the single largest audiences on the web.


All three of us announced today that Fox is running a massive production data warehouse built atop Greenplum’s data warehousing software on Sun’s Solaris/ZFS based OpenStorage platforms (a sea of Thumpers, to be specific). That is to say, open source software is at the core of one of the world’s largest – and most affordable – data warehouses.


Fox joins a series of joint Sun/Greenplum customers, from LinkedIn to the New York Stock Exchange, in looking to open source databases and innovation as a vehicle to drive better insight, faster decisions and more efficiency.

Which is to say, customers that are tired of proprietary vendors with a knack for raising license fees during economic downturns have a clear set of remarkably affordable alternatives. Based on commodity economics everyone can understand.


Congratulations to all involved!


______________________________


UPDATE: I’ve gotten a fair number of inquiries from folks wanting to know how the Greenplum/Thumper data warehouse discussed above prices out against its competitors – given that one recently announced proprietary entrant has suggested $15,000 per terabyte is acceptable to customers. My view is that’s a pre-bubble price, and roughly an order of magnitude too expensive in today’s market – and unlikely to garner more than headlines. But that’s obviously a biased view, I’d check with a few customers to find out what they want to pay.

27 Comments

Filed under General

27 responses to “Saving a Fortune in Data Warehousing

  1. Anonymous

    It’s amazing to see a billion dollar corporation get open source like this. Years from now, the simple idea that sharing innovation can only move everyone forward will seem obvious and it’ll be in large part to visionary efforts like your own. Keep rocking. Your innovations are energizing millions.

  2. Great! that’s real Sun tech making real gains possible for customers.
    But it does raise a question: when can we expect to get a CMT powered Thumper? It’s nice to
    see big customers like Fox and LinkedIn use stuff like this, but I mainly work with much smaller players – people who need a couple of boxes for redundancy, not performance, and prize reliability (aka Solaris/SPARC) over just about everything else.

  3. SunEmployee + ShareHolder

    JS,
    Good post, but I really need to admit something. The whole IT world is against Sun. Oracle, Cisco, IBM, HP, DELL, PwC, InfoWorld, etc.; all these guys are doing their bits to label Sun as a slow, unimaginative, dying company.
    This blog (and others) are good; The PR is fine. But their reach is very limited. Our PR blast will never compete with a NYT article or a Cisco FUD campaign.
    I really really suggest that you spend $100MM on a TV + RADIO ad campaign to communicate our message. We have the cash, let’s use it!

  4. god

    Was just at Oracle Open World, yesterday, in the city….quite disappointing…an old dinosaur on it’s last legs…and seeing the hp garbage running it…:) Sun is the future, faster, leaner, meaner, cleaner…so, that also begs the question mySQL World conference to bring mySQL developers together in one place to demonstrate to the world how much better mySQL is, and show off the great Sun Hardware that knocks the socks off the competition? 🙂

  5. Mike

    It’s great to see some wins.
    What really concerns me is the prejudice of certain people when talking about Solaris and Sun Hardware.
    I’ve already got a few Linux Admins to try OpenSolaris and they liked what they have seen (especially ZFS).
    Sun really should put more effort in marketing OpenSolaris to Linux people. I can’t do it all alone🙂

  6. Meil Towne

    How will this compare with Oracle’s Exadata system? He said $15,000/Terabyte, what will Thumpre/Green look like?

  7. Diego.

    Hi Jonathan and Sun’s, i’ve been quite happy since Solaris went open, and i really like the general usage of the lastest deploy in almost every term including the new desktop, Nvidia support, Adobe support, perfomance, scalability and a few minors like the administrative tools and flawless install. It would be great of the taskbar were not a taskbar, probably XFCE, just because all those stacks of applications is quite old and the system really is the platform for great deploys. i’ll be soon at the low level of the Os trying to deploy a ultrasmall VM, that has been one of my thoughts for a long time.

  8. This is really great news. I am sure that with Sun’s new commitment towards open source ideology. We will enter a new era of computing where everybody is empowered with the Open source technologies and looking at Sun’s efforts, there is little doubt that Sun will emerge as a leader. Great work & looking forward to more.

  9. Opteron

    The following article except from
    http://seekingalpha.com/article/97543-microsoft-s-ballmer-few-sites-can-live-on-ads-alone?source=yahoo
    shows clearly that SUN has no grasp of the future of the profitable segment of the IT industry-
    if the smartphone market today is a lot like the PC market in 1983 as Ballmer says and it’s going to be a billion devices market in 5 years from 125 million today, why is SUN turning its back on that type of growth? Don’t get me wrong, they are going to use hardware but they can buy it from IBM, DELL, HP and even from Oracle (Oracle is entering the hardware market if you are not aware of it yet!).
    [Ballmer and Winblad also discussed the mobile market. Ballmer is focused on the smartphone market, where Microsoft software is a major player. 125 million smart phones will be sold this year, he said, and that will grow to a billion in five years. He says proprietary hardware/software stacks (iPhone) will have trouble competing with the more open approaches, such as Microsoft, Android, Symbian, etc. He ads a caveat, however: users won’t trade off usability just to have different hardware choices. The operating systems must work seamlessly across different hardware configurations.
    “The smartphone market today is a lot like the PC market in 1983,” he said.
    ]

  10. Pete

    Why does HP have the deal to build and provide Oracles new servers?
    How did sun lose this deal?
    Why does sun keep losing market share?
    What happen to project looking glass?
    Why doesn’t sun produce a java phone?
    Why does the stock go down every day?
    When will the stockholders be rewarded for investing in sun?

  11. Frank Lewis

    Opteron, you are so right! Jonathan and company are missing
    the market opportunity again – entirely. This is very hard to
    watch as an investor……….

  12. It’s amazing to see a billion dollar corporation get open source like this. Years from now, the simple idea that sharing innovation can only move everyone forward will seem obvious and it’ll be in large part to visionary efforts like your own. Keep rocking. Your innovations are energizing millions.

  13. Arthur

    Hello,
    I am pleased to hear that you decided to put your Web and your proxy server into the WebStack. That going to be great and I am looking forward to use it.
    If you now build a MessagingStack out of Java Messaging system, you would be the first top vendor with an opensource messaging system competing against IBM Lotus Notes and MS Exchange.
    To you have those kind of plans?
    Thank you!

  14. Sun Customer + EX-Shareholder

    I think there is something some of you don’t understand… OpenSolaris, MySQL are free as well as NetBeans and OpenOffice. How long a company can survive if they spend money updating, developing and they got ZERO in return. If you want to make an application in VB .Net you must BUY Microsoft’s Visual Basic .Net to program it. If you want to make applications in Java you can use the (FREE) Sun NetBeans, (PAID) ORACLE JDeveloper, (PAID) IBM Eclipse, and also notepad included with all versions of windows. Even other companies get money from a Sun product when Sun doesn’t!!! UNBELIEVABLE!!!
    In my opinion Sun has taken the fast way to die, and with the financial institutions crisis that are ones who spend a lot of money in IT equipment, it will accelerate the decline.
    It is okay that Sun machines are good, but HP and IBM machines are as good as Sun’s are!!!
    I agree with Opteron, Sun should have done something like a phone or a handheld device, although a single model it would have been something which would give them some money.
    Well happily I don’t have a single share (now 2 bucks above pre-split price and yesterday a new multi-year low) and happily I’m not focused on Solaris OS but Java development.
    I know Sun products are good and very reliable as I got two machines at home and use them almost everyday with heavy load almost all the times for testing performance on software (basically web apps) I made, but I don’t understand how Sun can make money looking at what they do.
    Please Jonathan, make this company shine again! So I will buy shares then.

  15. a) Dear John, there is NO way to contact you, can you please provide a way on your blog or sun website, to reach you ?
    b) Letter to CEOs of companies:
    Dear Sir, In light of the current economic crisis, it will be better for companies to apply salary cuts, instead of layoffs. Such layoffs will depress the economy further, and the people who remain on board the companies will have to work harder, which will have much more toll on their body and mind. On the other hand, if there are salary cuts, not only will everyone be weather the storm, but also it will bring US salaries closer to those of other countries, and will have much more overall ameliorating effects on everyone.
    Best regards, Avinash
    Published at : http://people20.blogspot.com/2008/09/companies-to-announce-salary-cuts.html

  16. god

    Actually, it’s just hp hardware wearing the Oracle brand…I really like Solaris, and I think I’m up to Solaris 4.0, already…:) I’m going with intel, though for hardware…time for sun to wave the white flag in that segment of the industry…but Sun does make the best software in the industry, in my opinion…

  17. Feedback

    Sun is a very innovative company. I really really hope and expect it will come out from the financial trauma (very) soon and bring new vision to the technology industry. GOOG might have taken the best of Sun’s employees. Only its a matter of time all those guys fly back to Sun. There are lots of opportunities for Sun going forward in many areas. There is lots of bad news out there but there will be also equal opportunities elsewhere so Good luck to Sun and keep the hope going high.

  18. Dennis

    Concerning the update:
    the Sun Thumper/greenplum solution is even more expensive: 17500$ / TB: see http://www.greenplum.com/products/data_warehouse_appliance/
    or did I misunderstood anything?

  19. Hi Arthur, you write…
    "If you now build a MessagingStack out of Java Messaging system, you would be the first top vendor with an opensource messaging system competing against IBM Lotus Notes and MS Exchange."
    We do. Check out http://openmq.dev.java.net.

  20. Alton Hill

    I think Sun’s ZFS Thumpers are something like $1-2/Gb, so that’s roughly $1k-$2k per terabyte. Not sure how Greenplum prices, someone ought find out.

  21. Kevin

    How’s the Sun "cloud" coming along? One of the sweetest things GreenPlum have made is a cloud-compatible "map-reduce" SQL engine. I do hope you can morph your Network.com from a grid into a cloud (a la AWS) sometime in the next 12 months. Forgive me if I don’t hold my breath.

  22. Carles

    For Alton Hill:
    I don’t know how did you make the calculation, but I just taken a random Sun storage server and did a calculation.
    Server: Sun StorageTek 6540 Array
    Disks: 6 servers * 16 disks each server * 146GB each disk
    Web price: $217995
    6 * 16 * 146 = 14016 GB
    $217995 / 14016 GB = $15.553296/GB
    $15.553296/GB * 1000 = $15,553.296/TB –> Higher than $15,000
    And far more higher than $1000 ~ $2000 / TB
    I would like to know how did your calculations as I cannot understand it.

  23. To Caries

    To Caries – you’re picking the last generation storage products, and not what you’d use in a data warehousing/computational data warehouse.
    You’d need to look at the Sun x4500 or 4540. 48 terabyte drives for roughly $50,000. Or, $1k/terabyte. Roughly 1/15th the cost of HP’s Exadata.
    The 6540 array has a very different price point/market segment.

  24. Dennis…I was scratching my head also. I emailed Jonathan…he got back to me…and I am clear now. 15K per TB is just for the hardware to support Oracle DW…before software.

  25. Blog troll

    @eduardo: I think the earlier poster meant email, calendar, im…

  26. Jeremy Chappell

    To the Sun employee – actually I don’t think Sun’s message is quite as lost as you might think. Clearly I have some suggestions:
    Sun should make it’s branding more obvious on it’s open source software projects, lots of people don’t know (for example) that OpenOffice.org has anything to do with Sun. Sun have a great logo, and perhaps shouldn’t be shy putting it on their software. Some might say Java already does this … not everyone knows that Java is a Sun technology, also Java is so ubiquitous that quite often one stops "seeing" it (when was the last time you thought about Java? It’s like oxygen, it’s "just there").
    I also think Sun need to keep (and preferably) expand a range of "cheaper" systems, after all, Sun should be the "open source geek’s computer of choice", but that isn’t the case – that’s usually either an iMac or a MacBook Pro (and let’s face it, that’s weird when you start to think about it). Now I say this as this is really what the press understand. They normally don’t see big datacenters, where Sun’s boxes are. They just see desktops, and departmental servers (often the Dell PowerEdge type of thing). Clearly if that’s your view of the world it’s easy the think Sun aren’t relevant.
    I think Sun should keep doing what they are doing, but expand their activities on the small end, try and make Sun the "Semi-Pro Developer’s Computer of Choice" – that might even mean a laptop. Part if that is getting the message about Solaris/OpenSolaris clear, in the Sun’s corporate mind as much as anything. Personally I think OpenSolaris is much more digestible for those coming from Linux, and I think in reality the effort in this area should assume that those customers are coming from that background.
    This is where much innovation comes from, and open source has proven that it’s not always the big tech powerhouses that set the agenda. It would benefit Sun if those projects often started on Sun’s technology (and possibly made use of it – I’m probably thinking of ZFS here, but I’m sure there are other examples).
    Also wouldn’t hurt too much if when some "small project" achieved critical mass, there were a few Sun logos visible in the press reports. Perhaps there is an argument for making Sun equipment easier to spot – in the past Sun systems have had wild designs and zany colours, is this so bad? Apple’s systems are usually pretty easy to spot, so it is possible without making the system look bad (yes you’re spotting personal bias here). If this makes any sense in the bowels of Sun’s design department, a word of caution: make it consistent, if you’re after instant recognition give things a family resemblance and don’t change the design cues too quickly. For this "geek" market, think about how the machine will be used. For example how often will it be opened up? (Does a door make sense?) How often will it be moved? (Do handles make sense – just the box or the display too?) Will it sit on or under the desk? What should be included? (Are any non-obvious parts required for upgrading? Can those cost effectively be included? It’s not good when you open up a machine expecting to be able to add say a hard drive only to find you need a sled for it that you don’t have. Such customers are unlikely do always get all parts from Sun). Most importantly, what OS will the customer choose? What’s the experience of that choice like for someone new to the platform? (Did the media get included? Is the firmware upto date to run that choice? Are release notes available for that choice on that hardware). This isn’t easy, but I think it’s worthwhile. There is a reason Microsoft make "Express" versions of their dev tools, and it’s related to why I think Sun should spend some time and effort on this. As I already stated, I think the OS choice is likely to be OpenSolaris, might even make sense to preload it (and make sure there are no "bumps in the road")?
    But given how much Sun software I use everyday, and rely on, I want Sun to remain relevant, and to been seen as such (you don’t need to be either a shareholder or an employee <wink>).

  27. What really concerns me is the prejudice of certain people when talking about Solaris and Sun Hardware.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s