Switching Subjects

Until very recently, Sun’s been hard to find in the world of high performance computing (HPC). Back in the 2001 timeframe, we lost the recipe for the P in HPC – customers that wanted performance were no longer looking at small numbers of big systems (a traditional Sun specialty), they were looking to large numbers, clusters, of small systems. And in 2001, that wasn’t our focus.


But over the last five years, we’ve been investing to change that. Our Galaxy and Niagara product lines are just about the fastest growing products at Sun. OpenSolaris is beginning to catch a wave of adoption on small systems, and we’ve been doubling down on compiler optimization and language innovation. All with a focus on extreme efficiency/performance. If there were ever a time to reenter the market, it’s now.


Rather than mimic the competition, we started out examining the issues and challenges facing the largest HPC installations we could find. Performance was certainly the main priority. But there were others – and not what you’d expect if your idea of a cluster was three PC’s in a closet.


At three or four hundred computers, the challenges of building a cluster shift around quite a bit. Dissipating heat, sourcing enough power, managing software versioning or hardware failures, just to name a few. Get to three or four thousand nodes, and all of a sudden everything from weight (floor loading), to the bend radius of optical cabling, to massive software provisioning challenges, even the speed with which data can be moved around a room become critical factors. And that’s where we decided to focus our efforts, at the extreme – on the assumption it would one day become the norm (as so often is the case in this industry).


I’ve read quite a lot of feedback from pundits and analysts over the past few days, and wanted to be sure to respond to one item – from those who believe the high end supercomputing marketplace is small, esoteric, and has very slim profit margins.


The high end of the supercomputing marketplace is small, esoteric, and has very small profit margins – they’re absolutely right.


And like the world of free software (in which no one’s going to get rich selling to the open source community), no one’s going to build a profitable business selling to the academics and researchers who dominate the extremes of HPC.


That’s not the point.


The academic supercomputing community (there’s that word again) sets the pace for enterprise computing across the world – which has grabbed on to HPC for an array of real world challenges, from virus, disease, and drug discovery, to customer purchase pattern analytics, capital markets trading, energy discovery, dynamic resource management – you name it, it’s one of the fastest growing segments in the marketplace. Proving that what starts in academia, ends up on main street. Industry looks to academia and research institutions to understand the innovations that enable breakthrough scale and performance (just ask Linus – who, come to think of it, still hasn’t responded to my dinner invite… I hope it’s not my cooking.)


What We Announced


In Dresden, Germany, earlier in the week, we announced the Constellation System – a set of generally available building blocks any customer, educational or commercial, can use to build from a few teraflops system, to more than a 2 petaflops system. As a part of this broad announcement, we unveiled a few component elements – notably…


Our commitment to the rise of OpenSolaris in the HPC community – joining Linux as a reliable, resilient platform for petaflops scale systems (those capable of executing a thousand trillion instructions per second). What’s driving preference for OpenSolaris? Legendary support for huge memory configurations, integrated virtualization, DTrace and the ZFS file system are probably the biggest drivers – but support for ROCKS, a price tag that says FREE/open source, and the fact it’ll run on any server built are a big help, too. Success in HPC is a very high priority for the Solaris team, and an area of investment for us and our partners. (And no, this doesn’t lessen our focus on Linux – if we can combine licenses, it’ll amplify it.)



Second, we unveiled an integrated 48 blade rack that supports all volume microprocessors, AMD, Niagara and Intel – in the same rack, with
standardized I/O. Picture on the left. We also announced a new blade, Pegasus, designed purely for HPC grids. No seatbelts, no redundant anything, just raw compute performance.



Third, and most importantly, we unveiled Project Magnum (at right), an absolutely massive (3,456 port – click here to find out the significance of that number) infiniband (IB) switch – designed to alleviate a ton (three tons, actually) of the cabling, weight, expense and latency nightmares saddling most supercomputing facilities. This one innovation, courtesy of the extraordinary Systems team led by chief architect Andy Bechtolsheim, allows those with serious computing needs to dispense with a massive amount of complexity and expense. The largest competitive IB switch in the market today is 288 ports – so you’d need a lot of them (with an equivalent proliferation of support nodes, cabling and complexity) to match Magnum. In an industry where size matters, we’re feeling plucky. (We expect the economics behind Magnum to prove out around 420 nodes – so even if you’re building a little grid, Magnum pays for itself.)


Our view is we can reduce by a factor of two or three, at least, the cost and complexity of building a supercomputer – in an academic or commercial environment. Bringing general purpose systems, and volume economics, back to a market that was starting to turn proprietary. What the Constellation System allows for is a transition from this first picture…



To this, a vastly simpler, lighter, easier to manage/maintain Petaflops scale HPC installation.


.


Three tons lighter, three times less expensive to build, a fraction of the cabling and vastly simpler to manage. And at up to two petaflops, I’m quite convinced we could spank Bobby Fischer…


For those interested in the details behind our win at the Texas Advanced Computing Center (TACC). Here’s what they’re running:


TFLOPs: approximately500 TERAFLOPs
Magnums: 2 (>2000 4x IB ports each, expandable to 6,912 ports)
Thumpers: 72 (1.728 PB)
Metadata storage: STK6450 RAID (9.3 TB)
Tape storage: STK SL8500
Storage/Data Management: SAM/QFS
Racks: 82
IB NEMs: 328
Pegasus blades: 3936
Aggregate memory size: 123 TB
Number of cores: 62,976


Total racks: 94
Approx footprint: 2,037 sq ft
Approx power: 2.4 MWatts
IB cable length: ~14 kilometers


To put that in perspective, their computing facility will be about half the size of an NBA basketball court. Not exactly small – and in fact, likely the largest on earth.


And for those curious as to why we settled on 3,456 ports…


____________________________



Begin forwarded message:
From: Andreas Bechtolsheim
Date: June 28, 2007 6:58:59 AM PDT
To: Jonathan Schwartz
Cc: John Fowler
Subject: 3,456


We implement a 5-stage fabric, and with a 24-port switching element
the maximum number of ports is n*n/2*n/2, or 24*12*12 =3456.


Other Infiniband switches in the market today are 3-stage fabrics
and they have n*n/2 or 24*12 = 288 ports.


Now you can build a 5-stage 3456 port switch with 12 288-port switches
and 288 24-port leaf switches but you end up with 300 boxes occupying
about 456U of rack space or 12 racks, and 6912 cables.
We use one double rack with 1152 cables, so it is 1/6th the space,
1/6th the cables and 1/6th the weight.


On Jun 28, 2007, at 6:36 AM, Jonathan Schwartz wrote:


so – why 3,456 ports?


—————————-


and last, but certainly not least – if you’d like to try a supercomputer on an hourly basis, just point your browser to network.com… we’ve made a ton of progress in the past 6 months…

43 Comments

Filed under General

43 responses to “Switching Subjects

  1. Dennis

    1. Will you offer Solaris based notebooks in the future? That s a product that I´ ll buy. (big markets. high profits)
    2. Will the TACC run Solaris or Linux in it?
    3. What about x86 based routers with 10Gbit Ethernet and Vlans?

  2. Anantha

    Good post shedding light on Sun HPC strategy and approach. I’m now anxiously awaiting how ‘real’ progress is being made on the financial front. All in all Sun in my view is making all the right bets. Good luck.

  3. Paul

    This excites me as a minor shareholder (albeit major for me) whose children are using computing power like I couldn’t have imagined at the time of Sun’s share price collapse. I will continue the gamble, loving especially the idea that Andy, despite his billions, is obsessed with this project. What the heck, I’ll even buy more on a dip and make the solemn vow, often broken, that I will not sell on a spike.

  4. Peter

    Andy makes it appear so simple that one must wonder: Why didn’t anyone think of if earlier?

  5. as

    network.com is down (at least from germany).

  6. Austin

    Jonathan thats something what team Sun has done to be really proud off and its good to see Sun being back in HPC business.
    Congratulation SUN for this extereme engineering achievement.
    Hopefully the Linus dinner is next on the card

  7. “The connection has timed out
    The server at http://www.network.com is taking too long to respond.”
    Do I win a prize?

  8. observer

    Hmm, (DTrace link above to) OpenSolaris is down – DNS, but no ping, no web. What’s that site running?

  9. This image http://blogs.sun.com/jonathan/resource/BadGrid.tiff should be converted to a GIF or PNG so the magnitude of the difference can be seen visually without resorting to an external viewer. Great summary post otherwise…

  10. Hi Jonathan, I’m the intern in Broomfield in charge of running this summer’s national internship program and I just want to say “WOW!” I’m excited about working at such an innovative and leading company with strong leadership.
    If you’re interested in seeing what we’re up to, please check out our blog at http://blogs.sun.com/intern
    Cheers!
    Mike

  11. Few years ago attending a seminar largely influenced by one of your competitors, a company associated with the color blue, the speaker, a company’s Fellow, used several subliminal messages along the lines of “… you should consider vendors that are going to be here in the long run, say in the next 5-years. Lets talk now about Sun …”
    I was so pleased today when I met with representatives of said company and I was able to point to the turnaround that Sun is demonstrating via an open culture and concrete solutions such as Constellation, Blade, Opteron-based products, Magnum, BlackBox, ZFS, DTrace, OpenJDK, JavaFX et al.
    Congratulations to Sun’s employees, you, Andy Bechtolsheim, Jeff Bonwick and the various teams for an open and results oriented approach to computing. I think there is a good change Sun will be here in the next 5-years. Now I hope you have time for that anticipated dinner and we will see soon a boot-able ZFS as part of the Linux kernel.

  12. Jan

    Sites Like this Helps Businessman to post their ads on the net wasting few dollars. It provides options for businessman to maximize their profit with less capital, nice. Exchanging ideas would be better. Godbless.

  13. Gumby

    hey jon you stand little chance of fending off the IBMs, HPs, Dells, to name a few others for the slioe of the pie. HP merged Compaq to survive even tho they are much bigger than your little corporation called Sun. I bought your shares at 3 1/2 and sold at around 6 and made a nice profit knowing that your corporation will never see the sunlight again. If you cant beat the hairball OS called Windows, then you have no chance… I never see a Sun product let alone touch it yet. Java? Solaris? .. never knew if I am using it on the Internet. Apple rocketed up. And you? nothing! Just nothing, yet… I aint seen nothing. You are basically blogging to your own employees, am I right? Are you trying to talk to your shareholders , yet? I am a shareholder, still. I am shrugging my shoulders and stratching my head. Duh?!!! You are not talking in layman’s terms. That is why your stock is still in the doghouse!! Arf arf

  14. Gumby

    Hey Jon
    If this is a rerun, no apology for not adding up your math quiz right…
    If you develop a software or a hardware that can automatically protect my precious date with all the pictures, spreadsheets, downloads, pdfs, htmls, etc without my having to manually click the stupid check boxes to backup the damn zillions of files in my giga drive, you will make a mint for Sun. You will really introduce your brand to the masses who never heard of Sun. They only knew Apple and Intel. AMD is finally getting into the spotlight. YOU?, hahahahahhaha you are nobody!!! you are just a geek lost in the cyberhood. You dont know how to develop and sell a hot item that everyone would run over themselves to buy or even stand in long lines. It is nice to have a petaflop available to the few luck bastards not the masses.
    can I make a suggestion? Why cant you develop a software that automatically save everything I create in same harddrive or other drives that must be permanently connected not loose USB drives. I see many hard drive in the bays and i only have a damn drive perching on it. I have to buy a Windows Pro or Windows Server to be able to use the damn RAID feature . I never knew that until recently.
    Your head is just in the clouds and you are not really hungry for dollars at all…. You are just a laid back hippie wannabe. I was a hippie back in the 70’s. I do everything they did and it made no damn difference with what I am thinking . I dont care what others think or get pissed off about. I just happen to have a friggin answer for every one they care to think about.
    Jon, you better damn think outside the box starting tomorrow…

  15. Gumby

    You are scared of my comments, right???????????

  16. Stenley

    Sun has made StarOffice as OpenOffice.org to OpenSource. Thanks Sun. Now Microsoft is on the turn, to make its MS-Office OpenSource.
    Sun has made Solaris as OpenSolaris to OpenSource. Thanks Sun. Now Microsoft is on the turn, to make its MS-Windows to OpenSource.
    Sun has made its Java as OpenJDK to OpenSource. Thanks Sun. Now Microsoft is on the turn, to make its C# with .NET to OpenSource.
    Sun has made NetBeans OpenSource. Thanks Sun. Now Microsoft is on the turn, to make VisualStudio/VS.NET to OpenSource.
    Sun widely contributes and supports Mozilla Firefox. Thanks Sun. Now Microsoft is on the turn either to support and contribute to Firefox or to make its InternetExplorer to OpenSource.
    Sun widely contributes and supports the OpenGL-implementation Mesa. Thanks Sun. Now Microsoft is on the turn either to support and contibute to Mesa or to make Direct3D to OpenSource.
    Sun plans to make in the future SunStudio with the C, C++ and Fortan to OpenSource. After that, Microsoft is on the turn, to make Visual C++ OpenSource.
    Sun contributes to GNOME. Thanks Sun. Now Microsoft is on the turn, to make its Windows-API to OpenSource.
    Now Sun plans to make Solaris Cluster to OpenSOurce. And Microsoft … do nothing.

  17. Gumby: Your choice of handle is perhaps more revealing about you than, on the evidence of your comment, you’ll ever be capable of imagining.
    You got a cell phone? Ever browsed the web? Ever bought or sold anything on ebay? If you’ve done any of those things, you probably touched a Sun product without ever even knowing it.
    In this blog entry, Jonathan explains how Sun’s stock got in to the doghouse, and how Sun might see sunlight again.
    If the only audience’s for Jonathan’s blog were his employees, his blog would be pointless. He could just use email within Sun. His blog is aimed users of Sun products, potential users of Sun products, partnerts, financial analysts, anybody else interested in how the network brings people together, and, probably lastly, Sun employees.

  18. nobody

    I’m buying SUNW because I trust you Jonathan. I have strong beliefs about FOSS, so strong some might say I have faith. Yet the first company I bought shares in is Sun. Why? Must you ask? GPL Java, Sparc, OpenSolaris, OpenOffice, and that picture of the Neo 1973. You’re awesome! Don’t let us down.. I really really wanna see a GPLv3 OpenSolaris, if its possible. I’ll keep buying as long as you keep progressing and raising the bar. Here’s to Open computing! Cheers.

  19. nobody

    Gumby, my dad was a Hippie. Long hair, played Jimi Hendrix out of his garage, smoked pot, fast V8, 2 1/2 kids, dog, and a nice house. What’s your point? He’s the hardest working man I know bar none.

  20. Ryan

    Incredible, good job Sun. It would be good if Sun would beef up the servers that run it’s own websites. Sometimes I just can’t believe that I have to wait 5-10 seconds for pages to load — especially when I’m trying to show my boss how great your servers are. It’s nice and fast at night, but during peek business hours it crawls. It doesn’t show off the serious computing power you have, and can be frustrating to use. We’re on a dedicated fiber connection and other websites load quickly. I don’t think it is our connection.

  21. chris

    For those of us whose idea of a cluster is our three pc’s in a closet, well I have to say we got no gripes either cuz damn, your stuff is fast. That x2200 sure kicks butt over the HP stuff I’ve worked with. I can see why Mr. Smugmug digs ’em. Yow. I think you guys are on the right track with the latest initiatives, we all have a need for speed and besides yr boxes; yr technology is tearing up towards the statosphere as well. Now if you could just throw some of that blade goodness in a Blackbox and drop it off in front of my house for a trial period, I’ll promise to get the Sun Logo tattoed on my arm. For the present, however, I’ll rock on my Sun boxes in the closet. Now, ‘scuse me while I kiss the sky…after all seatbelts are overrated……

  22. Arvinda

    Jon, coming straight to the point:
    We want an OS free of patent threats.
    We are willing to buy anything as long as it is clean and straight.
    With OpenSolaris and OpenJDK, we are almost there, but for the marketing.
    Java was made for Jini use. J2ME runs on mobiles. But people don’t know that as much – “SUN is shining in the sky. Stanford University Network? What is that? A group of professors? ”
    And we really want UltraSPARCs and the lot, put more than one of them in a box, instead of bothering too much about clock speed.
    —————————–
    What you’ve done with HPC(3456), do with the PC – 3,4,5,6 CPUs.
    And RAM _is_ cheap.
    —————————–

    And for once, for Heaven’s sake, use the _established channel_ of distributors and vendors.
    Direct sellers are booted out by cartels and politicians.
    Mighty Dell had to bend.
    No one here knows about SUN grid (IIRC, it’s around since 2003).
    No one knows about Sun Global Desktop, dammit!
    You should really do some marketing here. There are tons of Java programmers out here. What are you waiting for?
    Jon, I hate you and your company.
    You make the breakthroughs, year after year, don’t market them, and let others copy and earn.
    W H Y ?
    Just as a piece of superb design backed by a robust people-participation methodolgy, Java needs more fame with non-tech humans. What to say of OpenJDK, all the Java code out there and OpenSolaris.
    Finally, stick to your promise, GPLv3.
    And, let us all make Heyyyyy!! while the SUN shines.

  23. Al

    I have some questions about server growth according to IDC and Gartner.
    Both are looking for no better than single digit (anemic?) growth for
    the rest of the decade. But set against that, and seemingly starkly
    inconsistent, are the following: 1)The growth in Youtube, Joost, etc and video in general is causing a capacity crunch in the internet 2)Most of the data center companies (Digital Realty, Equinix, etc.)
    report a boom in data center construction 3)In a recent seminar conducted by Gartner it was estimated that half the world’s server installations would be obsolete in 2008 because of power and cooling requirements 4)We all know about Google, Microsoft, Yahoo, etc. and
    their massive expansion of server infrastructure and how they are scurring around to hydroelectric sites and other cheap sources of power to build server farms; and 5)take off growth in software as a service and HPC. With all that IDC and Gartner stick to their single digit server growth forcasts. What gives?

  24. W. Wayne Liauh

    I don’t know if anyone noticed this: An e-mail was sent from the CEO at 6:36 “am”, and it was promptly replied by one of Sun’s top employees at 6:58 (OK OK let’s make it 6:59) am.
    Can’t see how in the world this company will not succeed.

  25. Mark

    In the area of InfiniBand HPC clusters, Constellation is an interesting concept. SGI released a similar concept with its Altix ICE. It will be interesting to see how IBM, with BladeCenter H, HP, with C-Class, Dell, combined with Cisco, Voltaire, and QLogic respond.
    The 48 blade chassis also explains why Sun never attempted to buy a dense rack computer vendor such as Rackable or Verrari.
    I would like to see Sun also offer the 48 blade Constellation chassis as an SP play (with low-power chips), now that Rackable seems to be struggling.

  26. Your article on supercomputing technologies and enterprise solutions, and the differences between the cabling, was really interesting. I”m living in Bucharest, Romania, and in various reports alot of the big-picture problems seem to be about system management – energy sector, heating-system sector, energy-efficient sytems, water management, waste management, etc. Technologies that simplify systems here and at the same time save money seem like a very relevant solution.

  27. Kevin

    How is your Sun Google partnership going? Is there any chance that Google might buy some of these Magnum switches? Or maybe some Niagara 2 processors to put into custom Google boxes? Just as the Google-ization (cheap PC clustering) of servers caused the SUNW dip, I can see a time when Google’s adoption of new Sun servers (or processors) accelerates your resurgence. Great stuff.

  28. Rob

    I agree with Dennis..why not come out with a laptop that people can buy and actually use Solaris on instead of jumping through hoops to find one that might work. Are Sun techs still coming to do onsite support with Windows laptops? Are you and your execs still using Windows laptops?
    Anyone can build some monstrosity that about 0.005% of customers will need or want. It would actually take skill and intelligence to come up with a Sun Solaris x86 laptop..so how about it?

  29. I would like to know more about parallel computing and clustering.
    Can anyone one help!🙂

  30. Kevin

    A Sun laptop idea: License Palm’s Folio as a Sun Ray client. All it would need is a smart card slot and a firmware upgrade? Somebody once said “the network is the computer.”

  31. Phenom

    Aside from the diatribes posted earlier today, this is EXCITING news!! Bechtolsheim, Fowler and others within Sun have shown repeated proof that Sun is growing, (re)gaining relevancy in the marketplace and innovating in various ways beyond other high-tier companies, such as Dell, IBM and HP. The Sun continues to rise/shine! Simply glorious, Jonathan. Stay the course! Onwards to JavaFX innovations! I want to hear/see how Sun’s renewed commitment to microelectronics is coming along. (I also want to know if Linus ever accepted your dinner invite…)

  32. Tim Scanlon

    Running it on Linux is a performance loss, it’s too bad that’s what they’re using here. I can’t really see the point of doing that at all.
    As for running Solaris on laptops, I use a Dell D810 at work running Open Solaris with the wireless when I want to use that, it works fine. I have no need for Windows, and like it that way too. FUD is what keeps most people on Windows, nothing more.

  33. I see the latest product announcements by Sun Microsystems SUNW as a huge step forward. Many posts on this thread are missing Jonathan’s point. For the last few years I have heard “Innovation Matters”. It does and as those words have been spoken and emailed, the divisions and employees of Sun Microsystems have listened, developed, designed and executed. All technological innovations have a trickle down effect thereby affecting competitors and causing innovation in the computing marketplace as a whole. If Sun had not produced the Niagara chip with 8 cores, then would AMD have produced a multi-core chip as soon as they did? And, would Intel have then followed on? All of this innovation at the core level, starting at Sun, I believe, has caused a trickle down effect to these two competitors, who then feed upon themselves at beating each other with more processor cores – Intel and AMD. And, then Sun will most likely raise the bar twice as high with Rock when it arrives and I can’t wait for that announcement.
    Those skeptics who easily have thrown stones here on this blog thread – do you realize now, that any multi-core processor you buy – be it AMD or Intel probably has the advantage of being in your HP or Dell computer by virtue of the time to market increased by Sun innovation?

  34. As for people thinking Supercomputers are not directly useful in their lives and wanting data security – consider this: In a few years, Wimax and Wifi will cover many cities. Solid state flash based hard disks are decreasing in price. Any company can use Open Solaris or Linux to build a laptop where as the laptop uses the Wifi network to download only part of Open Solaris or Linux that the user desires. And, all data can be kept or left on the supercomputer. Hard disk crash? Just boot again from Wifi with a RAM disk in memory, your files are still save on the supercomputer which, by the way, is running 10,000’s of users all remotely connected via Wifi or 3G. Sun could deploy this in a year if it decided, by turning a Sunray with an SSD disk into a laptop. I hope they do. And, I will buy one. And, I hope it costs $300 or less so everyone can buy one. You could boot Solaris or any flavor of Linux you choose all from the network. If you choose to “keep” that OS as you are going remote and off network, just write it to the SSD disk and your’re set. When you need super computer power and speed, you’ll have access to a super computer or perhaps many on the net. Need a word processor or spread sheet? Use Open Office remotely or locally. Or, use Web 2.0 apps which are in development now. It all trickles down and eventually, we will all have access to a supercomputer on the net from perhaps our wireless mini laptop running “what if” scenarios which may save our lives realtime. The innovation you are experiencing on this blog – is just the beginning of it. Thank you Linus, Jonathan w/ Sun Microsystems and Nicolas Negroponte for raising the bar.

  35. Computing Guy

    I’m happy to see you are entering this market. I was even more excited to see that you offer programs such as R and FreeMat over at network.com. These are programs I use, and you might see me buying some of Sun’s services.

    One suggestion that would probably help you immensely, if for no reason other than marketing purposes: please provide plug-ins for Netbeans for these programs, in particular for R. Eclipse has a mediocre plug-in available. I much prefer Netbeans but it’s limited in its support of languages. The R user base is huge. Providing a cross-platform IDE would be good for everyone.

    You could even include a link to network.com as well as a connection from within Netbeans.

  36. Alan

    Jonathan,
    Sun introduced Project looking glass. Why is it that Microsoft seems to be better at commercializing Sun ideas? Now MS has Vista. What does Solaris have built it?
    Solaris is great with container technology. Why can’t Sun put more effort in making a GUI for Solaris container configuration and monitoring. Come on! Put 1 and 1 together and isn’t that going to propel Solaris to greater heights? Is it so difficult? It’s about time Sun capitalize more on GUI type management capabilities for containers. Make containers and zones easier for people to configure using a GUI and make a GUI that allows one to monitor the zones and I guarantee you, people will love Solaris even more!! Don’t leave it to the admin/CLI geeks. Give IT managers a better tool for management.
    Dismayed

  37. Very informative article. Sun needed to clarify its position in the HPC space for some time now.

  38. I wonder if you can develop a system for laser transmission over the air without the ‘fiber optics’ just in case someone wants to set up a petaflop computing platform without so much ‘glass’ cabling, and with some distance betwen the nodes of the grid.
    It could be useful to wire conections in open space without air, and the perfomance of the transmissions would be brutal.
    Just like Intel is planning on their Satellite networks.

  39. Ivy

    Dear Sir
    Sorry for bothering you. I am a student from Msc Marketing, University of Birmingham in UK. I am conducting a research about corporate blogs. I really need your help to complete my research. The link is my on-line Questionnaire, please fill it for me.Thanks a lot
    http://www.my3q.com/home2/172/yiyien3/20502.phtml
    Best Regards

  40. Bill

    network dot com is down in Boston, MA too!

  41. Phenom-non

    Phenom, sounds a lot like you’re a stockholder doing some pumpin’.

  42. Rob-a-not

    “Skill to come up with a Solaris laptop”???
    Uh, Dell’s got a few. HP’s got a few, too.
    Bet it even runs on a Mac:/

  43. Hi, i live in brazil in goiania city. In my job here we use the servers on sun and are the best. Congratulations!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s