As we prepare for our next RealTime CrunchUp on November 20th in San Francisco, we’re seeing if anything an acceleration of the phenomenon known as RealTime. Startups, cloud platform vendors, the open standards community, and virtually every software and hardware category are being refreshed and reinvented in the new model. And while there are many familiar players talking and to some degree walking the RealTime walk, some have been busy for years building and deploying the fundamentals of this “overnight success.”

A few weeks ago, I traveled to Las Vegas to attend IBM’s Information On Demand conference, and took the opportunity to sit down with Big Blue’s Steve Mills, Senior Vice President and Group Executive of the IBM Software Group. In English that adds up to Steve being The Man at the helm of IBM’s embrace of Web Services, with the software group accounting for one quarter of IBM’s $100 billion business. While others have partied down on Web 2.0 and its various social themes in perhaps a more outward facing way, it turns out IBM is very focused in the same areas, albeit with an eye toward leveraging its deep relationships with the enterprise.

If raw information accounts for the lion’s share of useful data, IBM’s investment in analytics and “mining the nuggets” suggests the company’s history of eating its own dog food with early realtime technologies like Notes and Sametime will bear fruit as IBM begins to share its best practices with customers. But what of the TwitterSphere, the social media stream of micromessages — does IBM see what they call sentiment analysis as a different animal than the internal data that runs corporate systems? Mills says yes, citing IT and regulatory concerns about closely held data leaking across the firewall.

But when I cite competitors such as Salesforce who are rapidly harnessing social relatime data with their CRM and SFA applications, Mills says IBM is doing the same thing with its Lotus Connections technology when customers ask for it. “You have to allow these scenarios to exist if the company wants that scenario,” Mills says, noting that some companies would rather enable what they can’t stop and take advantage of the wave without being overwhelmed by it.

Mills says about half the IBM population microblogs today, using incremental additions to Sametime built not on open source but what Mills delineates as open standards. He sees microblogging as a way of triggering people finding people, with swarming around topics and events producing positive collisions. When I suggest he and his troops are real adopters of realtime, he suggests the word “fanatics.” This is a long-time IBM strategy of amortizing its development (and acquistion in the case of Lotus) costs with internal usage, then rolling the results of that experience out to its army of customers.

Mills was not sure companies would be ready for this free-wheeling technology in the face of regulations and data governance, but was pleasantly surprised when it took off like a rocket. “Businesses were more ready for it than we had thought,” and Mills sees a fairly profound shift as RealTime takes hold. “What’s acceptable to people today is probably something beyond anything they thought they would be doing ten years ago.”

As avid users of the stuff, IBM’s view is that the speed of the stream is not so much to be coped with as to allow the interactions to occur. “There is tremendous value if your company has a culture of problem solving…. There are cultures that are very internally competitive… that don’t want to share,” Mills debates. And he knows where Big Blue comes down on this, saying there’s no company in the world more prolific in its use of collaborative technologies.

IBM continues to invest in predictive analytics across structured and unstructured data, and IOD attendance was strong as companies began to flesh their IT muscles as the recession appears to be abating. The efficiencies of information management hold up well in tough economic times, and IOD featured many sessions and demonstrations of SPSS and earlier acquisition Cognos working together to reduce municipal costs by predicting crime and fraud and shifting resources to combat those drains on budgets.

For Mills, the experiment begun 10 years ago formerly known as Web Services has been wildly successful. The mix of service oriented architectures (SOA), Web 2.0, social media, ubiquitous bandwidth, and low cost microprocessor technology allow us to instrument and analyze the world around us as we’ve never done before. Is it good for IBM too? Mills says IT has always made its own market. “This is the most adaptive tool that mankind’s every created…. Who would have thought this would have been a $1.2 trillion industry?” That could double in size in 10 years, he predicts, as operating and capital expense from other unrelated areas get converted over into IT.

For years we’ve been told the key to the future is the Open Web. And for years it’s been true that taking the open path eventually pays off. You can’t deny the power of open technologies to disrupt the incumbents, whether they are operating systems or carriers or the media in general. Arguing about what constitutes open can be entertaining, but in a world where realtime dominates, we are starting to move on to capture the value of open for ourselves, in the private Web.

As social media clouds become more resilient, we are trusting them more. Twitter lists are a robust signal that the company has moved from keeping up to encoding the value of its network. We won’t see many new stars as lists proliferate, but rather a better sense of how to model the new media forms that micromessages enable. Boiled down to vertical niches, lists are the instantiation of a way of looking at the Web, a kind of Yahoo 2.0 based on people aggregation rather than sites or topics.

But what value do these lists have in raw form? It feels like a Wikipedia page, where you learn not to click on hyperlinked words for fear of getting lost in ever-cascading tangents based on ever-more generic topics. Instead, you rely on the intelligence of whoever constructed the page, scanning for clues as to authority, serendipity, social characteristics worth capturing for yourself. Two problems: the list architecture is splayed all over the place, and we have no tools for harvesting the value.

Of course, we’re just seconds away from the onslaught of third party takes on the subject. Surely we’ll see interesting aggregations of the Top 100, the best, brightest, sexiest, etc. We’ll recognize the familiar names and ratify their positions in the new marketplace. It’s a marketplace that will have its own hierarchy, its own Oprah, its own politicians, police, and underworld. And with all that will emerge its own underground economy.

What is the Private Web? It’s the private place only we know about (or think we do.) It’s the place where our deepest fears and instincts combine to produce the hunches that drive our lives. As a parent of a teenager, I’ve seen my hunches evolve to reflect the rapid pace of social media and my daughter’s use of it. Twitter is nowhere on her radar, Facebook serves as a gas station where she pauses for fill ups, and video chat and IM are interrupted only for food, homework, and periodic audiences for the purpose of fundraising for road trips.

All of the most important parts of her life are conducted on the Private Web. This is not a good or a bad thing; it’s just what it is. I can sense her world but only by inference — more by the difficulty in understanding parts of it than any rational tool such as asking questions or withholding permission until information is volunteered. I feel like Steve Wonder, blind but with some heightened power of perception that slowly carves out information from the resiliency of the difficulty of it.

Take this exchange:

When are you going out?

In a bit.

[some narrowing to an hour, say 4]

Who are you meeting?

Uhh, Amy and mumble and whatever. [co-conspirator, someone I don't know, and no mention of whoever I want to know about, usually boys]

So when will you be coming home?

I don’t know I’ve done my homework [usually not] and it’s [whatever day it is] and I just want to have fun with my friends, Dad. Jeez. [Obfuscation of the length of the excursion to allow for audibles at the line of scrimmage to do all the stuff I should be concerned about]

Lengthy negotiation based on the hunches I’ve collected.

What’s important to understand is that my daughter already knows exactly what she wants to do and has modeled it to the best of her ability to predict the future, online and through the social media framework. Facebook tells her where the opportunities lie, texting confirms or augments those clues, voice is only used to ratify plans once the permission map has been drwan and pre-tested for potential disruption. These kids are really good at this stuff, and we are learning more from them than they from us.

Some conclusions gleaned from observations of the Private Web:

• It’s not about Twitter, it’s about what Twitter has triggered.
• Realtime is the best way to get what you want, before defensive measures can be deployed.
• Friends are important, and particularly a deep bench. If one friend becomes overexposed, you switch to a backup.
• Texting is the prime channel, then video, followed distantly by email and IM.

To unpack, last in first out. Texting is tied to a hard coded identity, credit card, device. This provides two-way leverage, where the parent (boss) can monitor and require timely feedback, while the child (you) can meter out pseudo-information to keep you happy while navigating largely unseen on the digital network. It is much easier to project a sense of action, reliability, and strategic positioning via social media when you can downplay the value of moving physically through space and time. Foursquare will hit a wall once adults (companies) discover the existence of these breadcrumbs. Foursquare will counter by virtualizing location.

Just as location will become more editorially enhanced, so too will the role of the team in social hierarchies. It’s much more useful to have interchangeable friends or partners, so that the parent (company) knows there will be some coherent continuity regardless of conditions on the ground. People profess to value collaboration, but the strongest connections in the social graph are between groups of overlapping friends who in aggregate add up to a rational team but don’t require hardcoded roles. Put in nightcrawling terms, it’s “OK, I helped you out last night, tonight you’re my wingman.”

Realtime, of course, just plain wins. You may get away with almost a few times, but once people are onto you, they’ll start serving the ball to the weakest point. Realtime is inexorable because our sense of timing adapts to each generation of realtime and soon gets frustrated with how slow it is. How many times have you interrupted someone’s argument because you know what they’re going to say? How many times have you skimmed a post or even a tweet for some clue that it’s worth whatever miniscule time you’re now tuned to? That’s why video is right there after texting, because a picture is still worth a thousand words. “If looks could kill…”

And first but not least, Twitter is so not the point but what it has created is. The key to the Private Web is notification, not the actual content. The social signals that enable or disable connections are the new PageRank. It’s not a link but the ability to see the metadata that describes a link’s immediate value that’s valuable. My daughter uses speed to get off the phone or out of range before I can pin her down for the next number to reach her at. The data is sitting there in plain sight but where it is is obscured. Understanding her social graph in realtime is what we want to know and what she wants to obscure.

The Private Web operates on deeper emotions and instincts than we are accustomed to acknowledging. Where do I want to go? Who do I want to be? In the case of my daughter, how do I get to be who I want to be? The keys to the Private Web are shared, not at a location but via implicit and dynamic permissions to access the stream in realtime. Those who signal their understanding of this deeper value pool will implicitly advertise their value, and encourage us to request permission to share with them. Those deeper conversations will contain higher value as we trust those who share them to keep them private to the group who values them.

Twitter may not support conversations very well, but it provides clues to where the Private Web exists. These conversations live in the cracks between the public stream and direct messages, hidden either by obscurity or purpose. As they become more useful, the tendency will be to make them public, but in doing so they will lose that unique quality of trust and value. Instead, these private conversations will grow, until everyone is participating.

I spent this week at John Battelle and Tim O’Reilly’s Web 2.0 Summit, and thoroughly enjoyed myself. Partly because MG SIegler was on fire, doing a hybrid live blogging/news analysis stream that let me mine the hallway conversation, and mostly because John Battelle poured a ton of research and preparation into a relentless pursuit of the “story” — namely Twitter. John asked the questions we all wanted asked, leaving plenty of time to relax and enjoy the moment when the Big Guys finally showed their cards.

We’ll soon see how well these cards are being played, but for now the one fundamental fact is that, as with Noah, there are two of everything. Twitter and Facebook. Google and Microsoft. Scoble and Scoble (the one at the top of the thread and the one at the bottom.) This is very important because it undercuts the rationale for throwing FUD at the BigCos by turning us all into Missouri. If there are two locked trunks (varying degrees perhaps or not, doesn’t matter) then we can make up our minds for ourselves. The result: a valid user contract.

The reason a valid user contract is important is that it shifts the argument from who’s doing what to whom to what are we agreeing to do with our data. We may have argued over the value of Track on an individual basis, but this week’s announcements underline its value in aggregate. The hostility over the embargo of realtime search eased when FriendFeed opened things back up with realtime conversation; now the Facebook acquisition is being used to restart the notion of exclusion. But it has much less force once we notice that, just as with the Fail Whale, FriendFeed will continue until it morphs into a Facebook hybrid. We will continue to have a choice, and will validate those, preferably two, who continue to scratch the aggregate itch.

Viewed through the lense of the power of two, big memes like scalability and market force take on a different hue. What does it matter to me how good Windows 7 is in the abstract, as a revenue splash for Microsoft or as a funding mechanism for whatever the company is trying to do in the WebOS era? Not a lot, but certainly much less than in the context of OS/X, WebKit, iPhone, Android, etc. In context, Windows 7 drives the motion of the two forward. It means that Google Voice drives Apple to drive AT&T to open the door, while driving the Android ecosystem to firm up its AppStore and bake out its alternate proposition. It’s like what Tim Berners-Lee is doing, playing the US and UK governments off each other in a race to document transparency.

This counter-surge disruption draws its power from the elasticity of the network and the cloud computing model. Next to this inexorable self-correcting dynamic, the politics of both FUD and silence fail miserably. If the sound sucks on a Startup School webcast from Berkeley, wait a half hour and the chorus of Fails prompts a fix. If you have to leave to drive to the event, get someone to patch the feed into Ustream so you can monitor from the car. If the car doesn’t play iPhone app audio, get a car that does it right. These micro-decisions in ones and twos make a small ripple; in a cascading social wave, you get Twitter and FriendFeed and Facebook and Google and Microsoft. And in that world, you get a new media model.

The mistake (if that’s what it is and not the fuel for progress) that’s made in identifying any one node as directly competitive with another is that the least important aspect becomes the defining metric of success. In fact, FriendFeed is wildly successful because it does not compare directly with Twitter or Facebook in scale or “user friendliness” but rather creates the ability to do things in the context of those successes. If Twitter lists make some of those FriendFeed processes possible in the larger platform, it only accelerates the value proposition of the aggregate tools. The barrier to entry is in finding complementary roles for new players beyond the first two or three.

Evean Williams’comment about there being room for both Twitter and Facebook may have been good politics, but it also reflects the larger reality of the power of two. Minus Twitter, Facebook remains trapped in its internal domain, without the escape route opened by Adsense to allow Google to achieve scale to create pressure on Microsoft. Without Facebook, Twitter has no vehicle for moving into the larger company’s private (and enterprise) market. FriendFeed is as much the structural backbone for Twitter as it is for Facebook, and Williams’ comments about not seeing two-way synchronization as particularly useful between the two clouds was the least perceptive comment he made at Web 2.0. In fact, the power of two will mandate full sync whether he likes it or not.

Forget the noise about FriendFeed and its founders being elites or two engineering-focused or whatever Silicon Valley spam you hear. If you’re looking for signals about where this thing is moving, look to the voices that are making the turbine spin. It says nothing about the various entities that make up the technology environment, negative or positive, to deride any one node. Commercial, social, open, pay-per-view, whatever. The most disruptive thing I saw on stage at Web 2.0 came from ComCast’s Brian Roberts, with the beta on-demand service that erases the boundary of the TV and the computer. This has gone wll beyond the politics of exclusion, the swiftboating of any individual, company, ideology, format, or layer of the stack. That won’t stop the sniping, but nobody really cares.

I went to a birthday party this weekend where I ran into a Facebook guy, a smart guy who asked me to go off the record. In fact, the whole party was supposed to be off the record. So I ignored the off the record part by insisting that I already knew the thing I was being told, and then I told him on the record what I thought was about to happen for Facebook. This being my usual m.o. which is to insist on not being NDAed except for things I don’t really want to talk about anyway, like the next version of Office.

That way, I can just make up what I want to have happen, never breaking any confidence and yet at the same time painting as plausible picture of assumed reality that it is hard to deny or in fact slow down. So here’s what I told the Facebook guy: the company has at most 3 months window to absorb FriendFeed and open the Everyone News Feed, and if that’s true (again, making all this up) then the messaging about how that’s going to work must begin immediately, like in two weeks. Then I went home and saw MG Siegler’s post and Scoble’s remake of Frenzy on FriendFeed.

OK, so I was off by two weeks. The noise about the death of FriendFeed is already off the charts, and the proof is in the lack of rejoinder from the FriendFeed team. As in: of course FriendFeed is not dead, and here’s what we’re going to do to remake Facebook in the next few weeks. Actually, that is indeed the message from Twitter, what with Lists and ReTweets and the return of Track just as soon as, well, sometime next year or so. No need for FriendFeed real soon now, because these Lists will soon be carved up and meshed together into an authority stream by the 3rd party developers.

Siegler nails the one provable negative about FriendFeed Facebook edition which is the lack of any innovation moving forward. The one thing the FriendFeeders didn’t get in under the wire before the money arrived was stream splicing, the ability to mesh together lists into an authority stream. Is that coming soon from Twitter either? Nope. So the antidote to FriendFeed stasis is Twitter right up until stream splicing is enabled… by who? As of right now, that would be via the FriendFeed APIs. If Siegler/Scobler are right, the danger of doing that is iterating on a dead API.

Here’s where the FriendFeed is dead rumor falls apart. OK, you’re Facebook and you’ve just spent $50 million or actually $15 million plus NetWhuffie stock. Now we sell the deal as a talent buy, which of course it is because the talent built the damn thing. We put Brett Taylor in charge of the platform (API) and Paul Buchheit in charge of something else he hasn’t said yet. So Taylor can still deliver stream splicing, just not in the FriendFeed context. Buchheit, the Gmail guy, now what do we do with him….

Remember, where were the FriendFeed guys when the clock was stopped. They were streamlining not just the API but the architecture of what used to be rooms and what was now groups. Subsumed into that construct was the wonderful Imaginary Friend notion, another way of saying how do we capture individual streams and normalize personal and group communications. Meanwhile, RSS is dying and with it readers of same, and we begin to see seedlings like Threadsy and Brizzly popping up to address the vacuum. Can we assume the Gmail guy might be in a good position to noodle down on this, particularly given the Google Wave fork under way across town?

Obligatory Beatle analogy: I’ve spent the last few weeks partying on the Beatles Mono Mixes, in particular the period that began with Rubber Soul, followed by Revolver, and then jumped ahead to the White Album and its companion Part II, also known as Abbey Road. In just two years, the group transformed themselves from lovable moptops into four individuals who wrote music to avoid board meetings with accountants (Something) and seers who mixed Tibetan chants with backwards tape loops (Tomorrow Never Knows.) At the time it seemed like a lifetime, but in mono as they blended this stuff together it takes on a sense of purpose and inevitability that belies the official storyline.

In effect, with Abbey Road they tied up the loose ends of the White Album’s API and architecture, injected the realtime nature of Revolver and the betrayal of Norwegian Wood, and sold the company to EMI on the promise of new records from a group that had already broken up. Listening to the mixes and reveling in the deconstruction of Beatles RockBand, you can see how intricate their alchemy became during those brief two years, and how valuable it would become for the ages. As David Crosby said recently of a CSN project to do cover versions of favorites, when it came to the Beatles, they’d work up one or another from memory and then give up after listening to the original.

This then is what Facebook bought, or rather invested in: the best work and that yet to come from this group of engineers, strategists, and explorers of realtime. It’s easy to forget how completely wrongheaded it was to attack the realtime experience, how to this day Wave is reviled as an unbridled solution to a problem that doesn’t exist. What are people afraid of to fuel this alleged stampede for the exits, particularly given that there is no credible replacement for many of its most addictive features? The very hysteria of the charge, a cry for no one, a love that should have lasted years.

More likely is that we’ll see a healthy battle for the legacy of FriendFeed between Facebook, Twitter, and Google. First will be the conversational flow, still a pathetic hack in Twitter, possible in Facebook only with a fast (three month) Everyone newstream rollout with 2 way Twitter/Everyone stream sync. Never mind the Web site; this has to come from the API, complete with granular tools for filtering the flow by data type and guaranteed RSS-hubbed delivery. Facebook must invest quickly in stream splicing to make filtering useful, and Twitter Lists give them the room to build it while no such tools exist on the Twitter side.

Second, if there is a second after that first takedown, is splicing email and personal FB data into the stream, through a combination of intelligent push into email for filtered stream data for mobile clients and harvesting of the Twitter follow and List social graph as an open directory outside Twitter’s control. Google has a play here with Gmail, but they need to drive here with Wave taking a backup role rather than the other way around. Buchheit is the key here, regardless of who does it, and Twitter’s lock on direct messages needs to be factored in in the Facebook sync planning, something that didn’t survive the cut in the FriendFeed lockdown.

Yoko didn’t break up the Beatles. The Beatles did. FriendFeed isn’t dead. It’s just getting started. And the walrus is Paul.

Now that Google Wave is trickling out into the water supply, I’ve been sucked into “playing” with it alongside FriendFeed, Yammer, Skype, and email. Erick Schonfeld insists on discussing a project we’re readying, and unfortunately I’m able to sign in from my iPhone. The FriendFeed direct message interface is not exposed on the iPhone version, and Skype makes me feel stupid for entering a ping and then watching my battery indicator drop while waiting for some signs of life. In this environment Wave suddenly is acceptably lousy.

I’m down at Oracle’s OpenWorld event, which lands like Normandy on Howard Street with a massive tent that this year has proved helpful in the rainstorm hitting full force. The occasion was Salesforce boss Marc Benioff’s quasi-keynote in a small theater just off the main Moscone hub. I caught up with Marc after the presentation at the Salesforce booth, a massive presence punctuated by a giveaway of two SmartCars or equivalent. As Marc suggested, they were one of three exciting things at this year’s conference.

The first was the strange Scott McNealy/Larry Ellsion co-keynote on Sunday night, a delayed-on-account-of the-EU swan song by McNealy that picked up midsentence from his first take at JavaOne in June. Jonathan Schwartz was nowhere to be seen, though his number 2 John Fowler was given a visible role in the proceedings. Scott seemed determined to thread the needle of continuity with the old Sun vibe, but even Ellison’s rosy words of support for MySQL and IBM badmouthing in favor of Sun hardware did little to alter the truth that Sun survives only in its products but not the ideas that might have been coming.

Ellison made it clear in recent days that he’s still stuck on his who-cares-about-cloud-computing schtick, but Benioff for one thinks he’s missing the obvious. While Oracle allowed Salesforce to team up with Michael Dell in this little alternate conference within a conference, Benioff seemed happy to promote being right about the cloud in the midst of the lion’s den. From my vantage point at the back of the theater, there was a small hemorrhage of attendees when Facebook/Twitter integration with its Call Center and CRM services were demoed. But if 10% left, that meant 90% got a message 180 degrees from the show-wide Oracle Fusion meme, Marc’s third exciting theme.

So it will go with Google Wave, as it begins to penetrate the legacy environements of Gmail, Gchat, and Apps. Right now it’s a red-haired step child of both the cloud and the stream, pretty much useless in both courts. The more that user feedback intrudes, the more Wave will devolve to an email clone. But if Google resists that impulse, something big is likely to happen. This is not about user experience; it’s a war for control of the Google realtime platform, and the central mechansim to slow down Facebook and Twitter. As such, they need to fail, and fail fast — and then start integrating with or absorbing Gmail and Apps.

They can look no further than Yammer for tips; the Twitter enterprise clone has rapidly expanded its rich clients on the desktop and the iPhone to provide push notification at a precisely the time we need it. In particular the iPhone app is suddenly more useful than email, with new messages indicated on the icon as they are received (42 at the moment) and direct messages popping up in a Push notification. Email has too much flow and not enough filtering to avoid constant interruptions by needy PR and the like, and forget about Twitter or even FriendFeed in their current firehose status.

Wave could grab this away from Yammer with little trouble, but the big problem is one of focus. The prize is stream control, the metering of stream dynamics and some sort of pass-off to a stream feed that can be consumed after the high order Yammer-type business flow is absorbed. For now, Wave can co-exist with FriendFeed/Facebook, and ignore Twitter and email. That leaves Skype and Google Docs traffic, which suggests that Gmail and Gchat integrate Waves as a new datatype to be launched into once pressing messages are triaged.

A strategy begins to emerge: high priority messages, Wave document discussion, stream maintenance, and email/presence monitoring. Using the Oracle/Cloud example, start pilot cloud projects, task IT with building hybrid on-premise “private/public” clouds, then negotiate with hosting and virtualization layers as M&A activity drives the move to cloud consolidation.

Wave sucks in its implementation, in its stonewalling of our need for priority stream filtering, in its confusion about its entry point into Google, or vice versa. But that will get sorted out by brute force, more likely by Facebook and FriendFeed integration, and whatever Twitter can buy in house to avoid cannibalization by Track invaders from their increasingly quarantined thrid party vendors. And Microsoft and Apple loom to mop up if Google slips too much or dithers too long, which probably means they won’t Wave may suck, but it won’t matter for long.

For months now we’ve been wondering when Microsoft was going to start making moves in the social media space. Rumors of talks with Twitter have been swirling at all levels of the company, but now a subtle re-org may shed light on what Microsoft might do internally to shore up its presence in the RealTime Wave. Chief Software Architect Ray Ozzie has announced the formation of Future Social Experiences (FUSE) Labs, a new group led by general manager Lili Cheng. FUSE Labs is being seeded with a range of related talent and software by combining much of Lili’s Creative Systems Group with Rich Media Labs and Startup Labs in Cambridge, MA. In the past, she’s been able to move social technologies from the labs into product.

In an internal memo, Ozzie talks about the growing vortex of social media and realtime:

For many years, technology-based ‘social’ innovations have been most commonly viewed through the lenses of communications and collaboration: messaging, chat, calls, meetings, conferences, co-editing, document sharing, collaboration, multiplayer gaming and the like.

More recently, many factors have begun to transform all that which is ‘social’: the ever-present, high-bandwidth internet both wired and wireless; the ease of connecting people; the dramatic rise in digital cameras, camera phones and ‘app-capable’ phones; net-connected game consoles & TVs; and so on.

Myriad scenarios involving the notion of ‘social’ have now gone far beyond communications and collaboration and are transforming experiences that are key to our customers and key to our business, in leisure & entertainment; productivity & teamwork; experiences extending how we use the OS itself.

The three groups being combined have concrete skills and code in areas where ‘social’ meets sharing; where ‘social’ meets real-time; where ‘social’ meets media; where ‘social’ meets search; where ‘social’ meets the cloud plus three screens and a world of devices.

FUSE Labs will bring more coherence and capability to those advanced development projects where they’re already actively collaborating with product groups to help them succeed with ‘leapfrog’ efforts. Working closely with MSR and across our divisions, the lab will prioritize efforts where its capabilities can be applied to areas where the company’s extant missions, structures, tempo or risk might otherwise cause us to miss a material threat or opportunity.

Cheng, who will report directly to Ozzie, moves from Microsoft Research (MSR) and her Creative Systems team, which most recently produced Kodu, which teaches kids how to create games and stories on the Xbox. Previously, Cheng was in the Windows division where she managed the User Experience teams for Windows Vista. Before that, she ran the Social Computing Group within MSR, which developed projects such as Wallop and VChat. Cheng first joined Microsoft in 1995 as part of the Virtual Worlds Group within MSR.

Reading between the lines, Cheng’s ability to surface technology from the labs has now been focused on more immediate concerns. This mirrors Microsoft’s success with Bing, which has emerged with many MSR features as part of its well-received search engine. Most recently, some of the Visual Search features debuted at TechCrunch 50 take advantage of Silverlight deep zoom technologies. It’s not a stretch to assume that these features will be laced throughout whatever social media constructs might as Office Web Apps hit the beta streets later this year.
Cheng will retain most of her original team from Redmond while traveling to Cambridge to consolidate the other teams. Kostas Mallios, general manager of the Rich Media Lab, will continue to report to Ozzie and take on business development responsibilities assisting the incubations within Ozzie’s org. Reed Sturtevant, managing director of the Startup Labs in Cambridge, MA, has decided to pursue interests outside Microsoft.

Photo credit: Flickr/Joi Ito

AT&T’s decision to allow VoIP onto the iPhone changes the balance of power in the communications industry. The move underlines Apple’s dominant position as the prime mover in converting the phone into the core identity container on the network. As Mike Arrington intuited, number portability via Google Voice was worth more than the Apple device, at least for the political purpose of calling the issue to the floor.

It will be interesting to see how this plays out in the relationship between Google and Apple, but far more important than that cleanup operation will be the effect on other identity firefights. In particular, the SideWiki conflagration, otherwise known as Hey, Get Off My Lawn, threatens to alter our perceptions of what constitutes the basic unit of Internet communications.

If the most recent era of Net value has been dominated by the architecture of Page Rank, SideWiki and other realtime intermingled services including Google Wave, FriendFeed, Disqus, and Echo are just the first cuts at an atomized page that bases its content according to the identity of the viewer. Today, the look and feel of the “page” is determined by the viewer’s profile, which identifies the groups, communications channels, aggregated services, and “friends” they belong to. At its simplest, a browser detect delivers an iPhone interface to those users. At its more complex, FriendFeed filters content based on social data from many of these profile sources.

But SideWiki, PubSubHubbub, and Google Wave seem much more intimidating because of their actual or virtual support by a dominant player. What might seem acceptable writ small (realtime discussions attached to streaming video services on FriendFeed) suddenly appears threatening when viewed through the lens of Google patronage. Never mind that these so-called BigCo strategies are laced with open technologies, standards, and evangelists hired away from the open source community. It’s still a modern day Medici who rules with a benevolent velvet fist.

That of course provides an opening for the so-called Decentralization crowd, who argue that size does matter and that Google can’t be trusted any more than Microsoft or Apple was or is. Only a small band of new patriots can employ the open techniques. If a company starts small and then achieves mass (Twitter) then the analysis needs adjustment. At some point, these startups turn from being the good guys to the bad guys. Of course, the idea is that you should trust the new guys now that the old new guys have something to lose.

This is one of the problems with the success of RSS and, to some degree, podcasting. They appeared to be revolutionary in their disruption, but in fact tended to reinforce the medias they supposedly replaced as they gained traction. While we were busy enjoying the excitement of penetrating the closed walls of the media club, we conveniently overlooked how the winnowing process of the hunt would look a whole lot like the show business we were trying to break into.

The business model (or lack of it) that we accepted as the opportunity cost of getting in the game proved less acceptable once we were inside the circle. Debates about with or because did nothing to change the reality that in a world of unfettered access the index became the barrier. And when the index shifted to a more immediate and simpler technology (Twitter) the money flowed with it and away from the technologies that opened the door. The same thing happened with the carriers.

The iPhone brought down the wall of voice by transferring the index from the phone number to the URL and the new identity. As Web usage grew, the need for more powerful filters drove the increased value of our social addresses, making voice less significant as the gatekeeper to our attention. Twitter quickly gathered momentum by aggregating IM, email, blogging, and rich media under a single apparent transport, and its follow and track tools proved more fine grained and precise than the legacy alternatives. From there, it was only a short hop to money flowing into those priority lists as they became the new gatekeepers.

SIdeWiki appears to be a hijacking of the so-called original content of the blog post, but perhaps that’s looking at what’s happening through the other end of the telescope. In fact, SideWiki and Wave and Twitter are where the index is, and the posts are looking more and more like comments on the stream. Of course, that raises the specter of lock out by market force, and signals the attacks on the SUL, Google’s scale, and Apple’s Walled Garden.

But success at disrupting this grand conspiracy of size — VoIP coming to the iPhone — does not mean failure even for the so-called vanquished. In fact, the big will get a lot bigger, as Google and Apple have now been blessed by government pressure to provide the fruits of an effective alliance without the threat of antitrust action. That’ll teach ‘em. And SIdeWiki will be pushed to provide some form of control for page “owners” that will likely involve some available identity system to white- or blacklist offensive comments. Oh wait, we can use the Gmail contacts and mandate Google social constructs as part of the solution. Cool, no more clumsy backwards engineering of the Gmail social graph. The government insists.

It leaves us to ponder the rationale of rssCLoud and other attempts to foil the BigCos. Is there going to be a decentralized core of adoption that will blunt Twitter/PubSubHubbub/FriendFeed or SideWiki/Wave/Chrome’s control of our data? History doesn’t suggest that outcome. I think people are making a fundamental judgment about the framing of open v. closed. Steve Jobs bet that our desire for control of our identity would trump the hiccups along the way to that reality, and AT&T is validating it for what it is, good business.

The iPhone continues to show how a hybrid of open and closed has staying power with the voters. As long as it delivers increased opportunity, nobody really cares how it works. Faced with a shortened URL that requires a leap of faith, we opt for a social cloud that over time produces a good result based on the power of our peer relationships. The tools that give us the choice of modeling trust relationships will do better. Over time, we and our friends will gain more from faith in our harnessing of identity than fear of its dangers.

Google Wave is roiling the collaboration space as it moves out of the sandbox and into a wider beta. The ripples are being felt by vendors ranging from IBM to Cisco and even Google itself. IBM is challenging Google Apps with an iNotes offering undercutting on price (as well as features, as Google quickly points out.) Cisco is buying small business videoconferencing assets to bolster its Telepresence technology at the high end, and jettisoning IBM Sametime in favor of its WebEx tools.

While Google and many analysts see the iNotes move as a direct challenge to Gmail’s recent instability, the likelier motivation for IBM is Microsoft’s inroads with its Exchange Online product. Redmond may be months away from rolling out Azure, but the on-demand versions of Exchange, Shaepoint, and then Office Web Apps are going to hurt IBM where it counts. Why not attack the weaker target in Google’s consumer/corporate hybrid product to change the subject, Big Blue figures.

Meanwhile Wave continues to make people nervous about where email is going anyway. Wave remains a relatively siloed project inside Google, with its realtime constructs more on the bleeding edge than practical solutions for the problem of managing the growing information stream. Facebook’s acquisition of Friendfeed signaled the power of the micromessaging trend, and attacks on Wave’s experimental UI and metaphor belie the extent to which email is threatened.

Since Twitter went mainstream in the last year, micromessaging threads have become the main carrier of realtime news. Trending topics may seem the fundamental index, but information at the actionable level is carried in a smaller stream managed largely by retweets and overlapping follow clouds. A cascading series of Likes in the Friendfeed/Facebook nomenclature is far more efficient than other mechanisms, including email newsletters and RSS syndication.

This new brand of news is more CNBC than MSNBC. It’s based on a “what’s in it for me” dynamic, which prioritizes the flow based on business implications first, followed by political and social impact. The Letterman story spread on Twitter and Friendfeed once the show aired on the East Coast, giving viewers the heads up to tune in to the show on the Pacific run. The morning shows ran with it as the lead. The underlying reason for the position at the top of the news cycle: the potential impact on Letterman’s contract at a time when Leno’s move to primetime has changed viewing numbers in CBS’s favor for the first time.

Leno’s show is doing quite well with its reworked talk show formula, which can be recorded and consumed in cherry-picked segments. It won’t take long for these microstream swarms to be packaged via realtime alerts, mostly because the microcommunities formerly known as the broadcast audience are under increased pressure to consume more efficiently. Comcast’s move to buy NBC comes after their success in shifting recording to on-demand; it’s a small stretch to adopt that model of server side recording for a netwrok they control. The studios will need to bend to the new model or risk the same squeeze Steve Jobs performed on the carriers with the iPhone.

Email, like RSS, is being supplanted by socially-filtered microbytes reassembled under user control at the client. The SideWiki argument is the stalking horse for this wave of transformation, as users increasingly paint what used to be known as a page with dynamic inline updates of information they’ve told information providers they are willing to incorporate. The end game for email is the understanding that public and private streams are only delineated by the social contours of each granular element of the flow. It suggests the modern update of the phrase For Your Eyes Only.

The Beatles Rock Band game is now in its third day here at Abbey Road West, and so far it’s getting better all the time. As social media, it’s the off the charts monetization winner Wall Street is beginning to think Twitter and Facebook are becoming. As my wife keeps saying, it’s got real Beatles songs, not some cover band. How cool is that?

BRB is an extension of the Beatles Love mashup, where producers George and son Giles Martin went back to the basic tracks and transferred them to digital for remix. This is as distinguished from the new remastered mono and stereo catalog, where only the final mixdowns were brought up to date with modern analog-to-digital techniques. The Beatles recorded on two and then four tracks up until the White Album, bouncing down preliminary mixes and overdubbing additional parts as they went.

The Love mixes built on a technique explored during the production of the Anthology series and fleshed out with the remixing of the Yellow Submarine record. Laying all the original pieces onto a digital checkerboard, the Abbey Road engineers could recreate the original mixes with individual control over many more elements of the recordings. Love expanded on that by literally deconstructing the various elements and intermingling them with other tracks, as in the layering of the Tomorrow Never Knows drum track under Within You Without You’s Indian percussion.

With this digital map already assembled, Giles Martin could turn his attention to the Rock Band game, compositing guitars and harmonies while adding live effects and studio chatter to create yet another mashup, this one a fascinating illusion of being in the studio or in concert with some degree of input in the mix. Of course, the game opts for scoring the closeness to the actual reality you come with vocals, guitar, and most powerfully, drums. The actual music remains Beatles, but you have the feeling of getting inside the music.

Ringo comes through in the way his bandmates saw him, the actual spark of the thing that became Beatles only when he joined the group. The recordings have always reflected the alchemy of the four members and George Martin, who handled many of the keyboard parts as the group’s palette expanded in the studio. But Ringo’s parts, as reflected by the dumbed down notes you sync with, underline how much the drummer shaped the variety of feels of the group’s enormously productive output.

What emerges is a kind of dynamic blueprint that threads through the band’s history, regardless of the period (touring, studio, early, splintering, the end) and almost in spite of the differences in song writing and production. The Beatles throughout their career benefited from this kind of stylized fundamentals, fitting their “real” personalities into their film and studio images and creating a new hybrid that expanded both identities into a newer one.

So too do Twitter and Facebook and other social media experiences, fusing the twin streams of public and private persona. Much is made of the falseness of the online you, but the reality is that the combination of digital gestures and individual desires, thoughts, ideas, and emotions is something we’ve only perceived indirectly through artists in novels, paintings, and music. Social media may be an imperfect label, but the synthesis is as real as the elements pre-aggregation.

The mono remasters are bundled in a separate collection, running from the beginning through the White Album. After that, only a Yellow Submarine soundtrack with several “throwaways”, the Let It Be “live” recordings massacred by Phil Spector as the band fell apart, and the final Abbey Road were left to be mixed in stereo. But for most of its life, the group and producer recorded and mixed in mono, an earlier predictor of the power of 140 characters.

The Beatles Rock Band remixes suggest the rest of the catalog will be rendered in new versions that take advantage of technology to get inside the music, not subverting the original mono intent but allowing us to explore the brilliance of these collaborations. Musical anthropologists will be able to take apart the harmonies and build new ones, discover our great treasures in new ways, and share them with our children here and across the net. Calling this a game is misleading only because it is so much fun.

Of course, there will always be those who posture about the authenticity of these experiences. McCartney has the most authority here, saying the game is fun but actually making the records was more so. But playing along with Ringo in even this most elemental and simulated way reminded me of similar moments in my past playing drums with the “real” Richard Manuel on a snowy night in the Catskills. Beatles Rock Band is not the same as being there, but that doesn’t mean it’s not real. Beep beep yeah!

With Windows 7 shipping in less than a month, we’re sure to smell a whiff of the Microsoft of old from the Pacific Northwest. After years of dropped balls and transitions from the Gates era to whatever we’re now in, Steve Ballmer should have plenty to feel good about. Steve Sinofsky has completed his personal reworking from Office chief to Windows czar, and the new OS arrives just in time to crest with the netbook wave.

On the Office front, the O2010 tech preview is in a classic Microsoft holding pattern, waiting to touch down next year in the wake of the new Windows release. Microsoft marketing managers still won’t answer the simple question (What are the features NOT available in the Web Apps?) They’re glad to rationalize the answer as: we’re providing the features Web users want. The real answer continues to lie in the politics of the transition from disk to Web.

What is new is that Powerpoint and Excel the Web versions can now function with somewhat greater fidelity than Google App’s competitor. Compare them to the desktop versions, you’re in the old political weeds. Compare it to Google, not bad. SkyDrive starts to look like an interesting service, if only (and importantly) as a reason to get a Windows Live ID and 25 gigs of free storage.

I find the desktop/Web Apps comparison a cul de sac Microsoft will do well to get away from as fast as possible. The central message for nextgen realtime apps is that Office Web Apps are soon to be highly competitive with the alternatives. The collaboration features alone are so basic that my fundamental question is whether the desktop apps support them in realtime, not the other way around. Word is still not baked and seems to continue to suffer from oldthink strangleholds, but if the first wave gets some traction with realtimers it may tip things our way before ship date.

The other mixed messaging is of course Silverlight. It’s easy to wonder whether Microsoft really gets what they have going here, but the underlying answer is yes, with the political caveat. Reports of a recent internal meeting were devoid of mentions of Silverlight, with the usual Bill-era science projects around Walls and other esoteric research fantasies taking up most of the troop rallying.

Nonetheless, Silverlight is the most strategic part of the new Microsoft message, that Redmond services will catch, match, and exceed existing competitor platforms. If you count the Windows 7 launch as the left side of the timeline and the PDC launch of Windows Azure a month later as the right, Silverlight is the connective tissue that drives the new hybrid platform.

With Windows battened down and Office Web Apps in the evangelism stage, Microsoft is competing primarily with Google’s notion of radical disruption financed by advertising. Coming as it does from the enterprise IT stronghold of corporate hegemony, Windows and Office adoption collides with the new monetization challenge because of the lack of a legitimate user contract for behavioral data. We’ve known all along that Microsoft could see what we do on the desktop, but they haven’t got our permission to do anything with it.

By contrast, Google and other Web services require that attention farming to do anything, and offerings like Gmail remind us of how our text is being tracked even as they reassure us that that data is only being used anonymously as part of the larger algorithms behind the Google engine. Microsoft’s investment in the Web Apps is in fact the establishment of a user contractual rationale that will pay off with its forthcoming competitive offerings.

Those offerings will not be in the desktop space, however. They will be in the Silverlight engine, mixing the richness Google struggles to reach with HTML 5 with the monetization of attention and, increasingly, gestures mined from socially-aware microcommunities. Don’t let that jargon scare you; it just means Facebook/Twitter 2-or-3 degrees of separation harvesting of the recommendations of people whose recommendations in aggregate are much more efficient than the current search haystack models.

SIlverlight neatly manages the intersection of the desktop circa October with the cloud circa November, and is missing only the orchestration of realtime layer that, pardon the expression, meshes incoming alerts with the behaviorally-filtered authoritative stream. Today, I watch alerts bubble up in an Adobe Air FriendFeed app, sitting atop Gmail, Yammer, and stream aggregation tools. Tomorrow? Silverlight is the only answer this year, and Steve Ballmer must know it.

The good news for those who think this may work strategically but not politically is that Office Web Apps is close enough to provide cover for an aggressive framework built on Silverlight to go after the social crowd. Facebook is within 90 days of an API that can subsume the FriendFeed core, and there are very few things you can’t do today with a combination of Javascript and Silverlight to provide a useful console for managing social flow.

There are at least several projects under way but unannounced that could signal Microsoft cluefullness, but even if an internal project hooked up SIlverlight with, say, Facebook, it would still be messaged as “we’re a platform company providing opportunities for developers”. But Office Web Apps and SkyDrive will need to drive home a Now proposition to have some impact on Google’s aura of inevitability.

Here’s where Bing becomes significant. These guys are kicking ass. They are innovating and pushing out babies faster than the Scobles, and show no signs of slowing down. Does Silverlight have a play in future Bing expansion. Yes. Does the iterative success of a reasonably independent unit inside Redmond provide political cover for similar ground gained by a social unit. Yes.

The key is to compete not with the desktop guys but with Google directly. Office Web Apps stand a good chance of smoking Google Apps in the short term, by competing directly on price and adding features (Silverlight) Google can’t touch. Like Bing, invest in the intelligence of the back end and the editorial layer of services that pry mindshare loose or at least into consideration. My 8 year old daughter talked to me last night about Binging something; I made her repeat it to be sure.

Bing. Social media. Silverlight. SkyDrive. Benioff’s Salesforce is pushing the enterprise envelope with the Google stack, but what does that do for Microsoft but open the eyes of its dominant channel. While we all look at the deltas between Office Desktop and Web, the real disruption may be the extent to which the enterprise is being swallowed by social media.

If virtualization effectively abstracts out the back end hardware and software boundaries, the end result is a new value chain where access to institutional and user data is parsed across business lines, where software “suites” are organically grown and spliced together across social boundaries. Social media IS the enterprise. Bing. Bing. Ballmer’s Silver Hammer.

When Cnet blows the whistle on social media, via Sean Parker of Napster fame, it’s time to break out the wallet and double down on that very same “immature” social media. We’re standing at the gaping maw of the biggest gold rush in many cycles, and the refs have just called a TV time out to allow us to regroup.

Certainly we’ve seen altogether way too much social community constructs at just the moment when we can’t stomach another additional stream entering the firehose. We’re fed up with authority algorithms, freemium models, top ten lost Twitter features, and bogus realtime protocol wars. It’s fine by most of us if it turns out Dave Winer invented everything that could or will be thought of back in 2001.

But calling the moment when the startup market matures is like giving your 16-year old the keys to the new car. No matter how sure we are that we’ve seen the last good idea and that’s it’s now all about sensible implementations around direct monetization, we’re absolutely sure of nothing of the kind. That’s because the very notion of maturity is a strong buy signal for the next stupid idea that’s about to spring up and conquer the world. Calling the bottom is easy; calling the top is hard.

Are we at the top just because our heads are exploding at the sound of the rain of stupid startup names that crescendo at TC50 and other industry events. This TC50 was the first one where I immediately deleted the name of the company from my awareness and never looked back. The only one I still remember (other than Bing which is now Old Stupid and therefore exempt) is Threadsy, and that’s because I love the thing, even though I doubt I’ll ever use it.

Now that’s an interesting data point for me, coming as it does after signing up for Twitter and Facebook at the insistence of Ross Mayfield and Gabe Rivera respectively and then promptly forgot about it for years. So I actually think the Threadsly experience will probably be highly successful precisely because I have no idea how or why I would use it. My idea of solving the Inbox crisis is to try and make the stream so informative I can skim email in seconds because I already know about everything there.

Similarly, looking at the cloud surrounding a person’s identity is a really cool trick, and one big reason why Threadsy is a two-trick pony. If I were trying to figure out authority or reputation, this would be a great pushbutton way of doing it, and the fact that the app is pulling all this social stuff together with some degree of coherence is a plus for the demo. But the real reason I like it is because of what it will do rather that what is currently does.

Just as Twitter emerged because of the intersection of a user-demanded function (Track) and its Follow social graph modeling, so might Threadsy emerge from the intersection of the death of email and RSS and its social whuffie modeling. It’s always instructive to remember that most big startups only take shape in the second wave of their existence, after the early rounds separate them from the pack and the talent congeals around the disruptions that emerge on the margins. Mint’s Intuit exit served more to validate the TC50 model than it did to signal the maturity of a social market that is barely at the breakout point.

It’s easy to call a top when Facebook goes revenue positive, but what it really says is that Facebook is now at the point where it can put pressure on Twitter to put up or shut up. By that I mean that they have had no pressure whatsoever to actually compete up til now, what with a virtually indestructible momentum that was only enhanced by the Fail Wail. But now, with FriendFeed driving the API strategy of the Everyone status of the open Facebook opportunity, Twitter has to put their A-Team on the floor. They’ll do just fine, by the way, as Dick Costolo demonstrated from the TC50 Peanut Gallery.

Notice how he didn’t see that much in Threadsly, questioning how easy it isn’t to monetize clients or some such. I could feel invisible fingers reaching into my wallet (not altogether an unpleasant experience strangely). Wait, that must mean there’s money in my wallet, or some on the way. Hmmm, that might mean that social media has some end game we haven’t yet seen. Uh huh.

All told, TC50 was a power shot at the notion of a frozen market, a clear signal that we’re finding out what we’re going to do when we grow up. Of course, we look a lot like our parents in the process, but that’s just something to deal with along with the mortgage and car payments.

Maturity can mean many things — responsibility, stability, success — but none of those things rule out the wild side of the realtime moment. Drama can come in many shapes and sizes, and this drama is gathering steam. When they ship AnyClip, you can look it up. As Margo Channing said in All About Eve: Fasten your seatbelts… it’s going to be a bumpy night.”

It’s been quite a while (May) since I wrote a post about the then-less-obvious move away from RSS toward micromessaging. The observation that Twitter’s rapid growth was supplanting the use of RSS as a reading mechanism — in my case abandoning Google Reader in favor of FriendFeed — was met with emotional attacks from RSS’s prime mover, Dave Winer. Knowing from first hand experience in supporting Winer throughout the rise of RSS, I certainly was not caught unawares by the vehemence of the reaction, even as it became more and more personal.

What did surprise me was Winer’s rapid reworking of RSS into a “cloud” enabled solution, even as he continued to argue with such intensity that a centralized service (Twitter) could not sustain itself. In other words, RSS is not being replaced because it can’t be, but in any case it can already do everything the new message bus can. It’s certainly the best case to be made, but it has a weakness best illustrated by observation:

Twitter is winning. The stream is faster. Faster will increasingly win the hearts and minds of those who value speed when all other metrics are commoditized. In the history of technology, doing more faster creates economies of value, which in turn accelerate adoption. The most obvious example: RSS. A technology that has always been difficult to explain nonetheless emerged and spread rapidly to the point today where it dominates the ways information is spread over the network.

Nothing is more pervasive in the network than RSS. It’s the technology that turned the Web into a DVR. We interact with it every day more than any other tool at our disposal. Whether we perceive it that way or not, we do. RSS changed our relationship with information, from hunter to partner. It also changed the way we process that information, the way we create it, and the way we deal with overload.

In a way, it’s the polar ice caps melting that finally hipped us to the obvious. Those glorious days at the wheel of our youth, the sky expanding in front of us to the horizon, the feeling of unlimited power as we stepped on the accelerator. We knew in the exhilaration of the moment the special cost of that freedom, that now was our time and that someday we would need to accept the pull of gravity.

Part of the lure of the open road is outrunning fear. Time traveling. Beating the odds. The odds are Twitter won’t go away, won’t collapse, won’t drag its thrid party cloud into a black hole and vanish. It doesn’t even matter whether it’s called Twitter, or Facebook, or Google, or Microsoft. Just like the Internet, the reality of this thing built on top of RSS is that it is bigger than its parts, its companies, its parts. Each node may be centralized but in aggregate it’s the opposite.

If that’s true, attacking a popular service for its failings in business, ethics, or any lack of “openness” does little to slow it down and perhaps the opposite. It’s not a matter of politics, it’s just sheer numbers. People voting with their feet. The arguments about long form versus noisy tweets, or built-in versus external URL shorteners, or RSS versus Twitter. They beg the observation: people like Twitter, they get their information first from the stream, and they need to figure out how to manage the flow.

Is there any magic to bring to bear on managing the flow? Yes, that’s why Twitter is so central. [Editor's note: When I say Twitter read FriendFeed.] Twitter’s Follow cloud, combined with Track, enables an authoritative filter into the flow that extracts a high percentage of the aggregate consensus of that group into a manageable stream. How that manifests itself in its delivery to the screen is what the great software battle of the moment is all about. Which device or devices we use can be interesting to argue about, but they’re all really iPhones in one shape or another. Oh sorry, couldn’t help it.

The raging against the RSS, or Office, or Links is Dead meme — I get that it hurts feelings, annoys VCs, leads to RTBS (RealTime Block Syndrome). But no amount of shooting the messenger (another metaphor) will change the fundamental point I’m making: that RSS has triggered a new wave of innovation that will inevitably build out from where RSS has stalled and eventually create a similar disruption of what it produces. Everybody look up at the sky and smile. Click.

Today’s Gmail outage illustrates just how tolerant the new realtime architecture is to individual service failure. The initial surprise at the comprehensive nature of the flatlining may have caused some significant degree of marketing damage, my bet is that the end result will be a boost to the service’s popularity, and with it, realtime services including Twitter and FriendFeed.

For my part, moving over to FriendFeed direct messaging and private groups immediately took the outage offline for the work I was doing. The fact that traffic slowed dramatically on Gmail (IMAP and POP were still up) meant there was little to miss or catch up on. My iPhone let me know that I was missing very little, and FriendFeed aggregated both the Twitter flow that documented the extent of the outage as well as the message and blog traffic that explained the outage’s contours.

Within minutes it became clear how little I now depend on email in general. TechCrunch’s internal traffic is over Yammer, and FriendFeed’s realtime chat function has already scooped up most of the collaborative chatter. In recent weeks we’ve been readying a service to be released alongside the return of live video streaming of the Gillmor Gang, and the development team switched over to private groups (what used to be called rooms) a few days ago. Email continues to be a valuable source of one-to-one messaging in near realtime, but the collaborative filtering of social graph-based alerts is slowly but surely taking over from RSS readers.

Taken from a 10-thousand foot view provided by the outage, Gmail becomes a critical service that can be patched around with just a few key services, nailing up direct messaging and private groups configured to send email alerts of the resultant rerouted messages back to email when the outage is over. After 5 minutes or so, the problem went away from an operational perspective. Twitter outages have been more disruptive for their impact on information flow, and Google has learned valuable lessons which will reduce the likelihood of the current event significantly. We didn’t dodge a bullet as much as catch it in our teeth.

Watching Teddy Kennedy’s wake and funeral over the past few days has been a sad but wonderful lesson in the value of public service. By public service I mean the motive variously referred to as empathy, generosity, fairness, and other such terms demonized as liberalism by the new and to some extent old right wing. The Senator’s life consumed not only those labels but the full arc of a career among the people. And in doing so, he now offers a template for success in a polarized era, at a time when hard decisions are finally coming down to a vote.

Chief among his insights was the relentless pursuit of compromise as an art of war. His Republican friends spoke emotionally and graciously, but with strong doses of resistance to the famed Kennedy charm. They recognized him as a formidable opponent who could only be tamed, or at least harnessed, by finding common ground on which they could stand with him. If it was a win for them, they reasoned, I’ll gladly go along for the Kennedy ride. Not to say they didn’t enjoy the camaraderie, the chance to escape the dark hole of the sanctimonious extremists, the true spirit of enterprise that lurks in any politician’s heart. Kennedy gave them the running room to merge into the consensus of the times.

We all underestimated Teddy. From the Left, we were endlessly surprised when he championed the causes his brother Jack delineated but never had the political strength to produce. When Bobby was killed, nothing tore at our weary defeated souls like Teddy’s eulogy, which used the same tones and fierce despair Bobby used in Dr. King’s death. When he anointed Obama as the next brother, he casually let it be known first as Caroline’s idea. The master’s touch, from someone we always felt was a weaker version of the sons, fourth in line to the throne.

Now he’s gone, and we tremble at the thought there are no adults left to see us through. No matter how beaten up Obama gets, he retains double the popularity of the opposition. The cover of the latest Rolling Stone, with its year-long investigation of why the Beatles broke up (Yoko, for god’s sake), asks the musical question along the top: Is Healthcare Reform Doomed? What would Teddy say to that, we wonder. A quick journey to FriendFeed for the answer.

FriendFeed is the last remnant of the ’09 campaign. The realtime debate between left and right has calcified most other places into Death Panels versus whatever the Left is selling, oh I know, the Public option. For me, Death Panels is actually a legitimate argument, not for its factual basis (none) but for the same kind of appreciation Republicans saw Kennedy — a good frame of a fundamental fear. Just because it comes down on the wrong side of the argument detracts nothing from either its power or its usefulness in the debate.

What Kennedy understood was the power of the model, regardless of the details or even the wins and losses of the struggle. To get a foe to argue not about the issue but about the implementation was an instant and comprehensive victory, and a quick look at Kennedy’s issues shows he won them going away. FriendFeed won the realtime argument the second it launched real-time chat. And when ugly commentary surfaced about Teddy, it bubbled to the top and prompted some direct and unambiguous responses. One retort, and then silence when the tone remained strident and harsh.

What people miss with realtime is the deep change it produces in the arc of the conversation. Some decry realtime as too fast, as prone to snap decisions and 140-character cartoon oversimplifications of complex thoughts. These distinctions — RSS is long and detailed, Twitter is bursty and noisy — are political planks, not facts. They serve the implementations and the economics of the discussion, not the underlying art of war. Realtime already won the war; now we wait to have it explained to us, and carved out of compromise and perseverance.

When people attack FriendFeed for going away, I’ll see Teddy’s smile. When they attack Facebook for being closed, it’ll be tougher but still, Teddy’s smile. When Rolling Stone questions heathcare’s health, I’ll see it as evidence of Obama’s caucus troops going back to the well, energizing their troops with something akin but not yet fully engaged with Death Panels and town halls. The best thing we have going for us is no matter how much we think heath care reform sucks, it sucks less than the alternative.

Teddy’s death gave us something we’ve waited a long time for, a coda to the deaths of brothers, Beatles, and God herself. As the dusk shrouded Arlington and wrapped us all in our reveries, we could each conjure our own version of the future. At one point Chris Matthews complained about how President Obama wasn’t at the burial, and Olbermann countered by saying the eulogy was enough. Surprisingly, Matthews withdrew his comment, something he never does. I smiled Teddy’s smile.

Watching our fearless leader’s journey into AndroidWorld has been fantastically entertaining, as Mike’s mission to bring Google Voice to the iPhone is presented as a move to the free open Google voice device hoedown. Entertaining mostly because if the real truth were revealed, we would understand how Machiavellian this “revolution” really is, and how Arrington, Google, and Apple are locked in a conspiracy to oust the real villain of the era, the Dumb Pipe.

Strip away the religious fervor of the Arrington plan — the tyranny of the AppStore, the inability to crush your iPhone battery into rubble by running endlessly mesmerizing apps, the frustration with the speed of the massive disruption of social space triggered by the iPhone’s invention — strip the arguments away and you might glimpse the true reality of what’s going on. Namely, that Apple is conspiring with Google to force the FCC to “force” Apple to, regrettably, open the door to VoIP and the Universal Inbox.

Why did Apple let Google in in the first place, with YouTube, Gmail, and Maps? Because the alliance served Apple incredibly well in bringing an intuitive composite interface to the exciting new world of a real Web-aware phone. But the deal served a much larger purpose, to prod AT&T and by extension the rest of the carriers to move to flat data pricing and support for alternatives to the blockades erected against Flash, tethering, SMS, and other revenue safe zones “respected” by Apple’s political design structures.

In each case, Apple could sit back and wait for the market to get noisy about the restrictions, let Google carry the ball forward against the carriers, and then eventually cave in to the “realities” of the marketplace. If anyone noticed that it might appear to be in both Google’s and Apple’s best interest to squeeze the carriers into compliance, then all that would need to be done to avoid the appearance of a counter-cartel would be to resign a board seat or two, make a lot of noise for the benefit of an FCC who is desperate to have any role before VoIP demolishes their seat at the table, and tease the digerati avant-garde with cool services to the point where they can’t live without their 2010 fix.

Let’s get real about this Google Voice thing. It’s totally the wave of the future, a future invented by Apple via the iPhone and then handed off to Google and eventually Microsoft for shrinkwrapping. It’s easy to forget how pathetic things were before the iPhone, how we endured crappy cell service and endlessly stupid Web 1.0 stillborn services while we struggled to move from modems to DSL to shared cable to Edge to “3G” to realtime social media. Each level of service required the jujitsu and compromises of the previous era to pry open the running room to make the next corner turn. Jobs is no more the bad guy today than whoever will take Google’s place as the “revolutionary” next week.

The old saw is that you need to invent your enemy if they don’t exist, and nowhere is that more true in the realtime era. Watching Twitter struggle to stay ahead of its audience as it competes with Facebook for realtime services reminds us that doing this stuff is hard the first time but gets easier as the market rises to commoditize the disruption. It’s no accident that FriendFeed was co-designed by the Gmail designer, who leverages the emergence of a user base as a tool to drive wider adoption when the platform is more ready for them.

Arrington’s struggle to live in the future is our struggle of the near future, but it will be a lot easier for us as a result of his squeaky wheel. But let’s not kid ourselves: if Mike gets his way and Apple “caves” on Google Voice, he’ll dump the G2 only long enough to find a new leap ahead to accelerate. The good news for him and us is that we need just a very few such acquiescences to get to the point where the Dumb Pipes will blend seamlessly into a single platform. Of course, then we’ll all get iPhones anyway because they’ll run Google’s superior apps and all the AppStore offerings without conditions. Because the Dumb Pipes would have to be invented if they didn’t already exist.

The news that some loser URL shortener is closing down does not prove anything other than that those who promoted it with their links better have gotten something out of the deal already. It’s all part of the usual pony ride that starts with excitement about the possibilities and ends in recriminations about the winners controlling the market. Now Dave Winer is calling for an end to what he calls “at best a temporary workaround for a limit Twitter shouldn’t have.” So here are the facts:

Twitter came out of nowhere and dominated the information space by constraining the canvas to 140 characters. The URL became the application trigger, the wormhole through which we could leap back into the existing macrocosm of blogs, software, and services. URL shorteners then, and now, are the mechanism by which we send data to and from the message bus. Whether these URLs are controlled by a private or public group is entirely inconsequential, because whoever controls the flow of that data has an advantage over everyone else who doesn’t.

Twitter and other apparent control points have both power and limits. Twitter’s power is the constraint of the wormhole; its weakness is the assent of its users and third party developers. If users and apps can move from one to several wormholes, their leverage accelerates dramatically. How can this be accomplished with Twitter’s dominance? By piggybacking URL transforms on top of the gatekeeper’s single point of control. By establishing another (or several) URL transform sites that large or influential numbers of links flows through.

In order to reach that critical mass, the purveyors of these URL transformers must establish credibility in the marketplace of return on investment. In making assessments about technology, it’s vital to assess not only the value of the innovation but its chances of survival in the arc of its moment. In that crucible, arguments against market scale, open versus commercial, and so on are largely moot. Whether some person or some community has “control” is only relevant to whether we think it will be successful in its goals and therefore be a factor in how the network develops.

Let’s say we have two transformers in operation. They both operate in similar fashion, taking Bit.ly links and repackaging them in unique and proprietary ways. The end result: you click on the hybrid links and they uncompress in one, two, or potentially endless permutations. As Adam Bosworth once said about XML, its alleged verbosity is trumped by Moore’s Law. So too are transforms. In effect, the speed of the uber transform server is the gating factor, or put another way, the latency you don’t notice you is not a problem.

So now we have competing uber transformer chains of shortened or otherwise enhanced metadata firehoses. As they develop audiences with interesting characteristics — authority, realtime velocity, reach, etc. — they begin to establish a micro-community power base that can encourage mergers, acquisition by platforms, and the emergence of market standards. It does no good to rail against this or that bigco conspiracy; in a transformer ecology, Twitter or Bit.ly or any one service isolates themselves by not talking to the uber layer. The water will flow around those disruptions, so it’s unprofitable to pursue them.

It’s more likely that Twitter and Borthwick will play along rather than constrict the straw through which the firehose currently flows. Hybrid transforms can quickly cherry pick the valuable streams, as many are using FriendFeed as the uber Twitter API. If URL aggregators emerge now to wrap uber layers across the weaker URL shortener populations, it will be difficult for Bit.ly or Twitter or both to mop up and retain an effective monopoly. Even if it’s only a matter of time before URL shortening is absorbed at the OS level, buying up real estate is certainly a smart strategy if only as a hedge against the eventual consolidation.

But just as the iPhone will continue to overturn the market fundamentals by constraining its application space, so too will those who control the URL transform space continue to drive the realtime expansion. It’s not a religious argument but a pragmatic one, and no amount of pigeonholing or posturing about unfair good old boy arrangements will obscure the opportunity for the hybrid transformers to seed the emerging market. The first one in will be the second, and all we need to break the stranglehold that really doesn’t exist.

With Twitter down this morning and reports of failure all over the social Web, I figured FriendFeed would be up, if denuded by the Twitter outage. Well, sorta. In fact, FriendFeed searches are down. How the hell does a denial of service attack plague reach into the coolest service no-one will ever use, as former user Mike Arrington once put it. Is the realtime Web screeching to a halt on purpose, did rssCloud have something to do with making things too really simple, or what? For now, I’ll Blame FriendFeed.

As long as FriendFeed keeps rolling out updates to realtime, FriendFeediots like myself are smug in our uber view that no matter what happens to the rest of the Web, regardless of whether Google buys a patent-safe video codec for HTML 5, not caring whether Apple and AT&T collude to keep Google Voice out of the carrier business, not worried about whether the Sun/Oracle deal is held up for months, etc. — no matter what, FriendFeed will solve all of my social media problems before anybody else does.

Now search is back up, so I really should stop worrying. After all, one of the advantages of being small is that your problems are relatively smaller. Even as the normal realtime chatter resumes — Scoble at some Boulder startup event, someone asking how to import Twitter replies, Drudge headlines Twitter outage, Facebook “transport errors”, Leo Laporte says no Windows Weekly because Paul is in Paris and can’t get a decent connection — I still can’t get past the reality that when Twitter farts, FriendFeed becomes a complete and total gasbag. After months of innovation, revamped APIs, advanced realtime features, and Scoble pruning his Twitter follows because FriendFeed’s cloud is more manageable, FriendFeed is still a parasite on a pimple on Twitter’s ass.

Of course, I don’t feel better even now. Blaming FriendFeed doesn’t reduce my own guilt. Endlessly promoting the advantages of realtime, the ephemeral return of Track, playing Laconica and Facebook and FriendFeed off in a pathetic attempt at coaxing a response out of Twitter, goading Dave Winer into overhauling RSS. None of it in any way meaningful, when it all still comes down to whether Twitter is up or down. For all of FriendFeed’s power, it still is best at being a Twitter client. Albeit a Twitter client that is modeling the transition of micromessaging to an enterprise class backbone.

Alright, now I’m beginning to shake this off. Stream splicing is still to be released in the next 30 days, I’m told. The PubSubHubbubGlubglub integration with Google Reader is lightning fast. The RSS is better than Twitter argument thread is humming. Scoble is being attacked for starting a Twitter unfollow trend. Breathing deeper now, it’s all good. Get back to work, Arrington says less Yammering, more posting. I Blame FriendFeed.

OK. Did Microsoft just absorb Yahoo for zero additional cost beyond its oft-stated plans to spend some 10-15% of its budget on search per year? Did Apple really stifle innovation with its carrier-friendly App Store rules? And what’s common to both companies’ developer strategies? Yes, no, a lot.

Working backwards, Apple and Microsoft offer their developer communities their favorite incentive — money. Seven million .Net developers come out of the Yahoo deal stronger than ever, with very few public comments from Yahooers hoping to move over to Microsoft like the ones that drowned out the proposed buyout phase of this deal. Back then, it was all about the complexity of integration of the Valley stack with .Net, and how ridiculous it was to think Yahoo engineers would work for or even with the Redmond crowd.

But the Yahoo deal was never about Microsoft. It was about Google cleaning Yahoo’s clock, leaving it vulnerable to the takeover that’s been happening in slow motion ever since. The original deal was tied to a timetable designed to clear the Bush administration before it left office, and no one seemed to recognize how serious Ballmer was when that deadline expired. Given the trouble Google had with its subsequent deal with Yahoo, you might have a little more respect for Ballmer’s counting of the cards in the first phase. Now, with Google out of the antitrust argument and in effect validating the rationale for the new deal, Ballmer is sitting pretty.

With Yahoo out of the search business, a two-horse race provides incentive for an attack on Google’s realtime challenges. It’s not so much that they don’t understand the stakes, but as with Google’s undercutting of Microsoft revenue streams, Microsoft doesn’t have to defeat Google in order to profit from a healthier competition for the efficiency of click-throughs. Just as television advertising is distributed across a four-network market regardless of NBC’s or CBS’s current status, so too will efficient results work across a 70-30 split. What does make a difference are the Web properties that house the ads, and that’s where social media and its follow clouds can make a big difference.

Here’s where the comparison with Apple and its developer cloud begins to resonate. Every time we hear the outrage of an unfairly damaged App Store victim, our first inclination is to think there’s some fundamental flaw in the relationship between Apple and its third party developers. The argument: Apple’s arbitrary limits are designed to protect AT&T from applications that subvert their revenue model, and/or Apple from losing its leverage obtained through the exclusive deal. It’s a complex series of checks and balances — subsidized device prices to spur dominant market share, all-you-can-eat access, and the use of onboard WiFi to keep the pressure on for further concessions.

The best comparison is one with Twitter and its third parties — rate limits and shifting favoritism with developers based on allegiance for a few select winners. The models cross over with Tweetie, which not only enriched its developer and investors but established a poster child for how to play ball and win in both the iPhone and social cloud spaces. Contrast that with the unhappy video developers who tread too close to the firewall between excessive 3G saturation and the more constrained 3GS upload model, and most developers will overlook the losers and dream of hitting the App Store lottery. It’s not really a matter of rules but rather timing: what won’t work today inevitably will work at some point when bandwidth can survive the new usage pattern.

So the answer to App Store rate limiting stifling developers is no, and it points a way to the Microsoft opportunity that grows out of the slow-rolling Yahoo dismantling. With Bartz focusing on high value media plays, .Net and Yahoo transplant developers will start looking for an iPhone-like platform to get comfortable with in the horse race with Google. That platform is Silverlight, which offers advanced realtime rich media features coupled with access to the Microsoft advertising platform. Interestingly, Silverlight and the iPhone share some powerful characteristics — H.264 support, on and offline application models, and streaming deployment of both media and application update code tied to an iterative transactive micropayments fabric. And the keyword: money.

Google, on the other hand, will be watched carefully as it attempts to hurdle the application barrier Apple is defining with the Google Voice slapdown, as well as deal with the difficulty of integrating Google Wave with its apps or vice versa. At some point, Microsoft will have to decide how long they can protect their enterprise control of Office at the cost of losing the global consumer market of the next generation. They may realize a Silverlight socially aware Office for the consumer space (netbooks, XBox, and down into Windows Mobile) can jumpstart a Silverlight App Store before Google has a chance to react. Selling that for a bargain price (free) to the Yahoo crowd will look like the next logical step in the Yahoo takeover, and make the increasingly sweeter pill a lot easier to swallow.

Sandwiched neatly between the RealTime is God and RealTime Who Needs It crowds is a new group that embraces both positions while moving forward rapidly. These folks include Brett Slatkin of the PubSubHubub effort and Dave Winer of rssCloud.org. Slatkin and fellow conspirator Brad Fitzpatrick demoed the PSHBB architecture at the RealTime Stream Crunchup, and Winer quickly jumped in with his own implementation.

While Winer attacked PSHBB as a Google (and TechCrunch) conspiracy, the rhetoric seemed mostly in service of his own plans for a decentralized open platform not controlled by bigcos. The charges don’t make sense — PSHBB is not a Google effort but rather one started by Fitzpatrick and Slatkin on their own — and like Winer’s, proposed and delivered as an open architecture that can be spread across multiple players. In other words, they’re saying the same thing.

Digging deeper, it could be that Winer’s concern is that adoption by Google or other platform players will create a bandwagon effect that could lock out smaller independent developers. There’s certainly some truth to that dynamic, as PSHBB is already running in a number of cooperating platform implementations including FeedBurner, WordPress, and even FriendFeed, where Slatkin is active in tracking user issues. But ironically, by offering a competing framework, Winer takes away the argument of unfairness. If developers and publishers want to adopt one platform over another because it’s got more mainstream adoption, they’re free to do so. In the past, Winer has been the beneficiary of just such choices, with RSS 2.0 proliferating in no small part due to support from Microsoft on the platform side and the New York Times on the publishing side.

Both PSHBB and rssCloud sidestep the issue of whether RSS is on the decline, preferring to co-opt whatever momentum the technology has into the swarming viral nature of the realtime micromessaging disruption. Whether there’s consensus about the underlying value of the new platform, there’s no denying the uptake and attendant opportunities for gaining mind and perhaps market share of the new wave of approaches to managing the stream. If that were not the case, why would PSHBB be gaining adoption and Winer be spending time on his own cloud? Our own experience in managing the CrunchUp suggests an explosive market for startups; we spent most of the last few weeks trying to fit people into panels and demos that had already expanded to the breaking point.

If realtime is simply the latest buzzword, a placeholder for getting attention and perhaps venture money until the next buzzword comes along, then we’d expect to see some slowdown in those realtime projects that are out at the bleeding edge. Facebook certainly seems to have stopped short of a realtime updating stream, though I’m seeing more Facebook messages appearing in aggregate tools such as Seesmic and FriendFeed. But mostly I’ve seen complaints about intermittent slow updating from Twitter and an apparent move to a one-hour pinging of new WordPress items in the latest update. In other words, speed wins even if no one has yet figured out how to manage that speed efficiently.

Then there’s WiFi on planes, where I bet we’ll soon see numbers reflecting the impact on what airlines we choose. If realtime is now the new normal in publishing, then losing a working day to a cross-country flight is no longer viable. With price discounts for mobile access, the *Phone becomes the easy way to keep in touch with short, bursty messages the simplest way of managing the infostream. And the fundamental result: the URL payload becomes the lingua franca of the message bus.

It’s here where the damage is being done to RSS as a transport. As micromessage citation grows in importance, the sheer number of Web pages cited begins to overwhelm the RSS payloads. In recent days, Google Reader Likes have been added in an attempt to map some sort of social graph to the consumption of RSS items, but it’s still unreachable except from within the Reader UI. The trend moves away from RSS items and toward the original pages, which for publishers means a much greater opportunity for cross-linking, advertising, engagement, and analytics.

Podcasting is similarly disemboweled, given the ease with which YouTube videos are streamed to all always-on devices. It’s easy to decry Google market force here, but it’s no accident that Microsoft’s IIS Media Server now supports H.264 as does Silverlight cross-platform. I believe Silverlight becomes the default NetBook media server, based not on enclosures but rather streaming. We’re already seeing popular podcasts such as Leo Laporte’s TWiT gaining large realtime audiences as the number of streaming services proliferates. It’s only partially random that Twitter emerged out of the ashes of a podcasting service.

One argument against this future is that it tends to rely on centralized storage and a gated cloud of services. Winer personalizes this to the notion of TwitterCorp, that no single company should control this uber stream, that they won’t be able to, that if they could it would be a bad thing. He has a point, but whether rssCloud will solve this problem is not clear. He’s using Amazon EC2 and the open source Identica project for parts of his platform, both services provided by commercial companies.

In his rssCloud Roadmap, Winer asks us to think of rssCloud as Loosely coupled 140-character networks, and his goal “to instantaneously flow 140-character messages around, with metadata, including links, tags, enclosures, and whatever else (it’s as open architecture as RSS 2.0 is).” I’ll quote with interleaved commentary:

A network that works alongside Twitter, but outside the control of a company. You can be confident that no company will control it because it is being started by a person, not a company. Even if I wanted to crush you, I couldn’t. Even if I wanted to control all the users’ data, I can’t — since most of it won’t flow through my servers.

I’m not convinced there is much of a distinction between services offered by an individual or a company; they both can be “controlled” to the extent that users offer their data up. Perhaps the caveat of “most” lessens that control but to that extent the remainder of the data flowing through Winer’s servers would be the sum of his control.

Re Google’s PubSubHubBub, their goal appears to flow updates of blog posts in association with Feedburner. This is a good goal. I hope to help them get RSS compatibility into it. If your goal is to optimize the polling of existing RSS or Atom feeds, you can use the rssCloud network, but that’s not something I want to get involved in personally. My interested is in the micro-blogging application, at least at the outset.

Again, characterizing PubSubHubBub as “Google’s” is inaccurate. He offers to help provide RSS compatibility, and suggests that while rssCLoud can be used to optimize polling of RSS feeds, he’s personally only interested in the micro-blogging application for now. He’s surely threading the needle here, but how that will help spread adoption of his solution depends on the degree to which his solution appears more open and counter to the TwitterCorp threat he sees.

As with RSS, it is always a mistake to underestimate the power Winer’s bearhugging of technology can muster. But the context in which RSS emerged has as many differences as similarities with today’s circumstances. In particular, RSS was supported by two kinds of media — the emerging blogosphere and, eventually, the mainstream media. Today, these two medias have merged, primarily as a result of RSS and its democratic impact. It’s as though the New York State Thruway suddenly smashed all of its toll booths and allowed free travel. The impact was seismic, like the Guttenberg press. Shelf space became infinite.

That revolution took time and luck and plenty of political pressure to gain enough momentum. By contrast, Twitter bootstrapped RSS, broadband, social dynamics, and the combined medias. What took several years with RSS has taken a fraction of that time precisely because RSS proved you could tip markets without the dominance of bigcos. In effect, RSS proved we don’t need to fear the locked trunk, at least not in the classical way it’s been defined. How do we defend ourselves against bigcos like Google and Microsoft from using RSS tactics themselves?

At the Fortune conference in Pasadena today, Michael Arrington reported Twitter’s Biz Stone responding to a question about Discovery as saying “they need to assign weight to users or specific Tweets to do a better job of surfacing Tweets.” Inherent in the comment are the tools of control: “assign weight” and “surfacing Tweets.” These words imply ownership of the stream, just as Page Rank became the control point of authority in the search era.

Here’s Ray Ozzie on the same point:

I think it’s going to impact search, because the nature of search ranking is based on — at least classically is based on looking at what’s relevant by what it’s linked to, and if a lot of the links are moving from more static media into more dynamic media —

Me: Oh, so you agree with my thesis.

Ozzie: Yeah.

Me: Excellent.

Ozzie: Yeah. I don’t know to what degree the ranking will be impacted in that realm, but I think that’s a fairly important thing. And the Tweet itself is more or less the anchor text of the link and so on.

But wait, here’s Microsoft talking about how disruptive the stream will be, and it’s a littleco talking about controlling the high ground. Can Twitter establish control of the message stream to the point where no other stream can produce efficient filtered results to users? Again, Ozzie:

[W]hat I learned in the Groove experience is that if you have lots of things that notify things, the user wants to suddenly aggregate them and tune which ones they want high bandwidth instant notification and which ones they only want to occur every once in a while. So, having messages go through something where they can say, okay, I’m going to have all of these different classes of messages, move these classes into this bucket, these into this bucket, notify these on my phone, these somewhere else, it’s useful for the user….

[A]s humans we have these issues. And certain of the events, certain classes of the events we want to treat, as Dave [Winer] says, like a river where you don’t really care if you miss something. It’s where you’re not trying to keep up every little thing. Maybe it’s an amusement, maybe it’s just a background activity.

Some types of events you just want to see them. You just don’t want to miss even a single one in this big flood of notifications. And so we just need better tools.

It stands to reason that the more endpoints, the better the chance of no single company controlling the stream. Today Twitter talks like a bigco, but in a world of streams, they must compete on the quality of their discovery model, compete on how they “assign weight,” compete on how they “surface Tweets” to their cloud of third party clients and analytic engines.

Viewed in this context, PSHBB promises to bring Feedburner feeds, Google Reader shares and Likes, and various smaller RSS clouds such as WordPress up to speed and into the weighting and surfacing game. The coalition also benefits from bridges to services such as FriendFeed that Winer has eschewed because they have offered solutions such as SUP. rssCloud certainly serves as a reminder of the underlying values of simplicity, transparency, developer independence, and user control, but then again that’s what PubSubHubBub is selling too.