Techcrunch confirms that Microsoft has announced that they have acquired natural language search engine Powerset for a sum rumored to be around $100M. The acquisition is another key step in Microsofts efforts to gain market share in the online search market – currently dominated by Google and Yahoo.

The search team at Microsoft said today in a blog post that they were interested in Powerset because of the strong team of engineers at the company, a team lead by CTO and co-founder Barney Pell that will remain intact in San Francisco post-acquisition. Microsoft also say that Powerset has solved two key search problems:

• The difference in phrasing and context between search queries and the context on websites. For example – related words such as ‘tree’ and ’shrub’ or in context such as ‘what is’, ‘who is’ etc.
• Powerset, they claim, also offers more relevant and clearer search results. Users might find what they believe is a relevant result in a results listing on standard search, only to click through and find that the page is not relevant to their query.

Current search engines work based on keyword search – they find pages based on the density of keywords that the user has searched for and then rank the results based on a number of factors ranging from the credibility of the page (based on inbound links) through to simple keyword density. Powerset uses its technology to actually analyze natural language elements on each webpage. A query such as “who wrote catcher in the rye” using normal keyword search will attempt to find those very words, while with Powerset and natural language search it will analyze what the user is actually looking for, even though there may not be a website that directly spells out “who wrote catcher in the rye”. The technology is all based on analyzing the users intention when searching based on the words they use.

These advantages come at a cost – as natural language search requires more computational power to index web pages than normal keyword search. For this reason Powerset has been restricted to indexing only Wikipedia. The resources that Microsoft has in terms of server infrastructure provide Powerset a shortcut in achieving their goal of indexing the entire web – a task that would likely have cost tens or hundrends of millions of dollars in infrastructure investment had they attempted it alone.

The rumors of an acquisition were first floated last week, and both companies declined to comment further. The acquisition announcement comes on a day that Yahoo announced in their regulatory filing that they are certain that Microsoft was never serious about a full acquisition of the search company. The fallback at Microsoft now seems to be to make smaller but smarter bets, rather than big leaps, in an effort to gain search engine market share.

Microsoft has previously stepped outside of traditional norms to compete in the search space. Recently they announced a scheme called Live Search Cashback where Microsoft will be paying search users back a portion of advertising revenue. The strategy sees Microsoft forgoing short-term search revenues in favor of building market share, and in our previous Techcrunch coverage Michael Arrington referred to the move as both “desperate and brilliant”. The Powerset acquisition can be seen as the next step in their strategy, where they are attempting to gain market share through innovative technology and marketing.

Microsoft’s search engine market share is currently at 9%, far behind Google who has 61% of the market. The web division at Microsoft has been performing below expectations since its inception – and with the dominance of Google the challenges facing Microsoft are as big as any the company has ever faced in its history. With a dominant desktop position and a majority of the browser market with Internet Explorer, it would seem almost trivial that Microsoft would be able to leverage that position to take a larger slice of the annual $40B+ search advertising market. The failure of Microsoft in the search market is about the strength of Google and their advertising network and the inability of Microsoft to anticipate the rising dominance of the search company.

Microsoft has a challenge to not only unseat Google, but at the same time protect its flank where Google are striking at the core of the Microsoft business with products such as an online office productivity suite, email and shortly even a competing mobile platform in Android. With the Powerset acquisition and schemes such as Live Search Cashback, Microsoft has recognized that they can’t take Google on head-on, but that they must plan and implement today technologies that are a step ahead of their competitors.