A new generation of database products and companies is beginning to emerge, and one of the more interesting examples is Swedish-based Neo Technology, the developer and vendor of the neo4j graph based database (graph in the data structure sense). The neo4j product has been in development for over 8 years, and Neo Technology are today announcing a new $2.5M round of funding. The company has been developing the neo4j project as a commercial product, and is now taking it to market with a dual-license model.

A graph database is a more natural method for expressing, storing and retrieving data that does not fit well in a standard relational database schema. The best example is to consider social networking models, or other models with relationship elements that are either not easily expressed in a traditional table structure or where a table and relationship based structure does not scale.

In a demo of the product we saw, a mock social network structure was created where 1,000 users were defined, each with 50 friends. The traditional table based database took 2,000ms to query every friend from every user, while the graph-based neo4j database took 2ms. To demonstrate the efficiency of the database further, with 1,000 times more users at a million (and an order of magnitude magnitude more connections), the total query time was still 2ms. The graph model and the neo4j database are able to easily scale with complex relationships between entities and with a more flexible schema.

Neo Technology are providing a commercial version of neo4j, Neo, along with services, training and support for the product. The product is licensed under the AGPLv3. The company raised $2.5M from Sunstone Capital and Condor Venture Partners. They previously raising a smaller seed round of $300k from the Swedish government. Neo Technology was founded by a small team lead by CEO Emil Eifrém. The team originally developed the neo4j product as an internal database at a previous company, and have applied the technology in commercial environments for almost 10 years.

Neo Technology are closely following in the footsteps of another Swedish database company, MySQL. They not only share the same home country, but both companies started with a solid open source product, both are database companies and both share a similar business model around open source software.

Neo provides the next generation of database, more suited to most common data problems faced in the real world today. Graph databases are part of a group of technologies of non-relational databases commonly grouped under the ‘NoSQL’ name and movement. The NoSQL movement began with a recent conference in San Francisco, and a conference this week in Atlanta where the Neo Tech was represented.

Most application developers today are not farmiliar with non-relational data storage models, since RDMS dominate the market and most frameworks and language environments have little to no support for alternates. This leads to developers squeezing data models that are not well suited for a table based structure into a database such as MySQL, often leading to poor performance, scalability and reliance on code to do the heavy lifting.

The technology around graph databases was previously usually developed in house by companies who identified specific needs, with some commercial options available. With companies such as Neo Technology supporting a very stable and scalable open source product, the technology is sure to now start to find its way into enterprises and become popular with application developers.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

It was only three days ago that I wrote about the almost hopeless challenge of web security, specifically around new vectors with cross-site scripting attacks. Today came news that an XSS vulnerability had been found in the RubyOnRails development framework – and that applications built on the framework, such as Twitter and Basecamp, were vulnerable to XSS attacks.

The vulnerability was discovered by Brian Masterbrook. He probed Twitter with some Unicode characters and found it vulnerable, tried the same thing on Basecamp and found it vulnerable, and then deduced that it must be a problem with RubyOnRails. He has an excellent and detailed write-up on his site about the process he went through. If you are running RubyOnRails anywhere, stop now and read his post as well as the security notice from the Rails developers and get your servers updated (the patch is in the notice, it will be in the release branch ‘today or tomorrow’).

There are a few parts to this story. The first part is that it is great that Brian just happen to be one of the ‘good guys’. We find out about this vulnerability because he put the effort in to inform the parties involved and to get a fix out. We could have easily found out about this the hard way.

The second part is the poor response he got from the guys at Basecamp. I use Basecamp, I have two accounts there and we use it somewhat internally at Techcrunch. Not anymore. Their response to a major security issue does not inspire any confidence, at least not enough confidence that I would feel comfortable having my personal data residing on their servers (for more details on their response, see Brian’s post). Twitter I am not as concerned about – they seem to have their act together in terms of responding to issues promptly (they have had their fair share of security issues, no link required there I think). Brian originally discovered this issue almost a month ago – he seems to have spent a lot of time just communicating with these vendors.

Third part – learning from the mistakes of others. Point one would be: if you are a vendor of an application, get your shit together so that when one of the good guys wants to be able to tell you about a vulnerability he or she discovered, you don’t lose a day arguing about if an email had been sent or not (I am looking at you too Apple – that post will come later). ‘Getting your shit together’ involves having a clear contact page for security issues, a PGP key (afterall, you don’t want everybody reading about how your site is vulnerable), and perhaps a few lines of policy on what you will do if a security issue is discovered (‘its not our fault’ is not a policy).

Point two (of the third part – you following?) is about what developers can learn. If you scan the patch file that the RubyOnRails guys sent out you will see one word consistently popup in the code – ‘verify’. That is what this issue, and what almost all security issues, come down to – not trusting user data and making assumptions about what data your application will receive. The majority of applications take a blacklist approach – ie. check for that, check for this, filter, block, filter .. filter. The more correct approach is to assume that everything coming in from anywhere is going to completely screw you up, and then allow only what the application can handle and only what you expect. If your application is all in English, and you start seeing some funky Unicode characters that were previously only used by a long-forgotten African tribe – there is a safe bet that somebody is messing with you. Blacklist vs Whitelist.

Something else I noticed that has come out of this incident. The RubyOnRails notice attributes the cause of this vulnerability to:

“Due to the way that most databases either don’t accept or actively cleanse malformed unicode strings”

Oh, im sorry? Its the database! Remind me next time I pick a database server to use that I choose one that will filter and anticipate all my user data for me (hint: I was being a sarcastic, if you are a CIO who just picked up the phone to Oracle to ask for the XSS-proofing module, put the phone down). It is easy to shift blame – but this would be like the Microsoft IIS 4.0 team blaming Dennis Ritchie for all their buffer overflows.

We are going backwards with security. It is 2009 and vendors aren’t responding properly and blame is being shifted. In the meantime, RubyOnRails is now seeing a surge in downloads from Russia and various other places – not because those new users want to build apps, but because they want to own yours.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Looking back on the history of the web, classic security protection involved patching servers to assure latest versions were running, monitoring advisories from vendors, and maintaining some level of filtering and firewall to keep basic attacks out. Simple moves on the part of an admin or developer could protect sites from 99% of automated scripts. But a few years ago, a new security can-of-worms was opened, as new exploits that took advantage of simple oversights within web applications were being used to steal large amounts of user data. This new class of vulnerabilities took advantage of attack vectors within custom built web applications, using techniques like passing Javascript calls into web forms which would then be published back to an unsuspecting user. This new breed of attack was referred to as Cross-Site Scripting (XSS) — in short, the ability to manipulate a trusted website to run untrusted scripting code on a victim’s browser.

Cross-Site Scripting, and its related cousin, Cross-Site Request Forgery (XSRF), have led to attacks and exploits such as MySpace being taken down (via a worm, Sammy), data being stolen from 18 Million users of a Korean auction site, a Gmail weakness used to blackmail a domain owner and even an exploit targeted at changing the settings on a user’s local broadband router. All of these exploits were accomplished by convincing the user to click a link, an email (where an embedded image containing an exploit payload was displayed) or by simply visiting a site they trusted and had previously visited.

Various statistics claim that up to 80% of security vulnerabilities (pdf link) in the past 2 years have been the result of XSS and XSRF. There are claims that at various points, over 70% of websites were vulnerable to either one or the other. Anybody who understand how these attacks work, and who understands how to conduct a simple test (i.e. feed something like '<script>alert('y0');' into a web app and see if it pops back out somewhere unfiltered), would tend to agree that a large number of sites were, and still are, vulnerable.

Complicating the XSRF and XSS problem is the fact that not only does it take time to inform and educate developers, but that new ways of conducting such attacks against the most modern web apps and browsers are still being discovered. While application developers are busy cleaning up their code to protect against simple vectors discovered years ago (eg. escaping simple input text with addslashes()), security researchers are discovering new ways of exploiting the trust relationship between a user, a website and the web browser. These ‘new ways’ are being discovered all the time, and often fall outside of the box of previous thinking on what it takes to secure a web app.

For instance, today I read about (via dalmaer of Ajaxian) a newly discovered potential means for XSS and XSRF exploits by forcing a browser to talk HTTP to a non-HTTP service and have the code response interpreted, bounced-back and executed by the browser (that is my single-sentence attempt at condensing this brilliant description, which should be required reading for every app developer). It seems that every few weeks I stumble on yet another description of how to manipulate the trust relationship to exploit a user.

What is worrying is that these attacks exploit the foundation of the web — a network that was built with an implicit level of trust assumed between users and servers. To keep up with security requires a key re-think of how data is transported on the web and destroying the assumption that most data is safe data. Also worrying is that in all likelihood, most successful attacks exploiting these methods are likely to go unreported, as they can be used to silently attack a targeted individual who would usually have no way of knowing what is occurring underneath the hood of their browser. The black-hats have no incentive to share new methods they discover, forever locking developers and corporate security researchers (or those working on the ‘good’ side) in a race to stay in front.

Having performed bare-bones testing of new web applications I see, as well as monitoring the security announcement lists of web applications I use myself, I can safely say that most web application developers today are at least a year or more behind on the latest security vulnerability methods being discovered. Complicating this is that browser manufacturers themselves do not completely understand the issues involved, and in some cases are moving backwards (ie. the new IE8 is now allowing XmlHttpRequest across-ports). Scary? Yes. What to do about it? I have no idea, other than to get educated and attempt to stay on top of it.

Update: A somewhat ironic twist to this story. When I included the code example above (ie. how to test for XSS) it actually passed through the CMS running this blog and kept triggering when I would attempt to preview or publish this post.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Amazon today launched a new web service – EBS, the Elastic Block Store (yes I also first read it as ‘Elastic Book Store’) for EC2. EBS provides persistent storage for EC2 computing instances, and the service is public today and available to all customers after a period of alpha testing with some users.

Previously EC2 instances were able to access temporary storage as part of the compute instance, or persistent storage only on S3 – the Amazon online storage service. The difference between EBS and S3 is that EBS allows block-level access, so that it can be mounted just like any other local storage device from within EC2 and can be accessed across servers and between instances. S3 is accessed as a web service, so performance for latency sensitive applications was never optimal (such as running a database store). EBS provides a much higher level of performance comparable to high-grade local storage in terms of both access times and availability.

Persistant block-level storage for EC2 is perhaps long overdue, as one of the criticisms of EC2 when it first launched was the inability to run a fast data store across snapshots, which made running databases or other data-intensive applications slightly more complicated. Services such as RightScale have built products around helping developers scale and manage MySQL instances on EC2. Other cloud-based computing services such as Mosso or virtual servers from providers such as MediaTemple have had persistent storage options, although what Amazon have developed with the combination of EC2, S3 and now EBS is a tiered approach which provides more flexibility to developers.

Users of AWS can from today create up to 20 EBS storage points utilizing up to 20 terabytes of storage. Snapshots can be stored to and retrieved from S3. Pricing is based on both storage used (10 cents per GB) as well as raw IO requests (10 cents per million). Storing back to S3 is charged at the standard S3 rates.

EBS is certain to open up new territory in terms of the ease at which certain types of applications can now run on AWS. The most obvious is the ability to now run high-performance and high-availability database instances, which is an essential part of the standard LAMP or RubyOnRails stack that many web applications use.

Perhaps somebody will build a simple setup and installer for running Laconica instances on AWS, with auto-federation built in. EBS will certainly make such an application, and a lot more, much easier now.

Atlassian has announced the availability of their JIRA Studio suite as a hosted Saas service. JIRA Studio is a hosted integration of popular Atlassian products starting with the JIRA issue tracker and Confluence, their wiki and knowledge management product. Atlassian has partnered with Contegix to provide the hosting for the new suite of applications.

The JIRA Studio solution includes issue tracking, wiki software, SVN repository management, a single-sign on server and continuous integration. The solution as a whole is a development community and/or development suit in a box – providing almost all the tools that are required for software management. JIRA is a well known and established issue tracker, with custom workflow and a number of other features that sets it apart from competing products. Atlassian have built their suit out around the issue tracker, and overall it provides a unique and customizable environment.

Mike Cannon-Brookes, the CEO and co-founder of Atlassian commented that it was the partnership with Contegix that allowed them to enter the hosted application and Saas space. Atlassian has to date built a base of over 12,000 customers in 104 countries by licensing their collaboration and development tools. The partnership with Contegix and the new hosted solution allows the company to acquire new customers who are seeking an instant install without hosting and administrative overheads. Pricing is set at $25 per user per month, available in blocks of five licenses (so $2500 per annum). Bulk pricing rates are available beyond 100 users.

Contegix are an enterprise hosting platform that offer a 100% uptime guarantee. Their hosted service is fully managed and supported – allowing companies to focus on their product or solution rather than the hassles of hosting. At the OSCON conference, Atlassian and Contegix are offering a 5 user license of JIRA studio for free to anybody who attends a demo at their booth in the hall. For those of you not at OSCON, a live demo instance of the JIRA studio is available here.

Microsoft announced today that they have acquired data warehousing application provider Datallegro for an undisclosed amount. The acquisition was run out of the Data and Storage Platform division at Microsoft, with an intention to integrate the Datallegro application into Microsoft SQL Server and the data application suite.

Microsoft are rapidly building out their data services offerings and have made a number of acquisitions as part of their strategy to compete in the enterprise market. Microsoft recently also acquired Zoomix, an Israeli company that provides automated data intelligence. The data and storage division at Microsoft is growing rapidly, and with the upcoming release of Microsoft SQL Server 2008 and an agressive marketing push they should be able to steadily chip away market share from Oracle in the enterprise market.

Datallegro was founded in 2003 by Stuart Frost and Mark Thacker in California. They just raised 19.6 million dollars in their D round in May. The round was led by an affiliate of the Hillman Company, with Adams Capital management, Focus Ventures, Intel Capital, JAFCO Ventures, Palomar Ventures and Venrock Associates joining.

Drizzle is a newly announced fork of the open source MySQL project. The developers of the project are taking MySQL back to its roots as a light-weight web application database by removing many of the features introduced in MySQL 5. The fifth version of MySQL was in development for years as some users demanded features such as views, stored procedures, transaction handling, clustering and more. The early releases were bulkier and not as stable as the ultra-popular version 4 of MySQL, and now somebody has forked the codebase into a new project to take the database server back to what it was.

For most web application developers, only a basic database system is required. The original popularity of MySQL was because of its simplicity and ease of use. Postgres was always a full-featured open source database server that offered all the enterprise features of competing commercial systems. MySQL was a lighter alternative which was easy to install and learn, but a lot of that simplicity was lost as the development of MySQL progressed towards competing in the enterprise.

Drizzle would seem to have an instant user audience and developer base amongst those longing for the old MySQL. These developers are likely low to medium-end web application developers using a scripting environment and don’t require or don’t need an advanced database system. MySQL was a key part of the default LAMP stack that pioneered simple web application development which went on to open a whole new market. While MySQL 5 can be componentized and customized, developers seeking a smaller and lighter-weight database can revert to Drizzle, at least until the MySQL team see the demand and offer something themselves.

Kontagent is an application that integrates tightly with platforms such as Facebook to offer widget and application developers a high level of analytics data. Current analytics packages for social network applications or widgets offer nothing more than pageview stats or simple user numbers (such as what Facebook publishes itself) while Kontagent is able to bury itself deeply into the platform and extract detailed user and conversion data. Kontagent, which is currently in private beta, has been in development for over 12 months now and is currently being used by a small number of high profile companies in tracking their application usage and adoption on Facebook.

We tried out Kontagent and looked at all the data it was collecting, and were impressived with the results. You can narrow down visitor information based on geographic location, age groups, sex and many other variables. All of that data is being taken out of the Facebook platform and fed into the analytics engine. The larger widget companies such as Slide and RockYou have similar deep analytics at their disposal, but Kontagent is about even up the playing field by allowing the smaller developers to get access to similar technologies. One of the most interesting statistics is virality, which tracks the number of average users each new user invites and over what period of time.

From an integration perspective, there are both client and server components that must be tied together. On the backend, Kontagent provides an API (and a library in PHP) that is used to send data back to its servers. On the frontend a Javascript include is used to send client data back. Integration is a bit more complex than a standard analytics engine, but if offers much richer and more interesting data. For an average developer, integration should take an hour or two, and shouldn’t require long-term maintenance within the application itself.

Kontagent was founded by Albert Lai, who previously was the founder of BubbleShare and Jeff Tseng. They have raised a small round of funding from valley-based sources and their team is split between San Francisco and Toronto, Canada. Their intention with the product is to make it available for free to low-end developers, while charging a fee for commercial applications. Currently Facebook is supported by in the near future support for MySpace will roll out as well as other platforms. Users interested in participating in the private beta can submit their details on the website.

News broke out across the world yesterday that Bosnian Serb war-time leader Radovan Karadzic had been captured after 12 years on the run. Karadzic had adorned the top of the Interpol most wanted list for over a decade, commanding a reward as large as that offered for Al Quaeda terrorists such as Osama Bin Laden. What was most surprising about the arrest was that Karadzic had been living in Belgrade amongst the public, even working as a new energy practitioner.

Under his new name of Dr Dragan Dabic he had a website where he outlined his services, complete with an email contact address. What is even more surprising is that the email contact leads back to a Gmail account and username which has been registered for at least two years. For years authorities searched wooded areas of Bosnia, caves, underground complexes and monasteries and the whole time the answer to the thorn in their side was within a US-hosted email account.

Accounts of the arrest from Serbian authorities and the media claim that the local police were tipped off to an approximate location by a foreign intelligence agency. This lead to a day-long house to house search in the suburbs of New Belgrade, an old communist-era planned suburb full of filing-cabinet like concrete structures that served as homes for the party elite during the Tito era. While it is not entirely clear and details are likely to never be revealed, the pattern of having an approximate location and then relying on a detailed search suggests that a technology trail was traced either through a cell phone or an IP address.

Google has responded to our request to comment on suggestions that the company may have provided information leading to the capture of the suspect via his Gmail account by saying:

“Users can sign up for Gmail accounts without providing any information about their identity, and Google does not seek to determine the identity of Gmail users. We do not publicly share information about which users or email addresses are or are not the subject of law enforcement requests.”

While users do not have to provide their details on an account, it was apparent in this case that the real identity wasn’t associated with the account (that would have been too easy). Google are refusing to confirm or deny that Karadzic was using a Gmail account, and thus are not going to confirm or deny if the account hosted with Google played a role in his capture.

Google has previously (along with Yahoo and other web companies) played a role in assisting both the US and foreign governments with tracking down users in criminal cases. In one specific case Yahoo CEO Jerry Yang was criticized over the role Yahoo played in relaying information to the Chinese government that resulted in the arrest and detention of a ‘dissident’ blogger.

It is well known that other fugitives make innovative use of technology to communicate while remaining hidden. For instance the Taliban and Al Quada are known to use satellite telephones in Afghanistan and Iraq to co-ordinate their activities, along with public email services with simple steganography tools (the ability to hide messages within images, video or sound files). These technologies and the web are a double-edged sword for authorities as on one hand it is known that in the USA and throughout Europe there is a level of communication monitoring, while on the other terrorists and fugitives use the same tools to keep themselves concealed and to continue operations.

The big questions arise when it comes to the privacy of users, especially with the growing trends of both web applications and cloud services. In each of the known cases to date, the information was provided volunteeringly by the respective corporation as opposed to being processed through courts and international laws. The companies hosting these services are taking a role of arbiters of justice by involving themselves in both international politics and the internal politics of foreign nations.

Note: we removed the Gmail username that was linked to the Karadzic identity he used on his website

Chris Saad, a co-founder of the Data Portability project has posted that tomorrow at OSCON a new Open Data Web Foundation will be announced by David Recordon and others.

The goal of the new foundation is to set out the actual data specifications, legal structures around data portability and in helping to evangelize set formats. Saad says that the initiative is different to the Data Portability project in that it is details oriented around specific technology and legal implementations rather than the broader evangelizing effort that has come out of Data Portability:

It seems like the foundation is well placed to provide a much needed level of oversight and legal protection for fledgling open standards. These standards will ultimately contribute to the ‘data portability’ vision of an inter-operable, standards-based web of data.

While Saad is diplomatic in his response, I can’t help but think that the efforts around standards and data portability being split into multiple groups is the ultimate definition of irony. The Data Portability project has seen great results today with companies such as Google, MySpace and Facebook participating and backing its charter. The results to date have been applications such as Friend Connect and an overall establishment of goodwill between companies who previously competed with their own data and user silo’s.

A long running problem in messaging and consistency from advocates of both open source and standards has been the duplicate and overlapping efforts. The best recent example was the split within the RSS camp that resulted in a new Atom syndication format, which in the long-term did not manage to displace RSS and instead divided evangelism efforts. While a similar split along technology lines does not exist in the case of the new Open Data Foundation and the Data Portability project, it would seem that a more united and single-branded front would be more appropriate considering the shared agenda of both camps.

The division of labor seems to be that the new Open Data Foundation will focus on technical specs and formats as a form of umbrella group covering protocol specific efforts such as RSS, Atom, OpenID, oAuth etc. We don’t have the details of what is being announced yet, but the initial response from Saad does not bode well for an initiative that has managed to achieve so much in so little time.

A week ago we reported that Apple had finally filed suit against Mac clone maker Psystar. Apple claimed that Psystar was trading on the Apple brand and illegally releasing the Mac OS X operating system on clone hardware. Today we have learnt via Macblogs of a new clone maker, Open Tech, who are planning on filling the void that is likely to be soon left by Psystar.

Open Tech are taking more precautionary measures than Psystar, who traded openly from Florida, by setting up their company and domain hosting in the tiny atoll nation of Tokelau. The atolls, a former British Protectorate, measure only 10 square kilometers (5 square miles in funny units), have a population of 1,500 and an annual GDP of only $1.5M USD. It seems that the domain registry business associated with their .tk TLD has increased their annual GDP by 10% a year.

Domain name and host information reveals that the website is hosted in Germany and the company behind the domain is registered in The Netherlands. It may be that the company operates in Europe but has shielded itself behind a tiny nation in the Pacific.

The fact that Open Tech is setting up in such an esoteric location is a sure sign that they expect the wrath of Apple to come down hard and are taking precautionary measures. The website claims that their product range will retail soon, with a starting price of $620 USD for a clone machine with about four times the power and storage of a similarly-priced Mac mini. There is no information on where manufacturing takes place (I doubt they are setup on one of the Atolls) nor who is behind the company. We have sent an email to their PR contacts to get more information and will be updating this post accordingly.

The good computer hackers are similar to war veterans in that they spend hours talking about how great the scene used to be “back in the day”. I first heard similar stories way back in 1995 but today it is actually somewhat true as the final HOPE conference has now been held, Black Hat is now a corporate event, the good e-zines are long gone and hacking is now associated with Russian crime gangs.

Now is a good time to go back and list the top 10 hackers who shaped both the computer and security industries and who have left their mark.

1. Shawn Fanning – What was once an IRC handle became synonymous with the free exchange of music files. Along with Jordan Ritter and Shaun Parker changed the music industry forever and set the course for the next decade of online media. Managed to turn a cool rock metal band into a bunch of whining babies and had his high moment when he appeared on the MTV music awards wearing a Metallic t-shirt. ADM were also the best and most consistent group ever (wikipedia).

2. Robert Morris – Discovered the buffer overflow and exploited both sendmail and fingerd with the first ever internet worm – now known as the Morris Worm. Went on to become a co-founder of ViaWeb with Paul Graham. Downside is that his story was responsible for Dade Murphy and the Gibson OS, but it also bought us Angelina Jolie. (wikipedia)

3. VallaH - jolt.c and the ping-of-death kept kids around the world amused for years and big corporations on their knees. The first smart denial-of-service attack.

4. Gordon Lyon / Fyodor – Created Nmap, the
first tool in every hackers arsenal that has since gone on to make cameo appearances in both The Matrix and the Bourne Ultimatum. Nmap pioneered OS and service fingerprinting. Also behind insecure.org. (wikipedia)

5. Kevin Mitnick – Most famous real-life hacker who could have used a few tips on how cellular triangulation works. Imprisoned by the government without charge for years and restricted with a gag order thereafter. FREE KEVIN became a hacker freedom call for years. (wikipedia)

6. The Mentor – Wrote the Hacker Manifesto which was published in Phrack magazine. Inspired kids worldwide to hack to learn (myself included). Phrack magazine deserves a mention as it was the best and most consistant e-zine in the past two decades. Ran for 65 issues until 2005.

7. Karl Koch / August Diehl – German hacker in the 80s who drove himself insane with cocaine abuse, obsession over the number 23 and paranoia over the Illuminatus. (wikipedia)

8. Electron / Richard Jones – Aussie kid who wrecked havoc all over the world with nothing more than a modem. Arrested by the Australian Federal Police in 1990. Went on to become a security researcher and consultant.

9. Kevin Poulson / Dark Dante – Phreaked his way to a new Porsche with a radio contest on an LA station. Arrested and jailed and went on to become a journalist and editor at Wired. (wikipedia)

10. Adrian Lamo – showed the world (literally) that you can take down large websites with just URLs (such as Worldcom). Now also a journalist. (wikipedia)

What is amazing is just how many of the same guys who used to sit on IRC day and night discovering security holes and breaking into servers for fun ended up shaping and influencing the IT industry as a whole. There are hundreds of thousands of hackers who you have never heard about, mostly because they never got caught, who went on to start companies and attain high positions within corporations.

The attack relies on poor random sequence generation and poor port selection used in parent lookup requests. A DNS server querying up the chain for the response to a request authorizes the response based on the port it is being received on (which is usually easy to guess) and a sequence ID – which is often not very random. An attacker simply sends thousands of ‘response’ packets back to the requesting server after forcing a lookup, and if the correct sequence number is guessed then the server will cache the response for as long as the TTL specifies.

The attack is incredibly simple, and exploits a core weakness in the way that the domain name system operates. This attack is not new either, as a number of other researchers have pointed out. The flaws in the DNS system have long been known, and script for guessing sequence numbers for forged responses have been around for years. DNS poisoning attacks are difficult to detect and can lead to simple large-scale phishing attacks on vulnerable servers.

If you are running a DNS server the best course of action is to contact your vendor or to point your parent requests to OpenDNS, who are not vulnerable to this method. A full description of the attack is included below:

Reliable
DNS Forgery in 2008: Kaminsky’s Discovery
from Matasano Chargen by ecopeland
0.
The cat is out of the bag. Yes, Halvar Flake figured out the flaw Dan
Kaminsky will announce at Black Hat.
1.
Pretend for the moment that you know only the basic function of DNS —
that it translates WWW.VICTIM.COM into 1.2.3.4. The code that does this
is called a resolver. Each time the resolver contacts the DNS to
translate names to addresses, it creates a packet called a query. The
exchange of packets is called a transaction. Since the number of
packets flying about on the internet requires scientific notation to
express, you can imagine there has to be some way of not mixing them
up.
Bob goes to to a deli, to get a sandwich. Bob walks up to the counter,
takes a pointy ticket from a round red dispenser. The ticket has a
number on it. This will be Bob’s unique identifier for his sandwich
acquisition transaction. Note that the number will probably be used
twice — once when he is called to the counter to place his order and
again when he’s called back to get his sandwich. If you’re wondering,
Bob likes ham on rye with no onions.
If you’ve got this, you have the concept of transaction IDs, which are
numbers assigned to keep different transactions in order. Conveniently,
the first sixteen bits of a DNS packet is just such a unique
identifier. It’s called a query id (QID). And with the efficiency of
the deli, the QID is used for multiple transactions.
2.
Until very recently, there were two basic classes of DNS
vulnerabilities. One of them involves mucking about with the QID in DNS
packets and the other requires you to know the Deep Magic.
First, QIDs.
Bob’s a resolver and Alice is a content DNS server. Bob asks Alice for
the address of WWW.VICTIM.COM. The answer is 1.2.3.4. Mallory would
like the answer to be 6.6.6.0.
It is a (now not) secret shame of mine that for a great deal of my
career, creating and sending packets was, to me, Deep Magic. Then it
became part of my job, and I learned that it is surprisingly trivial.
So put aside the idea that forging IP packets is the hard part of
poisoning DNS. If I’m Mallory and I’m attacking Bob, how can he
distinguish my packets from Alice’s? Because I can’t see the QID in his
request, and the QID in my response won’t match. The QID is the only
thing protecting the DNS from Mallory (me).
QID attacks began in the olden days, when BIND simply incremented the
QID with every query response. If you can remember 1995, here’s a
workable DNS attack. Think fast: 9372 + 1. Did you get 9372, or even
miss and get 9373? You win, Alice loses. Mallory sends a constant
stream of DNS responses for WWW.VICTIM.COM. All are quietly discarded
—- until Mallory gets Bob to query for WWW.VICTIM.COM. If Mallory’s
response gets to your computer before the legitimate response arrives
from your ISP’s name server, you will be redirected where Mallory tells
you you’re going.
Obvious fix: you want the QID be randomly generated. Now Alice and
Mallory are in a race. Alice sees Bob’s request and knows the QID.
Mallory has to guess it. The first one to land a packet with the
correct QID wins. Randomized QIDs give Alice a big advantage in this
race.
But there’s a bunch more problems here: * If you convince Bob to ask
Alice the same question 1000 times all at once, and Bob uses a
different QID for each packet, you made the race 1000 times easier for
Mallory to win. * If Bob uses a crappy random number generator, Mallory
can get Bob to ask for names she controls, like WWW.EVIL.COM, and watch
how the QIDs bounce around; eventually, she’ll break the RNG and be
able to predict its outputs. * 16 bits just isn’t big enough to provide
real security at the traffic rates we deal with in 2008.
Your computer’s resolver is probably a stub. Which means it won’t
really save the response. You don’t want it to. The stub asks a real
DNS server, probably run by your ISP. That server doesn’t know
everything. It can’t, and shouldn’t, because the whole idea of DNS is
to compensate for the organic and shifting
nature of internet naming and addressing. Frequently, that server has
to go ask another, and so on. The cool kids call this “recursion”.
Responses carry another value, too, called a time to live (TTL). This
number tells your name server how long to cache the answer. Why?
Because they deal with zillions of queries. Whoever wins the race
between Alice and Mallory, their answer gets cached. All subsequent
responses will be dropped. All future requests for that same data,
within the TTL, come from that answer. This is good for whoever wins
the race. If Alice wins, it means Mallory can’t poison the cache for
that name. If Mallory wins, the next 10,000 or so people that ask that
cache where WWW.VICTIM.COM is go to 6.6.6.0.
3.
Then there’s that other set of DNS vulnerabilities. These require you
to pay attention in class. They haven’t really been talked about since
1997. And they’re hard to find, because you have to understand how DNS
works. In other words, you have to be completely crazy. Lazlo Hollyfeld
crazy. I’m speaking of course of RRset poisoning.
DNS has a complicated architecture. Not only that, but not all name
servers run the same code. So not all of them implement DNS in exactly
the same way. And not only that, but not all name servers are
configured properly.
I just described a QID attack that poisons the name server’s cache.
This attack requires speed, agility and luck, because if the “real”
answer happens to arrive before your spoofed one, you’re locked out.
Fortunately for those of you that have a time machine, some versions of
DNS provide you with another way to poison the name server’s cache
anyway. To explain it, I will have to explain more about the format of
a DNS packet.
DNS packets are variable in length and consist of a header, some flags
and resource records (RRs). RRs are where the goods ride around. There
are up to three sets of RRs in a DNS packet, along with the original
query. These are: * Answer RR’s, which contain the answer to whatever
question you asked (such as the A record that says WWW.VICTIM.COM is
1.2.3.4) * Authority RR’s, which tell resolvers which name servers to
refer to to get the complete answer for a question * Additional RR’s,
sometimes called “glue”, which contain any additional information
needed to make the response effective.
A word about the Additional RR’s. Think about an NS record, like the
one that COM’s name server uses to tell us that, to find out where
WWW.VICTIM.COM is, you have to ask NS1.VICTIM.COM. That’s good to know,
but it’s not going to help you unless you know where to find
NS1.VICTIM.COM. Names are not addresses. This is a chicken and egg
problem. The answer is, you provide both the NS record pointing
VICTIM.COM to NS1.VICTIM.COM, and the A record pointing NS1.VICTIM.COM
to 1.2.3.1.
Now, let’s party like it’s 1995.
Download the source code for a DNS implementation and hack it up such
that every time it sends out a response, it also sends out a little bit
of evil — an extra Additional RR with bad information. Then let’s set
up an evil server with it, and register it as EVIL.COM. Now get a bunch
of web pages up with IMG tags pointing to names hosted at that server.
Bob innocently loads up a page with the malicious tags which coerces
his browser resolve that name. Bob asks Alice to resolve that name.
Here comes recursion: eventually the query arrives at our evil server.
Which sends back a response with an unexpected (evil) Additional RR.
If Alice’s cache honors the unexpected record, it’s 1995 —- buy CSCO!
—- and you just poisoned their cache. Worse, it will replace the “real”
data already in the cache with the fake data. You asked where
WWW.EVIL.COM was (or rather, the image tags did). But Alice also “found
out” where WWW.VICTIM.COM was: 6.6.6.0. Every resolver that points to
that name server will now gladly forward you to the website of the
beast.
4.
It’s not 1995. It’s 2008. There are fixes for the attacks I have
described.
Fix 1:
The QID race is fixed with random IDs, and by using a strong random
number generator and being careful with the state you keep for queries.
16 bit query IDs are still too short, which fills us with dread. There
are hacks to get around this. For instance, DJBDNS randomizes the
source port on requests as well, and thus won’t honor responses unless
they come from someone who guesses the ~16 bit source port. This brings
us close to 32 bits, which is much harder to guess.
Fix 2:
The RR set poisoning attack is fixed by bailiwick checking, which is a
quirky way of saying that resolvers simply remember that if they’re
asking where WWW.VICTIM.COM is, they’re not interested in caching a new
address for WWW.GOOGLE.COM in the same transaction.
Remember how these fixes work. They’re very important.
And so we arrive at the present day.
5.
Let’s try again to convince Bob that WWW.VICTIM.COM is 6.6.6.0.
This time though, instead of getting Bob to look up WWW.VICTIM.COM and
then beating Alice in the race, or getting Bob to look up WWW.EVIL.COM
and slipping strychnine into his ham sandwich, we’re going to be clever
(sneaky).
Get Bob to look up AAAAA.VICTIM.COM. Race Alice. Alice’s answer is
NXDOMAIN, because there’s no such name as AAAAA.VICTIM.COM. Mallory has
an answer. We’ll come back to it. Alice has an advantage in the race,
and so she likely beats Mallory. NXDOMAIN for AAAAA.VICTIM.COM.
Alice’s advantage is not insurmountable. Mallory repeats with
AAAAB.VICTIM.COM. Then AAAAC.VICTIM.COM. And so on. Sometime, perhaps
around CXOPQ.VICTIM.COM, Mallory wins! Bob believes CXOPQ.VICTIM.COM is
6.6.6.0!
Poisoning CXOPQ.VICTIM.COM is not super valuable to Mallory. But
Mallory has another trick up her sleeve. Because her response didn’t
just say CXOPQ.VICTIM.COM was 6.6.6.0. It also contained Additional RRs
pointing WWW.VICTIM.COM to 6.6.6.0. Those records are in-bailiwick: Bob
is in fact interested in VICTIM.COM for this query. Mallory has
combined attack #1 with attack #2, defeating fix #1 and fix #2. Mallory
can conduct this attack in less than 10 seconds on a fast Internet link.

The Public Interest Registry, operators of the .org top-level domain name, was today granted permission from ICANN to implement Domain Name Security Extensions (DNSSEC). DNS is inherently insecure, which has become more apparent recently with security issues found in most server implementations (to do with easily guessing client ports used for responses) and the lack of a trust mechanism that results in DNS poisoning (the process of injecting a false response to a DNS lookup and redirecting the user to another site)

DNSSEC adds extensions to the DNS protocol that provide a layer of authorization between requesting clients and DNS servers. The extensions provide a way for clients to check the authenticity of a response to protect against both poisoning and other redirection methods used in man-in-the-middle and phishing attacks.

The move could be the impetus for the other TLD managers to also adopt the emerging standard. DNSSec-Deployment, an advocacy website for DNSSEC, estimates that up to 10% of DNS servers on the web today are vulnerable to simple attack techniques that could compromise a visitors web traffic. If DNSSEC works out for the .org group we could see it implemented broadly across .com and .net (with Verisign) and a more secure DNS system for the web.

Forking comment thread here. Thanks!

Today at Techcrunch we announced that we are building our own web tablet hardware device. This all stems back from a conversation a few weeks ago when we were discussing the ultimate web browsing/cloud computing client hardware. The iPhone is nice but too small, and most laptops are over-powered for the task. With applications on the web most of us just need a web browser most of the time, so the ideal device would be a light-weight small tablet running nothing more than Firefox on a decent screen and with a WiFi connection.

The software development aspects of the project will be managed here from TCIT. Our goal is open source from top to bottom (including the full design, eventually). Leave a comment and we will get in touch with you and send you an account on the project management system we will be setting up (I am also nik at techcrunch on email which might be easier than flooding comments). It would be interesting to hear general feedback and ideas – we really want to open this up to everybody.

The planned stack so far is to run BSD or Linux, with the Gnome desktop. We will probably take the Gnome Onscreen Keyboard project and adapt that as the primary input device (the hardware design includes multi-touch under the LCD screen, I will have the full specs shortly and will post them to the wiki). Then there will be Firefox, running in a stripped down interface mode with a simple system tray showing battery life and wifi (and simple settings for the device). Plugins would include Gears, Flash and probably either VLC or Mplayer with open codecs for media.

Once the stack comes together and we can set the hardware spec in stone, we will do a small manufacturing run and ship some devices out to developers so that we can work on specifics. Once that is done there will be a larger manufacturing run with hopefully a retail price of $300 or less. If you are excited about this as we are, please get in touch.

COMMENTS CLOSED HERE. PLEASE CONTINUE AT NEXT POST

The Live Mesh team at Microsoft announced today that P2P storage between Mesh nodes is now available in the latest client update. Previously storage on the Mesh worked by utilizing the 5GB allocated to each user on the central hosting platform at Microsoft. Users are now able to sync their files between connected nodes automatically.

The new technology works by synching the files between computers, but the metadata is still stored centrally (although it does not count towards a users quota usage). All Mesh objects and folders can be copied between devices directly, except for the Live Desktop. The video below is from the Live Mesh team talking about the P2P technology and how they solve conflict and other issues that have plagued other P2P storage attempts in the past.

David Steere and Trevor Robinson: How Live Mesh P2P Syncing Works

As we wrote previously, the applications currently on Mesh.com are implementations of a broader development platform being built and developed based on open formats and standards. Live Mesh has been available to anybody with a Microsoft Passport for a few weeks now, with Vista and XP clients available for Live Mesh and a Mac OS X client is in the works.

Android is the new highly-anticipated “open source” mobile platform that is currently under development at Google. There are a high expectations of Android as the first viable open source based mobile platform and a viable competitor to the dominance of Symbian, Microsoft and now Apple. Android is based on Linux and developers can write applications in managed code that use the Google Android SDK, which is developed in Java.

Initially with the Android SDK, there were three ways that a developer could communicate with other services or devices. The first was through text messaging with SMS (or MMS), with HTTP (or the secure variant) or with XMPP, the open messaging protocol. This list has since been struck down to two and a half options, with the XMPP implementation with Android being replaced with a more generic GTalk client library. This means that all real-time communication on Android not using SMS or HTTP must pass through the Gtalk servers.

The initial XMPP implementation was provided via the Smack library, an XMPP implementation developed by Jive Software. Developers started complaining back in January of this year that the implementation had been changed, and that it wasn’t adequate for implementing messaging services on Android. Google has since responded and the official line now is that a more specific library for GTalk is more than adequate, and they also claim that full XMPP isn’t very good for mobile anyway.

Android started as an idealistic open source platform, with a large number of hardware manufacturers lined up supporting it. It is now becoming just a Google platform for mobile, as forcing Gtalk on developers has a chain effect of enforcing Google ID’s on users and all communication to pass through Google servers. Developers are becoming increasingly frustrated by first denying Android SDK updates to some and now by closing up and deciding what can and can’t be done on the platform. Instead of being an open platform, they seem to want to funnel everything through Google services.

As one commentor on the developer thread said, what is next? GHTTP, GSMS etc? Google sucked up credibility by claiming to be the open platform alternative for mobile, but they are burning that credibility up at a fantastic rate with the decisions they are making.

Since launching TechcrunchIT we have been pounding away about open standards and data availability. One of the biggest victims of this focus recently has been Twitter, who went from being a leading light in the field, to closing everything up and now finally today have gone back to being open again. Gnip has announced this morning that they now have access to the Twitter XMPP feed, and that they will be re-publishing it and making it available to everybody.

We first wrote about Gnip when they launched. At the time they supported a large number of data providers but the obvious missing provider was Twitter, who had at the time shut off access to their XMPP ‘firehose’.

This is an important move for a number of reasons. First, it allows users to access their data from any other application including IM – its no longer up to Twitter in terms of what can be done with the data, which should result in some interesting applications and mashups. Second it better establishes XMPP as the standard for communications messages on the web. Previously applications integrating with Twitter were polling its API and checking for changes every x minutes, much like RSS. You only have to look at the widget in the sidebar of this very blog to see the problems with that – the requests are throttled, data is dropped and it results in 98% of traffic to a server not actually carrying any data. With XMPP and the push model, all that is redundant. XMPP is the HTTP for communications on the web, and soon we will look back at the days of polling a web service and wonder what the hell we were smoking.

Last of all, this is a perfect case study for the next applicaiton built in terms of what to do and what not to do. One of the biggest drivers of user adoption at Twitter was the large number of applications that were available to give users access to the data. This opens up user choice and a market around a platform, and the only way a firmly established platform can be displaced is either that a better solution comes alone, or more likely, the platform owners theselves screw it up by demanding too much from users.

The AMD Q2 earnings report today recorded a $1.2B loss for the second quarter, as the company replace CEO Ruiz with Dirk Meyer. The board voted out Ruiz, who has been CEO since moving to the company from Motorola in 2001. Dirk Meyer is taking over as CEO from his current position as both COO and President. Meyer has been with the company for 12 years, and presided over microprocessor design at the firm. Ruiz is out after a painful term at AMD, which has seen heavy losses in the past year.

It is definitely a day where attention is reverted back to the old IT giants as Microsoft today announced in their Q4 earnings results that they have seen the biggest earnings-per-share growth since 1999. Revenue reached $60 Billion for the first time, with $15.48 Billion for the quarter, an increase of 18% over last year.

Microsoft COO Kevin Turner said:

“Delivering $60 billion in annual revenue is an outstanding accomplishment and a testament to the powerful combination of great technology solutions and strong execution by our partners and global sales and marketing teams,” .. “The outlook for fiscal year 2009 is positive given the breadth of our impressive technology portfolio and the expanding collection of online services we are bringing to market.”

In a day that saw Google tank and IBM grow, the titans of tech seem to have returned. Despite the results, Microsoft shares have still softened in after-market trading after finishing the trading day up 1.5%

Moments after the disappointing results from Google, IBM announced their Q2 results and their reported revenue increase beat out analyst expectations. IBM stock is up 0.5% in after-market trading as the company reported second-quarter earnings of $2.8 Billion, up 22% on the same time last year. Revenue rose 13% to $26.8 Billion. IBM have again upgraded their earnings forecast for the year to $8.75.

Sales in the Americas rose 8%, and they also grew strongly in other parts of the world. IBM is selling more services and earning more in a weak US economy, and the company has beaten the expectations of all on Wall Street.

Google announced their Q2 results today after the market had closed. Shares are down by as much as 12% as earnings growth reported was below analysts expectations. Net income for the quarter was $1.25B, up from $925M. The company reported a profit of $4.63 per share, which was slightly below the $4.72 average expectation from Wall Street analysts.

Todays fall is an indication that the economic slowdown is affecting Google, which has experienced its largest one-day fall in stock price since it went public in 2004. The US economy grew only 1% in the last quarter, down from 3.4% from a year ago. The ad market is expected to only grow at a rate of 2% this year – and with Google and Yahoo together making up 95% of the search market, the impact is expected to be felt throughout the web sector. Google has a high reliance on their search advertising revenues, with no other formidabble revenue streams. For that reason the company, and its growth expectations, are heavily affected by the slightest changes in the vulnerable advertising market.

The good news for Google is that web and search spending continue to take up a higher proportion of a shrinking ad market. The bad news is that the growth rate of web spending as overall ad spending is not adequate to make up for the high expectations set on Google. The company is also yet to realize real revenues from its application suite, which is where it has set out to compete directly with Microsoft and their profitable Office suite of products. While Google has won large contracts for email hosting, it is still just a drop in the ocean compared to the larger office and enterprise markets.

Related: Meanwhile IBM Q2 beats expectations >>

When Gmail first launched, it was lauded for its spam filtering and security – there was no spam leaving the systems and almost no spam coming in. But all that has changed as a study from Roaring Penguin has found that spam originating from Gmail has risen from 6.8 to 27% in just the last month. The impetus for the big jump is likely to be that the Google account registration CAPTCHA has been completely defeated and almost doesn’t even stand in the way anymore.

We pronounced that CAPTCHA’s were dead some weeks ago, as there are now hundreds of thousands of ready-to-go accounts for Gmail, Hotmail and Yahoo now available to purchase online. With Gmail now responsible for 27% of spam, the trust mechanisms have completely broken down to the point where Google must take some drastic action to filter out automated users who are sending out bulk email.

They could do that by throttling the number of outbound emails, and some common sense user tracking. I don’t think any real user would be sending thousands of emails an hours, like most of the spamming scripts do – so it shouldn’t be too hard to detect and shutdown. Google have a big problem on their hands, and it will affect other Gmail users as the email coming out of the system is now being treated with increased suspicion by other mail handlers.