Twitter Shuts Off the Firehose as Data Analytics Moves In-house

mobile dataTwitter has terminated its existing agreements with third-party companies reselling its firehose data, the complete, unfiltered stream of tweets and metadata, in an effort to grow its own in-house big data business.

The social network has been looking to increase its revenues from areas outside advertising, and in 2014 acquired data analytics company Gnip, which now largely serves as its big data team, providing data companies, brands and other firms with Twitter figures that can be used to measure metrics like consumer sentiment and market trends.

With the help of Gnips expertise, Twitter is already licencing its data to firms such as IBM, Saleforce and Oracle, and its revenue from data sales had increased 105 per cent year-on-year in Q4 2014.

“One of the reasons Twitter acquired Gnip was because Twitter believes the best way to support the distribution of Twitter data is to have direct data relationships with its data customers – the companies building analytic solutions using Twitters data and platform,” said Zach Hofer-Shall, head of Twitter Ecosystem.

“Direct relationships help Twitter develop a deeper understanding of customer needs, get direct feedback for the product roadmap, and work more closely with data customers to enable the best possible solutions for the brands that rely on Twitter data to make better decisions.”

Shutting down the firehose for third-party companies who provide similar services, such as DataSift and NTT Data, could well bring thousands more businesses to Twitters analytics services, but places doubt on the future of all the other companies affected now they can no longer directly access this data.

Nick Halstead, CEO and founder of DataSift, seemed confident about the firms ability to keep functioning without the firehose data, saying that “our business model has never relied on access to Twitter data. Weve built a robust big data processing platform which is data source-agnostic, capable of dealing with billions of interactions a day from 20 other social and news networks were working with.”

However, he also said that Twitter had “seriously damaged the ecosystem” with the move, and that the social network “doesnt understand the basic rules of this market”.

Array