“Everyone is building custom stuff right now”
The quote in this post’s title is from Gnip’s CEO on their challenges of handling a sustained 35MB/sec stream of constant updates from Twitter. Gnip is the only partner that receives the entire Twitter data stream for analysis.
“Everyone is building custom stuff right now” echoes what I’m seeing at companies handling big data like Clearspring. Although NoSQL data stores and tools like Hadoop are gaining mainstream acceptance, the companies really handling big data don’t have tools they can use out of the box to perform analysis. The big data trends are just beginning to take shape and no one is yet offering the right solutions to handle them.
Link: Gnip CEO on the Challenges of Handling the Real-time, Big Data Firehouse (ReadWriteWeb)
