Dec 20, 2011
Historical Architecture: Data Mining Billions of Tweets
The DataSift platform allows users to define a “stream” using filtering parameters such as keywords or locations. Users can immediately begin to receive data in real time as comments are posted on social media sites. With a license to access Twitter’s full “Firehose”, we offer users the ability to search for posts using all the metadata contained in a Tweet, making it a far more powerful search. Though we make the search available via the simple Datasift API, there’s actually quite a bit that goes into it.
In fact, we’re making the technology behind the DataSift platform available for historical data, too. Developers can now run queries against stored data and export the results, and all of this is through the same Conceptual Schema Definition Language (CSDL) and interface.
more http://blog.programmableweb.com/2011/12/07/historical-architecture-data-mining-billions-of-tweets/