xero

"One of the coolest things about FriendFeed is how you can see everything that's happening on the site in real-time. Starting today, this is even more true with real-time search. Just enter a search query as you always have, and see new results instantly stream in at the top of your screen in real-time. No need to constantly click refresh. It's an extremely useful (not to mention mesmerizing) way to immediately see what people are saying about a particular topic." - xero via Bookmarklet

"Refollow helps you discover, manage, and protect your Twitter social circle." - xero via Bookmarklet

Xero that is the best Twitter follower manager I've seen yet. - Keith (aka Tsudohnimh)

I've been wanting Twitter to implement all of this stuff, but they seem to not care about their UI or tools anymore. Then again, obviously someone else will build whatever is needed, so I can't blame them. - xero

"Camera and music applications aside, the iPhone and iPod touch hold a lot of potential for creative users. Today, we look at another well-represented genre of creative tools: 15 different drawing and painting applications, ranging in price from free to $6. While they vary in utility and execution, and most are useful only for casual doodling, each allows the user to create art using nothing but the touchscreen and a finger. Some of the programs even offer intriguingly broad sets of tools and novel features such as wireless collaboration." - xero via Bookmarklet

Interview with Jan Lehnardt for CouchDB. Apache CouchDB, the distributed, schema-free, document-oriented database accessible via a RESTful HTTP/JSON API. - xero

Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API. Among other features, it provides robust, incremental replication with bi-directional conflict detection and resolution, and is queryable and indexable using a table-oriented view engine with JavaScript acting as the default view definition language. - xero

Podcast episode on CouchDB, interview with Jan Lehnardt http://twit.tv/floss36 - Micah Wittman

Thanks Micah! - xero

HBase is the Hadoop database. Its an open-source, distributed, column-oriented store modeled after the Google paper, Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Hadoop. HBase's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. Try it if your plans for a data store run to big. - xero

As its name implies, NoSQL is not an SQL database but rather a shell-level tool, as explained in Philosophy of NoSQL. NoSQL data is contained in regular UNIX ASCII files, and so can be manipulated by regular UNIX utilities, e.g. ls, wc, mv, cp, cat, head, more, less, editors like 'vi,' etc., as well as by powerful versioning systems, such as RCS and CVS. The form of each file of data is that of a relation, or table, with rows and columns of information. To extract information, a file of data is fed to one or more "operators" via the UNIX Input/Output redirection mechanism. There are also programs to generate, modify, and validate the data. A more through discussion of why this type of relational database structure makes sense is found in the book, "UNIX Relational Database Management", in the references section. - xero

Dynomite is an eventually consistent distributed key value store written in Erlang. The design is based off of Amazon’s Dynamo paper. - xero

Voldemort is not a relational database, it does not attempt to satisfy arbitrary relations while satisfying ACID properties. Nor is it an object database that attempts to transparently map object reference graphs. Nor does it introduce a new abstraction such as document-orientation. It is basically just a big, distributed, persistent, fault-tolerant hash table. For applications that can use an O/R mapper like active-record or hibernate this will provide horizontal scalability and much higher availability but at great loss of convenience. - xero

This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon’s core services use to provide an “always-on” experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use. - xero

Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google's BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems. - xero

Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable. - xero

Hypertable is an open source project based on published best practices and our own experience in solving large-scale data-intensive tasks. Our goal is to bring the benefits of new levels of both performance and scale to many data-driven businesses who are currently limited by previous-generation platforms. Our goal is nothing less than that Hypertable become one of the world’s most massively parallel high performance database platforms. - xero

There is currently considerable enthusiasm around the MapReduce (MR) paradigm for large-scale data analysis [17]. Although the basic control flow of this framework has existed in parallel SQL database management systems (DBMS) for over 20 years, some have called MR a dramatically new computing model [8, 17]. In this paper, we describe and compare both paradigms. Furthermore, we evaluate both kinds of systems in terms of performance and development complexity. To this end, we define a benchmark consisting of a collection of tasks that we have run on an open source version of MR as well as on two parallel DBMSs. - xero

Their conclusion? Databases "were significantly faster and required less code to implement each task, but took longer to tune and load the data," the researchers write. Database clusters were between 3.1 and 6.5 times faster on a "variety of analytic tasks." - xero

MongoDB is a high-performance, open source, schema-free document-oriented database. MongoDB is written in C++ and offers the following features: * Collection oriented storage - easy storage of object-style data * Dynamic queries * Full index support, including on inner objects * Query profiling * Replication and fail-over support * Efficient storage of binary data including large objects (e.g. videos) * Auto-sharding for cloud-level scalability (Q209) A key goal of MongoDB is to bridge the gap between key/value stores (which are fast and highly scalable) and traditional RDBMS systems (which are deep in functionality). - xero