Support Engineering Group : NoSQL This page last changed on Jan 08, 2013 by kgomes. Why NoSQL? Here is a tidbit on why you might use NoSQL over RDBMS from Martin Fowler's NoSQL Distilled book: Application development productivity. A lot of application development effort is spent on mapping data between in-memory data structures and a relational database. A NoSQL database may provide a data model that better fits the application's needs, thus simplifying that interaction and resulting in less code to write, debug, and evolve. Large-scale data. Organizations are finding it valuable to capture more data and process it more quickly. They are finding it expensive, if even possible, to do so with relational databases. The primary reason is that a relational database is designed to run on a single machine, but it is usually more economic to run large data and computing loads on clusters of many smaller and cheaper machines. Many NoSQL databases are designed explicitly to run on clusters, so they make a better fit for big data scenarios. Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. xiv). Pearson Education (US). Kindle Edition. You can go to http://nosql-database.org or http://nosql.mypopescu.com/kb/nosql to get an up-to-date list of the NoSQL data products and the categories they fall in to. Fowler also give this as a criteria for NoSQL: "there are two primary reasons for considering NoSQL. One is to handle data access with sizes and performance that demand a cluster; the other is to improve the productivity of application development by using a more convenient data interaction style." Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. 12). Pearson Education (US). Kindle Edition. And finally, Fowler summarizes: Relational databases have been a successful technology for twenty years, providing persistence, concurrency control, and an integration mechanism. Application developers have been frustrated with the impedance mismatch between the relational model and the in-memory data structures. There is a movement away from using databases as integration points towards encapsulating databases within applications and integrating through services. The vital factor for a change in data storage was the need to support large volumes of data by running on clusters. Relational databases are not designed to run efficiently on clusters. NoSQL is an accidental neologism. There is no prescriptive definition— all you can make is an observation of common characteristics. The common characteristics of NoSQL databases are Not using the relational model Running well on clusters Open-source Built for the 21st century web estates Schemaless The most important result of the rise of NoSQL is Polyglot Persistence. Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. 12). Pearson Education (US). Kindle Edition. Here are some notes about various NoSQL products
Product | Written in | Main Point | License | Protocol | Best Used | For Example | Features | Not Good For
--- | --- | --- | --- | --- | --- | --- | --- | ---
MongoDB | C++ | Retains some friendly properties of SQL (Query, index) | AGPL (Drivers:Apache) | Custom, binary (BSON) | Master/slave replication (auto failover with replica sets) Sharding built-in Queries are javascript expressions Run arbitrary javascript functions server-side Better update-in-place than CouchDB Uses memory mapped files for data storage Performance over features Journaling (with --journal) is best turned on On 32bit systems, limited to ~2.5Gb An empty database takes up 192Mb GridFS to store big data + metadata (not actually an FS) Has geospatial indexing | If you need dynamic queries. If you prefer to define indexes, not map/reduce functions. If you need good performance on a big DB. If you wanted CouchDB, but your data changes too much, filling up disks. | For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. |
Riak | Erlang & C, some JavaScript | Fault Tolerance | Apache | HTTP/REST or custom binary | Tunable trade-offs for distribution and replication (N, R, W) Pre- and post-commit hooks in JavaScript or Erlang, for validation and security. Map/reduce in JavaScript or Erlang Links & link walking: use it as a graph database Secondary indices: but only one at once Large object support (Luwak) Comes in "open source" and "enterprise" editions Full-text search, indexing, querying with Riak Search server (beta) In the process of migrating the storing backend from "Bitcask" to Google's "LevelDB" Masterless multi-site replication replication and SNMP monitoring are commercially licensed | If you want something Cassandra-like (Dynamo-like), but no way you're gonna deal with the bloat and complexity. If you need very good single-site scalability, availability and fault-tolerance, but you're ready to pay for multi-site replication. | Point-of-sales data collection. Factory control systems. Places where even seconds of downtime hurt. Could be used as a well-update-able web server. | Use this only if you know the keys for the key-value store as you cannot search within the value.
CouchDB | Erlang | DB consistency, ease of use | Apache | HTTP/REST | Bi-directional replication, continuous or ad-hoc with conflict detection, thus, master-master replication. MVCC - write operations do not block reads Previous versions of documents are available Crash-only (reliable) design Needs compacting from time to time Views: embedded map/reduce Formatting views: lists & shows Server-side document validation possible Authentication possible Real-time updates via _changes Attachment handling thus, CouchApps (standalone js apps) jQuery library included | For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. | CRM, CMS systems. Master-master replication is an especially interesting feature, allowing easy multi-site deployments. |
Redis | C/C++ | Blazing fast | BSD | Telnet-like | Disk-backed in-memory database, currently without disk-swap (VM and Diskstore were abandoned) Master-slave replication Simple values or hash tables by keys, but complex operations like ZREVRANGEBYSCORE. INCR & co (good for rate limiting or statistics) Has sets (also union/diff/inter) Has lists (also a queue; blocking pop) Has hashes (objects of multiple fields) Sorted sets (high score table, good for range queries) Redis has transactions Values can be set to expire (as in a cache) Pub/Sub lets one implement messaging | For rapidly changing data with a foreseeable database size (should fit mostly in memory). | Stock prices. Analytics. Real-time data collection. Real-time communication. |
HBase | Java | Billions of rows X millions of columns | Apache | HTTP/REST (also Thrift) | Modeled after Google's BigTable Uses Hadoop's HDFS as storage Map/reduce with Hadoop Query predicate push down via server side scan and get filters Optimizations for real time queries A high performance Thrift gateway HTTP supports XML, Protobuf, and binary Cascading, hive, and pig source and sink modules Jruby-based (JIRB) shell Rolling restart for configuration changes and minor upgrades Random access performance is like MySQL A cluster consists of several different types of nodes | Hadoop is probably still the best way to run Map/Reduce jobs on huge datasets. Best if you use the Hadoop/HDFS stack already. | Analysing log data. |
Neo4j | Java | Graph database - connected data | GPL, some features AGPL/commercial | HTTP/REST (or embedding in Java) | Standalone, or embeddable into Java applications Full ACID conformity (including durable data) Both nodes and relationships can have metadata Integrated pattern-matching-based query language ("Cypher") Also the "Gremlin" graph traversal language can be used Indexing of nodes and relationships Nice self-contained web admin Advanced path-finding with multiple algorithms Indexing of keys and relationships Optimized for reads Has transactions (in the Java API) Scriptable in Groovy Online backup, advanced monitoring and High Availability is AGPL/commercial licensed | For graph-style, rich or complex, interconnected data. Neo4j is quite different from the others in this sense. | Social relations, public transport links, road maps, network topologies. |
Cassandra | Java | Best of BigTable and Dynamo | Apache | Custom, binary (Thrift) | Tunable trade-offs for distribution and replication (N, R, W) Querying by column, range of keys BigTable-like features: columns, column families Has secondary indices Writes are much faster than reads Map/reduce possible with Apache Hadoop All nodes are similar, as opposed to Hadoop/HBase | When you write more than you read (logging). If every component of the system must be in Java. ("No one gets fired for choosing Apache's stuff.") | Banking, financial industry (though not necessarily for financial transactions, but these industries are much bigger than that.) Writes are faster than reads, so one natural niche is real time data analysis. |
Membase | Erlang & C | Memcache compatible, but with persistence and clustering | Apache 2.0 | memcached plus extensions | Very fast (200k+/sec) access of data by key Persistence to disk All nodes are identical (master-master replication) Provides memcached-style in-memory caching buckets, too Write de-duplication to reduce IO Very nice cluster-management web GUI Software upgrades without taking the DB offline Connection proxy for connection pooling and multiplexing (Moxi) | Any application where low-latency data access, high concurrency support and high availability is a requirement. | Low-latency use-cases like ad targeting or highly-concurrent web apps like online gaming (e.g. Zynga). |

Document generated by Confluence on Feb 03, 2026 16:22