Support Engineering Group : NoSQL
This page last changed on Jan 08, 2013 by kgomes.

Why NoSQL?

Here is a tidbit on why you might use NoSQL over RDBMS from Martin Fowler's NoSQL Distilled book:

  1. Application development productivity. A lot of application development effort is spent on mapping data between in-memory data structures and a relational database. A NoSQL database may provide a data model that better fits the application's needs, thus simplifying that interaction and resulting in less code to write, debug, and evolve.
  2. Large-scale data. Organizations are finding it valuable to capture more data and process it more quickly. They are finding it expensive, if even possible, to do so with relational databases. The primary reason is that a relational database is designed to run on a single machine, but it is usually more economic to run large data and computing loads on clusters of many smaller and cheaper machines. Many NoSQL databases are designed explicitly to run on clusters, so they make a better fit for big data scenarios.

Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. xiv). Pearson Education (US). Kindle Edition.

You can go to http://nosql-database.org or http://nosql.mypopescu.com/kb/nosql to get an up-to-date list of the NoSQL data products and the categories they fall in to.

Fowler also give this as a criteria for NoSQL:

"there are two primary reasons for considering NoSQL. One is to handle data access with sizes and performance that demand a cluster; the other is to improve the productivity of application development by using a more convenient data interaction style."

Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. 12). Pearson Education (US). Kindle Edition.

And finally, Fowler summarizes:

  1. Relational databases have been a successful technology for twenty years, providing persistence, concurrency control, and an integration mechanism.
  2. Application developers have been frustrated with the impedance mismatch between the relational model and the in-memory data structures.
  3. There is a movement away from using databases as integration points towards encapsulating databases within applications and integrating through services.
  4. The vital factor for a change in data storage was the need to support large volumes of data by running on clusters. Relational databases are not designed to run efficiently on clusters.
  5. NoSQL is an accidental neologism. There is no prescriptive definition— all you can make is an observation of common characteristics.
  6. The common characteristics of NoSQL databases are
    1. Not using the relational model
    2. Running well on clusters
    3. Open-source
    4. Built for the 21st century web estates
    5. Schemaless
  7. The most important result of the rise of NoSQL is Polyglot Persistence.

Fowler, Martin; Sadalage, Pramod J. (2012-08-08). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence (p. 12). Pearson Education (US). Kindle Edition.

Here are some notes about various NoSQL products

Product Written in Main Point License Protocol Best Used For Example Features Not Good For
MongoDB C++ Retains some friendly properties of SQL (Query, index) AGPL (Drivers:Apache) Custom, binary (BSON)
  1. Master/slave replication (auto failover with replica sets)
  2. Sharding built-in
  3. Queries are javascript expressions
  4. Run arbitrary javascript functions server-side
  5. Better update-in-place than CouchDB
  6. Uses memory mapped files for data storage
  7. Performance over features
  8. Journaling (with --journal) is best turned on
  9. On 32bit systems, limited to ~2.5Gb
  10. An empty database takes up 192Mb
  11. GridFS to store big data + metadata (not actually an FS)
  12. Has geospatial indexing
 If you need dynamic queries. If you prefer to define indexes, not map/reduce functions. If you need good performance on a big DB. If you wanted CouchDB, but your data changes too much, filling up disks. For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back.  
Riak Erlang & C, some JavaScript Fault Tolerance Apache HTTP/REST or custom binary
  1. Tunable trade-offs for distribution and replication (N, R, W)
  2. Pre- and post-commit hooks in JavaScript or Erlang, for validation and security.
  3. Map/reduce in JavaScript or Erlang
  4. Links & link walking: use it as a graph database
  5. Secondary indices: but only one at once
  6. Large object support (Luwak)
  7. Comes in "open source" and "enterprise" editions
  8. Full-text search, indexing, querying with Riak Search server (beta)
  9. In the process of migrating the storing backend from "Bitcask" to Google's "LevelDB"
  10. Masterless multi-site replication replication and SNMP monitoring are commercially licensed
 If you want something Cassandra-like (Dynamo-like), but no way you're gonna deal with the bloat and complexity. If you need very good single-site scalability, availability and fault-tolerance, but you're ready to pay for multi-site replication. Point-of-sales data collection. Factory control systems. Places where even seconds of downtime hurt. Could be used as a well-update-able web server.
  1. Use this only if you know the keys for the key-value store as you cannot search within the value.
CouchDB Erlang DB consistency, ease of use Apache HTTP/REST
  1. Bi-directional replication, continuous or ad-hoc
    with conflict detection, thus, master-master replication.
  2. MVCC - write operations do not block reads
  3. Previous versions of documents are available
  4. Crash-only (reliable) design
  5. Needs compacting from time to time
  6. Views: embedded map/reduce
  7. Formatting views: lists & shows
  8. Server-side document validation possible
  9. Authentication possible
  10. Real-time updates via _changes
  11. Attachment handling thus, CouchApps (standalone js apps)
  12. jQuery library included
 For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. CRM, CMS systems. Master-master replication is an especially interesting feature, allowing easy multi-site deployments.  
Redis C/C++ Blazing fast BSD Telnet-like
  1. Disk-backed in-memory database, currently without disk-swap (VM and Diskstore were abandoned)
  2. Master-slave replication
  3. Simple values or hash tables by keys, but complex operations like ZREVRANGEBYSCORE.
  4. INCR & co (good for rate limiting or statistics)
  5. Has sets (also union/diff/inter)
  6. Has lists (also a queue; blocking pop)
  7. Has hashes (objects of multiple fields)
  8. Sorted sets (high score table, good for range queries)
  9. Redis has transactions
  10. Values can be set to expire (as in a cache)
  11. Pub/Sub lets one implement messaging
For rapidly changing data with a foreseeable database size (should fit mostly in memory). Stock prices. Analytics. Real-time data collection. Real-time communication.  
HBase Java Billions of rows X millions of columns Apache HTTP/REST (also Thrift)
  1. Modeled after Google's BigTable
  2. Uses Hadoop's HDFS as storage
  3. Map/reduce with Hadoop
  4. Query predicate push down via server side scan and get filters
  5. Optimizations for real time queries
  6. A high performance Thrift gateway
  7. HTTP supports XML, Protobuf, and binary
  8. Cascading, hive, and pig source and sink modules
  9. Jruby-based (JIRB) shell
  10. Rolling restart for configuration changes and minor upgrades
  11. Random access performance is like MySQL
  12. A cluster consists of several different types of nodes
 Hadoop is probably still the best way to run Map/Reduce jobs on huge datasets. Best if you use the Hadoop/HDFS stack already. Analysing log data.  
Neo4j Java Graph database - connected data GPL, some features AGPL/commercial HTTP/REST (or embedding in Java)
  1. Standalone, or embeddable into Java applications
  2. Full ACID conformity (including durable data)
  3. Both nodes and relationships can have metadata
  4. Integrated pattern-matching-based query language ("Cypher")
  5. Also the "Gremlin" graph traversal language can be used
  6. Indexing of nodes and relationships
  7. Nice self-contained web admin
  8. Advanced path-finding with multiple algorithms
  9. Indexing of keys and relationships
  10. Optimized for reads
  11. Has transactions (in the Java API)
  12. Scriptable in Groovy
  13. Online backup, advanced monitoring and High Availability is AGPL/commercial licensed
 For graph-style, rich or complex, interconnected data. Neo4j is quite different from the others in this sense. Social relations, public transport links, road maps, network topologies.  
Cassandra Java Best of BigTable and Dynamo Apache Custom, binary (Thrift)
  1. Tunable trade-offs for distribution and replication (N, R, W)
  2. Querying by column, range of keys
  3. BigTable-like features: columns, column families
  4. Has secondary indices
  5. Writes are much faster than reads
  6. Map/reduce possible with Apache Hadoop
  7. All nodes are similar, as opposed to Hadoop/HBase
 When you write more than you read (logging). If every component of the system must be in Java. ("No one gets fired for choosing Apache's stuff.") Banking, financial industry (though not necessarily for financial transactions, but these industries are much bigger than that.) Writes are faster than reads, so one natural niche is real time data analysis.  
Membase Erlang & C Memcache compatible, but with persistence and clustering Apache 2.0 memcached plus extensions
  1. Very fast (200k+/sec) access of data by key
  2. Persistence to disk
  3. All nodes are identical (master-master replication)
  4. Provides memcached-style in-memory caching buckets, too
  5. Write de-duplication to reduce IO
  6. Very nice cluster-management web GUI
  7. Software upgrades without taking the DB offline
  8. Connection proxy for connection pooling and multiplexing (Moxi)
 Any application where low-latency data access, high concurrency support and high availability is a requirement. Low-latency use-cases like ad targeting or highly-concurrent web apps like online gaming (e.g. Zynga).  
Document generated by Confluence on Feb 03, 2026 16:22