NoSQL Databases

by K. Yue

1. Introduction

• NoSQL:
  • “Not Only SQL” or “Not SQL.” Relatively new kinds of non-relational database systems.
  • A loose category of many diverse database systems with different data models.
• Some common NoSQL features:
  1. Non-relational
  2. Distributed
  3. High scalability
  4. High availability
  5. High performance and parallelism
  6. No or flexible schema
  7. Weaker support of ACID properties
  8. Oriented towards semi-structured or non-structured data.
  9. Weaker support of data integrity and constraints.
  10. Usually more object-oriented.
  11. More analysis-oriented than transaction-oriented.
  12. Simpler APIs for database interaction.
• Major non-relational database systems as in top db-engines ranking, https://db-engines.com/en/ranking (rankings as of 9/1/2023):
1. Document models: e.g. MongoDB (rank #5), CouchBase (32), Firebase Realtime Database (38), CouchDB (44), Realm (56)
2. Key-value models: e.g. Redis (6), Memcached (33), etcd (52), HazelCast (55), LevelDB (111).
3. Wide-column models: e.g. Cassandra (12), Hive (17), HBase (26), Datastax Enterprise (60)
4. Graphical models: Neo4J (22)
   
• Some observations:
1. Search engines (not really a data model): ElasticSearch (8; document model), Splunk (14; key-value), Solr (24; document model)
2. Still dominated by relational databases.
3. Many DB support multiple models.
• Read:
  • A simple introduction to NoSQL database: https://www.guru99.com/nosql-tutorial.html
• Introduction to NoSQL: https://www.geeksforgeeks.org/introduction-to-nosql/

2. Advantages and Disadvantages

• Some advantages of NoSQL databases in general:
  1. Distributed
  2. High scalability: horizontal scalability, as opposed to vertical scalability.
     1. Horizontal scalability (scaling out): add more machines to the distributed system.
     2. Vertical scalability (scaling up): add more power to existing machines; replace existing machines by more powerful one.
  3. High availability
  4. High performance
  5. Flexibility: flexible schema or schemaless
  6. More object-oriented:
     1. Better abstraction model
     2. Better interoperability with programming language. No need of object-relational mapping.
• Some disadvantages of NoSQL databases in general:
  1. Weaker data integrity support
  2. Weaker transaction support
  3. Weaker theoretical and design methodology support
  4. Relative lack of standards
  5. Relative lack of tools and interoperability
  6. Complexity

3. ACID versus BASE Transaction Models

3.1 ACID

• Relational database support ACID properties to support data consistency and integrity of transactions under concurrent access: e.g., http://en.wikipedia.org/wiki/ACID
• ACID properties (review):
  1. Atomicity: A transaction is an atomic unit of processing. It is either performed in its entirety or not performed at all.
  2. Consistency preservation: A correct execution of a transaction must take the database from one physically consistent state to another. This is known as physical consistency.
  3. Isolation: A transaction should not make its updates visible to other tasks and transactions until it is committed.
  4. Durability or permanency: Once a transaction changes the database and the changes are committed, these changes must never be lost because of subsequent failure.
• Supporting ACID limits other desirable features: scalability, availability, and performance.

3.2 BASE

• To enhance scalability, availability and performance, most NoSQL DB do not fully support ACID.
• NoSQL supports different transaction models.
• The Basic Availability, Soft-State, Eventual Consistency (BASE model) for distributed database is the most popular one.
  1. Basic available: data is basically available across nodes of the distributed database, despite network failures.
  2. Soft-state: There is no immediate consistency. As a result, different replicas may have different values across the distributed systems at a given time. Thus, the state of the database is soft.
  3. Eventual consistency: eventually, data replicas will have the same value across the distributed database.

3.3 CAP Theorem

• Since NoSQL databases are mostly distributed, it is important to have some understanding of the famous CAP theorem for distributed data stores.
• See, for example, https://en.wikipedia.org/wiki/CAP_theorem.
• There are three desirable guarantees of distributed data stores, CAP:
  1. Consistency: the return value is always the same for the same data across the distributed systems.
  2. Availability: every request will return a response, either the data or an error. (Note that the return data may or may not always be the same).
  3. Partition tolerance: the database continues to operate in case of network partitions (one partition of the network cannot communicate with another partition because of message drops).
• The CAP theorem states that any distributed database can provide only two out of the three guarantees.
• Different databases based their designs on prioritizing two out of the three C-A-P.
• For a more detailed discussion, one may see:  https://www.instaclustr.com/blog/cassandra-vs-mongodb/ (optional read):
  1. It contains a discussion how Cassandra and Mongo trade-off CAP.
  2. It also includes a discussion of a more refined CAP theorem: PACELC Theorem: "PACELC is summarized as follows: In the event of a partition failure, a distributed system must choose between Availability (A) and Consistency (C), else (E) when running normally it must choose between latency (L) or consistency (C)."

4. Major NoSQL data models

4.1 Key-value model

1. Data is stored as key-value pairs.
2. Values can possibly be JavaScript Object Notation (JSON) strings, which store serialized objects.
3. Some key-value databases support rich JSON queries.

4.2 Document Model

1. Document-oriented databases store data as documents.
2. Documents can be considered as semi-structured data.
3. Thus, XML databases can be considered as employing the document model.
4. Modern document-oriented databases commonly employ JSON. E.g., MongoDB and CouchDB.
5. The document model can be considered as a subclass of key-value model.
   1. The stored value of a key-value model can be a document.
   2. The stored value can be manipulated by operations based on the selected document model (mostly JSON).
6. MongoDB is likely the most popular document-oriented NoSQL DB. It will be covered in more details in this class.

Example: In CouchDB, a key-value pair may be:

Key: "MBSEBaseModel~939c7672-5d2d-11ec-bf63-0242ac130002"

Value to be stored:

{
   "BCAssetId": "939c7672-5d2d-11ec-bf63-0242ac130002",
   "BCAssetType": "MBSEBaseModel",
   "BCAssetName": "Gateway-PPE-Base-Model",
   "BaseModelDesc": "PPE project's model.",
   "version": {
      "version": "2.1",
      "subversion": "4.6",
      "startTime": "2021-12-08T17:25:23+06:00"
   },
   "storage": {
      "isEncrypted": true,
      "EncrypMethod": "AES256",
      "EncrypKey": "q4t7w!z%C*F-JaNdRgUkXp2r5u8x/A?D",
      "useIPFS": true,
      "IPFSCid": "QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX",
      "IPFS_HashHead": "A0Xa",
      "payloadRaw": "raw PPE MBSE Base model description. V2.1.4.6.",
   }
}

Note:

• CouchDB adds two fields, _id and _rev, automatically if they are not supplied.
• The field _rev is used for multi-version concurrency control (MVCC) to ensure 'eventual consistency.'
• Some may consider CouchDB as a document-oriented database. The boundary between key-value model and document model is not clear cut.

To query CouchDB, one may use many methods. Examples:

1. CouchDB RESTful API: https://docs.couchdb.org/en/latest/api/index.html
2. MapReduce-based views: https://docs.couchdb.org/en/latest/ddocs/views/index.html
3. Mango query

An example Mango query that returns all CouchDB key-value pairs for MBSEBaseModel.

{
    "selector": {
        "BCAssetType": { "$eq": "MBSEBaseModel" }
    }
}

See https://docs.couchdb.org/en/latest/api/database/find.html for more information about selector syntax.

4.3 Wide-Column Model

1. A columnar DBMS or column-oriented DBMS stores data tables grouped by columns, instead of grouped by rows (as in most relational DBMS).
   1. For example, related columns may be stored together in a file for faster performance.
   2. Benefits:
      1. Faster access for certain types of queries.
      2. Better chance for data compression.
   3. Disadvantages:
      1. Slower update.
      2. Slower access for certain types of queries.
2. Read introductions to the wide-column model. Examples:
   1. https://dandkim.com/wide-column-databases/
3. In wide column model, data is stored as keys and columns. Each column contains a column-name and a value.
4. Thus, to get a data value, use (key, column-name).
5. Cassandra is one of the most popular wide-column databases.

4.4 Graphical Model

1. "A graph database stores nodes and relationships instead of tables, or documents."
2. Quite object-oriented, using a directed graph model.
3. neo4j is the most popular graphical database.
4. Introduction: https://neo4j.com/developer/graph-database/.
5. To start learning Neo4j, download and install Neo4j desktop.
6. Neo4j Query Language: Cypher, https://neo4j.com/developer/cypher/.
7. Basic Cypher syntax: (nodes)-[:ARE_CONNECTED_TO]->(otherNodes).