This is guest post by Sachin Sinha who is passionate about data, analytics and machine learning at scale. Author & founder of BangDB.

This article is to simply report the YCSB bench test results in detail for five NoSQL databases namely Redis, MongoDB, Couchbase, Yugabyte and BangDB and compare the result side by side. I have used latest versions for each NoSQL DB and have followed the recommendations to run all the databases in optimized conditions. I have also used the default six test scenarios as defined by the YCSB framework. I have restricted it to 10M records for each test. However, user can run the bench for as many numbers as they practically find suitable.

About YCSB

Following configurations were used for the evaluation purpose.

Each of these workload test runs in two steps, 1. Load and 2. Run. Load stage is to load the data and then run stage we run the test. I have run each test with clean database to reflect the numbers in fair manner

Summary

To summarize the test, here is the high-level report of the tests, load and run both. 50953465046 12b5de7c31 z

50953465106 e33f71d18e o

Load is consistent for all dbs for all tests as expected as this phase is to load the data. Run phase is where each db is tested for different test conditions.

Workload A: Update heavy workload

This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.

50952764148 447cbbc419 z

50952764203 bfb3cc53f9 o

The first graph shows the ops/sec (throughput) for the 10M records. However the second chart shows how quickly the test was completed by DBs.

50952785978 4f4d684eba

We note that for MongoDB update latency is really very low (low is better) compared to other dbs, however the read latency is on the higher side.

Workload B: Read mostly workload

This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags.

50953465121 2e8ffe0436 z
50953465176 5b0165fb5c z

The latency table shows that 99th percentile latency for Yugabyte is quite high compared to others (lower is better)

50953498056 4484b01fa8 z

Workload C: Read only

This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).

50952764223 c0c6f8ae6b z
50953465216 1ed8b8fa08 z

The latency table is following for test C and since it was read only test hence there is no update latency figure here. Again Yugabyte latency is quite high

50952804673 119f62be77 z

Workload D: Read latest workload

In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest.

50953570397 778289c174 z
50953465241 5ab29fa0e4 z

The latency table for test D is as below. Here Redis and Yugabyte have higher latencies, Yugabyte performs bad for both Insert and Read for the test

50952819068 b1039e9e52

Workload E: Short ranges

In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).

50953570447 0306c47057 z
50952764258 e8569bbb26 z

This requires a lots of scans, hence the throughput decreases for all dbs, however Redis is slowest, understandably so as it is also reflected in the latency table below. Yugabyte performs really good in this test

50953562136 f848b211f4 z

Workload F: Read-modify-write

In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity.

50952764283 54e3a20fe0 z
50953570482 07da5258b4 z

Here Yugabyte has high latency esp for 99th percentile for Update and Read-modify-write. However, Very high Read latency for MonoDB makes it the last db to finish the task

50952825938 cc35f93d8b

Conclusion

While each database has been designed for different goals and use cases, YCSB test provides somewhat a common ground for the benchmark, therefore the numbers shown in this document can be used by developers or users to help select the db suitable for their requirement. All of these dbs are available free of cost for download / install and it will be fairly straightforward to run these tests in your environment for further analysis. The tests can be modified or added in order to cover a set of specific scenarios as needed.

HighScalability?d=yIl2AUoC8zA HighScalability?d=ZMKU6pOd71k

V1UBTQoHi70