Guest Post: Intel Optane and In-Memory Databases

Today we’ll be publishing a guest post, co-authored by Konstantin Boyandin, a system administrator from Novosibirsk.

Konstantin worked with in-memory databases (memacached, Redis, Apache ignite, etc.) on a server built with an Intel Optane SSD. The specific device tested was the Intel Optane SSD DC P4800X and was connected using IMDT (Intel Memory Drive Technology), which made the drive available to the OS and applications as RAM.

Memory Drive Technology can also be used for managing large virtual machine farms. In this case, the hefty amount of RAM could arguably make software testing much easier and less expensive.

NVMe on its own is nothing new, but NVMe with Memory Drive Technology, now that’s something. For one of Konstantin’s projects, efficiently processing massive data arrays is key. The test results were extremely reassuring: DRAM with Intel Memory Drive Technology significantly reduces the amount resources required for processing.

We hope you find Konstantin’s experience interesting.

Description

We looked at how well the Intel Optane with IMDT performs. Performance was tested for Redis and memcached with tests from the Phoronix testing package, and RAM performance was tested using RAMspeed from the same package. For the sake of comparison, testing occurred twice on the same server with the same OS: once with the Intel Optane with IMDT, and once with the drive disabled.

The testing server ran CentOS 7.4 (64-bit) and was built on the following components:

• motherboard: Supermicro X10SRi-F
• RAM: 64 GB (8×8), 2133 MHz
• CPU: Intel(R) Xeon(R) E5-1650 v4 (3.60 GHz)
• hard disk (SATA): Intel SSD DC S3700 (480 GB)
• NVMe drive: Intel Optane SSD DC P4800X (375 GB)

The first phase of the tests was performed without restarting.

Brief overview of aforementioned DBMS:

• Memcached is used for storing and quickly retrieving data in key-value format; it was initially created to accelerate the blog service LiveJournal in 2003.
• Redis (short for “remote dictionary server”) is a network loggable database designed for quickly processing data. In this case, it was configured to work strictly with the RAM without dumping WAL (write-ahead logging) on the disk.

Purpose: To compare DBMS performance for RAM storage with IMDT and without the NVMe drive.

Preparation

For our experiment, we chose to run RAMspeed (RAM operations), redis (Redis performance), and mcperf (memcached performance) under CentOS 7.4 (64-bit).

To preserve the integrity of our test results, the OS did not access the swap in either case.

The Phoronix test suite 7.4 (and its included redis and mcperf tests) package and RAMspeed 3.5 package were used.

For the Intel Optane with IMDT (hereinafter “IMDT”) configuration, 320 GB of RAM were made available to the OS and applications.

For the configuration without the Intel Optane (hereinafter “RAM”), 64 GB of RAM were made available to the OS and applications.

In both instances, each test was run three times and values averaged. All of the services to be tested were restarted before each round.

Measurements and Results

For testing purposes, we used memtest. Between 50-70% of the memory space not in use by the test was filled at random.

Ramfs Sequential Recording

The first test measured how RAM performed as ramfs (sequential recording) using the following commands:

IMDT:

mkdir -p /mnt/ram
mount -t ramfs -o size=300G ramfs /mnt/ram
dd if=/dev/zero of=/mnt/ram/256gb.dat bs=4096 count=67108864

Total:
274877906944 bytes (275 GB) copied, 184.756 s, 1.5 GB/s

RAM:

mkdir -p /mnt/ram
mount -t ramfs -o size=62G ramfs /mnt/ram
dd if=/dev/zero of=/mnt/ram/60gb.dat bs=4096 count=15728640

Total:
64424509440 bytes (64 GB) copied, 18.0382 s, 3.6 GB/s

The ramfs file system was unmounted before all other tests were run.

Redis

While testing, we measured the number of DB commands (SADD, LPUSH, SET, GET) that could be executed per second.

The bigger the value, the higher the performance.

• IMDT — in the Intel Optane configuration, 320 GB of RAM were available to applications
• RAM — 64 GB of RAM were available to applications while Intel Optane was disabled

Mcperf

While testing, we checked how many commands per second could be executed for 1 and 5 simultaneous connections (speed was measured in commands per second for each connection).

RAMspeed

Tests were run for 4 simultaneously working threads. Memory block sizes are 4096 bytes. The bigger the value (MB per second), the higher the performance.

Conclusion

These tests demonstrated that despite the apparent differences in in write speeds (using the standard dd command), DBMS performance for storing data to the RAM while using an Intel Optane with IMDT was comparable.

The advantage of having NVMe with IMDT is that it’s otherwise nearly impossible to get the same amount of memory on most servers simply by installing RAM modules.

In terms of cost-performance (using 8 GB Kingston DDR4 2133MHz RAM modules), the same amount of memory as an Intel Optane drive runs about half the price (keep in mind that the Optane used in these tests was not ideal in terms of cost-performance).

As we see it, where Memory Drive Technology can be used (meaning a large amount of RAM is required), it’s either technically impossible to install that much memory (motherboards don’t offer that kind of support) or the configuration would be at least twice as expensive.

Supposing NVMe SSDs with IMDT drop in price (they’re still relatively rare now and fairly costly), the cost-performance advantage will be even more apparent.

Using NMVe SSDs with IMDT is a good prospect when you need a lot of RAM (using major databases like Redis/memcached, maintaining multiple virtual machines on one host, etc.).

In the tests we ran, performance was considered fair enough. Building a simulation of the testing environment like the one we had intended required too much time.

If you have any interesting ideas for uses of the Intel Optane, feel free to sign up to be a tester at SelectelLab.