SSD acceleration support on Linux

 

Block caching strategies

Read cache only (One way, write-around)

Changed blocks are written only and directly to the cached device. The SSD only invalidates its entry and fetch written data when a read request occurs. Only benefits for high read/write ratios.

Read and write cache (Full cache, write-through)

Changed blocks are written to both devices.

Read cache with buffered writes (write-back)

Blocks are written to both devices but delayed on the cached device, allowing writes reordering and optimizations.

Block caching modules

bcache

Linux kernel block layer cache, now mainstream.

flashcache

Facebook’s own contribution module to disk acceleration.

dm

The kernel device mapper. It is not intented to act as block caching/accelerator, but a trick makes SSD usable (or any other fast device) for acceleration purposes.

In fact, the point here is not to designate a cache (fast) device, but the slow ones: This is the write-mostly option that was primary intented to flag network block devices.

Features

I/O bottleneck redirection

When using a block cache for a accelerating a pool of devices, the cache device might not be able to withstand the total amount of requested I/O and then have the reverse effect. Detecting I/O bottlenecks allows overriding the cache device and directing I/O directly to the backend devices. A hard disk can withstand a max of 200 IOPS, where a SLC SSD handles 5000 IOPS. Probably effectless with a 25 disks less pool then.

TRIM

TRIM/discard support allows the filesystem to flag the physical block as no longer used. Otherwise, the device still see unallocated blocks as in use, and will experience performance decrease and write amplification problem as the block usage grows.

TRIM over mapper or RAID

This is the ability of passing trim commands to virtual devices, which then needs to issue the right/translated TRIM commands as well.

Write amplification

If the write amplification will happen by design, whether the device handles TRIM or not.

bcache flashcache dm (mostly-write)
mainstream (kernel) NO NO YES
cache size/device size ratio
<=1.0 <=1.0 1.0
write-around strategy
NO YES YES
write-through strategy YES YES NO
write-back strategy YES YES NO
I/O bottleneck redirection
YES NO NO
TRIM support YES NO

(Work In Progress)

YES
(upper layer passthrough)
RAID/DM TRIM support YES
(2.6.37)
write amplification issue NO YES
(lack of trim support from upper layer)
YES
(caused by 1:1 mirroring)

 

<table style=”text-align: left; width: 100%;” border=”1″ cellpadding=”2″
cellspacing=”2″ height=”145″ width=”1135″>
<tbody>
<tr>
<td style=”vertical-align: top; font-weight: bold;”><br>
</td>
<td
style=”vertical-align: top; font-weight: bold; background-color: rgb(204, 204, 204);”>directly
attached<br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>LVM</strong><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>DM</strong><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>LVM+DM</strong></td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>mainstream</strong>
(kernel)</td>
<td style=”vertical-align: top;”><br>
</td>
<td style=”vertical-align: top;”>NO</td>
<td style=”vertical-align: top;”>NO</td>
<td style=”vertical-align: top;”>YES</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>cache
size/device size ratio</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td style=”vertical-align: top;”>&lt;=1</td>
<td style=”vertical-align: top;”>&lt;=1</td>
<td style=”vertical-align: top;”>1</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-around</strong>
strategy<strong>
</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-through</strong>
strategy</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-back</strong>
strategy</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>overcommiting</strong><strong>
</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO
(W.I.P.)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>TRIM</strong>
support</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO
(W.I.P.)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(upper layer passthrough)</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>RAID/DM
TRIM </strong>support</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(2.6.36)</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><span
style=”font-weight: bold;”>write amplification </span>issue</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(without trim support from upper layer)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(caused by 1:1 mirror)</td>
</tr>
</tbody>
</table>