SSD acceleration support on Linux
Block caching strategies
Read cache only (One way, write-around)
Changed blocks are written only and directly to the cached device. The SSD only invalidates its entry and fetch written data when a read request occurs. Only benefits for high read/write ratios.
Read and write cache (Full cache, write-through)
Changed blocks are written to both devices.
Read cache with buffered writes (write-back)
Blocks are written to both devices but delayed on the cached device, allowing writes reordering and optimizations.
Block caching modules
bcache
Linux kernel block layer cache, now mainstream.
flashcache
Facebook’s own contribution module to disk acceleration.
dm
The kernel device mapper. It is not intented to act as block caching/accelerator, but a trick makes SSD usable (or any other fast device) for acceleration purposes.
In fact, the point here is not to designate a cache (fast) device, but the slow ones: This is the write-mostly option that was primary intented to flag network block devices.
Features
I/O bottleneck redirection
When using a block cache for a accelerating a pool of devices, the cache device might not be able to withstand the total amount of requested I/O and then have the reverse effect. Detecting I/O bottlenecks allows overriding the cache device and directing I/O directly to the backend devices. A hard disk can withstand a max of 200 IOPS, where a SLC SSD handles 5000 IOPS. Probably effectless with a 25 disks less pool then.
TRIM
TRIM/discard support allows the filesystem to flag the physical block as no longer used. Otherwise, the device still see unallocated blocks as in use, and will experience performance decrease and write amplification problem as the block usage grows.
TRIM over mapper or RAID
This is the ability of passing trim commands to virtual devices, which then needs to issue the right/translated TRIM commands as well.
Write amplification
If the write amplification will happen by design, whether the device handles TRIM or not.
bcache | flashcache | dm (mostly-write) |
|
mainstream (kernel) | NO | NO | YES |
cache size/device size ratio |
<=1.0 | <=1.0 | 1.0 |
write-around strategy |
NO | YES | YES |
write-through strategy | YES | YES | NO |
write-back strategy | YES | YES | NO |
I/O bottleneck redirection |
YES | NO | NO |
TRIM support | YES | NO
(Work In Progress) |
YES (upper layer passthrough) |
RAID/DM TRIM support | YES (2.6.37) |
||
write amplification issue | NO | YES (lack of trim support from upper layer) |
YES (caused by 1:1 mirroring) |
cellspacing=”2″ height=”145″ width=”1135″>
<tbody>
<tr>
<td style=”vertical-align: top; font-weight: bold;”><br>
</td>
<td
style=”vertical-align: top; font-weight: bold; background-color: rgb(204, 204, 204);”>directly
attached<br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>LVM</strong><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>DM</strong><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>LVM+DM</strong></td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>mainstream</strong>
(kernel)</td>
<td style=”vertical-align: top;”><br>
</td>
<td style=”vertical-align: top;”>NO</td>
<td style=”vertical-align: top;”>NO</td>
<td style=”vertical-align: top;”>YES</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>cache
size/device size ratio</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td style=”vertical-align: top;”><=1</td>
<td style=”vertical-align: top;”><=1</td>
<td style=”vertical-align: top;”>1</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-around</strong>
strategy<strong>
</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-through</strong>
strategy</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>write-back</strong>
strategy</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>overcommiting</strong><strong>
</strong></td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO
(W.I.P.)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>TRIM</strong>
support</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO
(W.I.P.)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(upper layer passthrough)</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><strong>RAID/DM
TRIM </strong>support</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(2.6.36)</td>
</tr>
<tr>
<td
style=”vertical-align: top; background-color: rgb(204, 204, 204);”><span
style=”font-weight: bold;”>write amplification </span>issue</td>
<td style=”vertical-align: top;”><br>
</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>NO</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(without trim support from upper layer)</td>
<td
style=”vertical-align: top; background-color: rgb(255, 255, 255);”>YES
(caused by 1:1 mirror)</td>
</tr>
</tbody>
</table>