This is not a big technology already, but can be very usefull for solid understanding of architecture and features this drives
Today Hitachi Data Systems is announcing the new all-flash Hitachi Virtual Storage Platform (VSP) F Series and enhanced models of the Hitachi VSP G series storage offerings with the next-generation Hitachi flash modules with inline data compression (FMD DC2) and enhanced Hitachi Automation Director and Data Center Analytics tools for improved response times and greater effective capacity.
My colleagues, Mike Nalls will be providing more details on the all flash VSP F series and Bob Madaio will cover our overall strategy and direction for flash. In this post I will cover the enhancements in our new Flash Module Device the FMD DC2, which includes data compression and is in position to displace performance disk drives.
The FMD DC2 Architecture
Flash drives require a lot of software and a lot of processing power for mapping pages to blocks, wear leveling, extended ECC, data refresh, housekeeping, and other management tasks which can limit performance, degrade durability, and limit the capacity of flash device. In order to support these processing requirements, the FMD from Hitachi Data Systems is built with a quad core multiprocessor, with 8 lanes of PCIe out the front and integrated flash controller logic, which supports 32 paths to the flash array. Having direct access to the engineering resources of Hitachi Ltd., Hitachi Data Systems is able to deliver patented new technology in this next generation FMD, which sets it further apart from competitive flash vendors and displaces the performance disk market with lower cost, higher performance, enterprise flash.
FMD DC2 Displaces High Performance Disk
This new generation FMD DC2 flash device, doubles the raw capacity of our previous generation FMD from 3.2 TB to 6.4 TB and, with the use of compression provides 4x more effective capacity for a lower TCO without penalties in performance or scalability. This higher capacity FMD DC2 is 28% lower than 15K RPM disk drives on a relative bit, street price, comparison and as much as 64% lower with a 2:1 compression ratio. The lower price is the result of a higher capacity 6.2 TB, lower 5 year support, power, cooling, and floor space costs. The FMD DC2 with 2.1 compression is even lower than the entry 10K RPM disk drives. The FMD DC2 essentially displaces 15K RPM disks and, depending on the actual compression ratios, is even lower than 10K RPM disks!
Designed for Performance.
The processing power and multi-pathing architecture of the FMD DC2 not only enables us to double the capacity of our previous FMD and double it again with compression, but also adds functions that increase performance. The FMD DC2 increases read IOPs by 50% while maintaining less than 1 ms response time even at PB scale with the new VSP F and enhanced G series storage controllers.
By virtualizing the flash capacity, the FMD manages exactly how and where data is stored so that both read and write IO is executed as fast as possible. The FMD also offloads tasks from the VSP storage controller to the flash devices. This distributes data service tasks, reducing system overhead and preserving more processing power on the VSP for tasks like replication. In the first FMD we offloaded inline write avoidance, which increased write performance.
The FMD DC2 features a new “always on” inline compression offload engine which eliminates the performance impact of compression/decompression that other vendors experience by doing this in the storage controller. It is a very-large-scale integration (VLSI) engine that enables lossless compression based on a derivative of the LZ77 sliding-window algorithm. The encoding is designed for high performance, using a small-memory footprint. Its parallel processing delivers real-time compression/decompression using a systolic array-content addressable memory architecture, which ensures there is higher throughput, better efficiency and no lag that can create response time spikes. The compression engine enables similar data reduction efficiency to the software compression algorithms found in other all-flash arrays. The difference is that the FMD engine performs at 10 times the speed of other implementations and is run on the FMDs and not in the storage controller. This eliminates the overhead associated with typical software implementations on Intel core and/or journal-based file systems. Since compression in the storage controller may impact flash performance, users try to manage which data streams are compressed and which are not in order to maximize performance and capacity. This is an added level of administration, which the FMD DC2 does not require. Since compression and decompression is done in the FMD all the benefits of compressed capacity is delivered with no impact to performance. Just turn it on and forget it.
The FMD also maintains sustained write performance by preserving reserved capacity for background tasks like garbage collection and wear levelling. The FMD eliminates the “write cliff” where housekeeping tasks like garbage collection blocks the write I/O, by taking these tasks out of the I/O path with its multipath architecture.
The FMD has a patented method to accelerate reads by allowing the system to read directly from the flash device and bypass the storage controller cache. This also minimizes the system overhead that can result in longer latencies.
Another patent improves read / write performance by providing multiple, parallel connections between the FMD controller and flash modules. This allows the controller to access a separate flash module if one is busy.
Designed for Durability.
One of the arguments against flash has been the durability of flash versus disk drives. Disk drives can be over written almost indefinitely (10¹⁵ overwrites) while flash cells wear out (10³ overwrites) and eventually lose their ability to retain electrons. However, with proper management, flash drives can be more durable than Disk drives.
The FMD exploits the differences in the failure mode of flash versus magnetic disk. Disks records data on physical blocks that are formatted on the disk. If the blocks get damaged through interdictions such as head crashes, the whole disk must be replaced. With flash, data is recorded on virtual pages, which can be relocated to another physical page if the original page is no longer usable. The FMD provides 25% extra capacity for spares, so the FMD does not need to be replaced until all the spare pages are used up.
Every FMD is designed to monitor flash storage for issues so that they can be resolved quickly, preventing long-term outages and data loss. Since flash technology can only sustain a specified number of write/formats a key design consideration is to reduce the number of unnecessary writes.
Compression helps by reducing the number of blocks that need to be written.
The FMD has a block write avoidance feature which can recognize any data stream of all “0 s” or all “1 s” in real time and remap that data with a pointer. This not only eliminates writes but also effectively increases the over provisioning space for spares.
ECC (Error Correction Code) has been extended to preserve the integrity of data writes and correct up to 59 bits per 2KB of compressed data which exceeds the MLC (Multi Level Cell) spec of 40 bits per 1.1KB of data. This ensures that even if a bit error is discovered it can be easily corrected. This correction enhances the ability to monitor the degradation of pages and avoids any premature page rewrites. (There are actually two levels of ECC. The first level of ECC is done per transaction and provides end-to-end error checking through a custom eight byte DIF (Data Integrity Field) appended to write I/O recorded on the FMD DC2. This ensures that what was sent to the FMD is what was received.)
The FMD manages wear leveling at both the page and block level so that leveling occurs across a much broader capacity – reducing failures and overhead.
Virtualizing the flash capacity enables the ability to control exactly where data is stored for better wear leveling and to mask normal cell failures. Virtualization of flash capacity also allows us to prevent write inefficiencies by coalescing and allocating a page / block depending on need.
If an FMD reaches 95% of its specified write endurance, a service information message (SIM) is issued to prompt the replacement of the FMD. A service processor (SVP) or a report from Hitachi Storage Navigator can confirm the write capacity of the FMD. FMDs are covered by a 5-year warranty but as long as a customer pays for system maintenance they are covered.