ISP Agent: A Generalized In-storage-processing Workload Offloading Framework by Providing Multiple Optimization Opportunities

2.1 Cosmos+ OpenSSD Architecture

Cosmos+ OpenSSD [14] is a customizable SSD platform with a flash programmable gate array (FPGA)-based flash device controller. OpenSSD connects to the host via PCIe, and the host requests I/Os via the NVMe protocol. Figure 1(a) shows an overview of the simple architecture of the Cosmos+ OpenSSD system.

Fig. 1.

View Figure

Figure 1(b) and Algorithm 1 shows the OpenSSD I/O handling process. When a host application requests I/Os with the file information, such as file position and I/O size, the operating system (OS) first calculates the low-level storage address of the requested data, known as a logical block address (LBA). The OS generates non-volatile message express (NVMe) commands, which include the LBA and the target host memory. These commands are then transmitted to the OpenSSD, which uses slice (SSD page) units to access the NAND flash. When OpenSSD receives I/O requests from the host, it partitions an NVMe command of any I/O size into fixed-size slice requests. For each slice request, the OpenSSD performs NAND flash access and direct memory access (DMA) with the host. After processing all the slice requests, OpenSSD returns the NVMe command.

Specifically, OpenSSD uses a NAND buffer, which is a cache for NAND flash data in the DRAM of the SSD. The NAND buffer comprises several entries, where each entry caches a single slice (SSD page) of NAND flash data. Algorithm 1 demonstrates how OpenSSD handles slice requests. OpenSSD first accesses the NAND buffer by using the logical address of the target SSD page (line 1). If there is no matching NAND buffer entry (miss), then the OpenSSD retrieves the oldest NAND buffer entry from the least recently used (LRU) list (line 3) and evicts the SSD page data to the NAND flash that the entry was holding (lines 2–8). After obtaining the NAND buffer entry corresponding to the target SSD page, OpenSSD validates it. If the NAND buffer entry is invalid, then OpenSSD reads the data from the NAND flash module (lines 9–11). This NAND READ operation can be omitted if the slice request is a write and covers the entire SSD page. OpenSSD then performs a DMA operation using the valid NAND buffer entry (line 16) and updates the LRU list (line 17). As shown in Figure 1(b), every operation that includes NAND flash access and DMA is queued to the job queue of each NAND buffer entry. The operations within each job queue are executed sequentially, while those in separate job queues are performed concurrently.

2.2 In-storage Processing and ISP-assisted Database Checkpointing

ISP is the method of processing data within a storage device instead of transferring it to off-storage computation units, such as the CPU or GPU. Because most SSDs have a higher internal data bandwidth (SSD to NAND flash module) than their external data bandwidth (host to SSD), ISP is highly effective by rather exploiting the internal bandwidth. Furthermore, the utilization of workload offloading through ISP has the potential to enhance overall system performance by reducing host-storage bandwidth, host memory space, and host computation unit usage.

ISP-assisted database (DB) checkpointing [23] is a representative example of ISP. DB checkpointing is an operation executed periodically that stores dirty DB pages on a physical device. The aim is to reduce the recovery execution time, which can result in heavy I/Os and degrade foreground transaction performance. To alleviate the DB checkpointing overhead, ISP-assisted DB checkpointing has been proposed. ISP-assisted DB checkpointing gathers only the changes (deltas) in the dirty pages, transforms them into a constant workload (delta update) list, and offloads the workloads to the ISP device. It can greatly minimize I/O overheads by solely transmitting small-sized deltas to the SSD instead of entire dirty DB pages. ISP-assisted DB checkpointing does not impact foreground transactions as it updates dirty pages solely in the ISP device, which operates in the host’s background. As a result, ISP-assisted DB checkpointing can notably enhance DB performance.

Figure 2 shows the storage-side process of ISP-assisted DB checkpointing. The DB collects the changes in one DB page, packs them in the format of page ID, size, position (location), and the actual data of each delta, and writes them to the delta file. We call the packed data for one DB page a single-page delta (SPD). The delta file contains multiple consecutive SPDs. When the tasks of the DB checkpointing are transferred to the SSD, the ISP kernel reads and parses the delta file into SPDs. Based on the parsed delta, the ISP kernel reads the target page of the SPD. After reading the DB page, the ISP kernel updates the deltas into the read DB page and writes the updated DB page. Following the repetition of these processes for all SPDs (read DB page, update delta, and write DB page), the ISP kernel reads and parses the delta files. This process continues until all offloaded deltas are updated.

Fig. 2.

View Figure

3.1 ISP Requirements

Based on the ISP-assisted DB checkpointing described in Section 2.2, the following requirements are necessary for ISP offloading:

To meet the requirements, there are several challenges in exchanging data between the ISP kernel and the storage for ISP offloading.

3.2 Challenges of ISP

3.2.1 Information Gap between Host and Storage.

The first challenge is obtaining the low-level storage address (LBA) for the ISP kernel. As shown in Figure 3(a), when the host application requests file I/Os, the OS transforms it into an NVMe command and transmits it to the SSD. The NVMe command encompass the LBA of the intended data, which the OS computes based on the file-to-LBA mapping. Generally, the SSD cannot access this OS file-to-LBA mapping information. As a result, it cannot retrieve the LBAs for the storage data used by ISP kernel, which includes file-level information like file descriptor, offset, and size.

Fig. 3.

View Figure

3.2.3 NAND Flash Module Access Requirements.

Due to the limited data access capability of page size for NAND flash modules, OpenSSD divides I/O requests into slice units. Additionally, to access data, NAND flash modules necessitate address-aligned memory, an SSD DRAM area whose address is a multiple of the slice size. Unlike address-aligned memory (e.g., 0x4000), NAND flash access is accomplished abnormally with unaligned memory (e.g., 0x4123). Figure 4(a) shows that the ISP kernel necessitates arbitrary-sized data at random file locations and typically relies on unaligned memory. Therefore, buffering the NAND flash access with address-aligned memory is crucial, and reorganising input or result data is necessary when using the unaligned memory with the ISP kernel.

Fig. 4.

View Figure

Accessing NAND flash modules is subject to certain limitations. The process has to be performed on one slice (SSD page) at a time, with address-aligned memory employed as the source or destination. Address-aligned memory refers to memory whose address is a multiple of the slice. To satisfy this requirement for host I/O, OpenSSD partitions NVMe commands of arbitrary sizes into slice-sized requests and employs the NAND buffer, which consists of address-aligned memory. However, as shown in Figure 4(a), the ISP kernel requires arbitrary-sized data stored in random file locations, and typically operates with unaligned ISP kernel memory. Therefore, the ISP kernel necessitates buffering NAND flash access with address-aligned, slice-sized memory and rearranging data between the address-aligned buffer and the ISP kernel memory.

3.3 Conventional ISP Approaches

3.3.2 FIFO-ISP.

Many general-purpose ISP studies, such as Summarizer [13] and INSIDER [27], proposed FIFO-ISP approaches to solve this problem. FIFO-ISP delivers storage data to the ISP kernel based on the host I/O process. When the host program launches the ISP kernel, it specifies the input and output data of the kernel. The specified data is prepared in the storage memory in accordance with the current host I/O handling process, as shown in Figure 5(a), and the storage executes the ISP kernel with the prepared data. By managing the input and output data of the ISP kernel on the host side, FIFO-ISP can deliver storage data to the ISP kernel avoiding the problems discussed in Section 3.2. FIFO-ISP typically integrates ISP offloading with host I/O to enhance overall performance by decreasing the I/O size through ISP kernel. Furthermore, the FIFO-ISP typically pipelines data transfer (DMA) and ISP kernels, dynamically allocating workloads to both ISPs and CPUs, resulting in improved overall performance.

Fig. 5.

View Figure

However, the FIFO-ISP restricts the access of the ISP kernel to arbitrary storage data, thus limiting the types of workloads that can be offloaded. Workloads that demand I/O to an arbitrary file area or contain both input and output data in the SSD, such as ISP-assisted DB checkpointing, are not suitable for offloading by FIFO-ISP.

3.4 Insights for Generalized ISP Offloading: ISP Agent

We propose an ISP agent, a framework for general-purpose ISP workload offloading. The ISP agent, similar to FTL-based-ISP, transfers file information from the host application to the SSD before offloading the workloads. It builds a File-to-LBA mapping table named the F2LBA table on the SSD. This table converts file-level ISP kernel I/Os into a low-level storage address-based I/Os. The ISP agent manages and launches the ISP kernel in a separate subsystem called the ISP subsystem. The existing firmware, called the storage subsystem, performs FTL, GC, and NAND scheduling. Each subsystem has its own memory and core. When the ISP kernel requests I/Os, these are transferred to the storage subsystem by the ISP subsystem instead of directly accessing the NAND flash module. The storage subsystem handles ISP kernel I/Os in the same manner as the host I/Os but uses a separated and pre-reserved buffer to access the NAND flash data instead of the NAND buffer. The ISP buffer, a pre-reserved address-aligned buffer, enables the ISP subsystem to perform the rearrangement process rather than relying on the storage subsystem, which otherwise results in significant degradation in host I/O performance. Hence, the ISP agent can process ISP kernel I/Os effectively, even if the I/Os are of arbitrary sizes and are directed to any file location with unaligned memory as the source or destination. Furthermore, the ISP agent provides additional opportunities to optimize the ISP kernel by providing a non-blocking I/O capability and an aligned memory allocation interface.

4.1 Programming Model and Application Library

Figure 6(b) shows the programming model of the ISP agent. The primary objective of this model is to permit programmers to utilize pre-existing programs as ISP kernels without making significant code changes. The ISP agent accomplishes this by providing interfaces to the ISP kernel that match precisely with the host memory and I/O interfaces, such as malloc_ik, pread_ik, and pwrite_ik. The only difference between the host program and ISP kernel is that I/O requests require a predefined file number instead of a file descriptor.

The host program initiates file registration to be utilized by the ISP kernel by invoking the register_file() function within the library. The registration necessitates only parameters that are identical to those of the open() system call, excluding the user-defined file number. The library builds the file LBA information from the OS and constructs the F2LBA table based on it. Finally, the library transmits the table to the SSD through NVMe commands. After registering all files, the host application requests to launch the ISP kernel along with its arguments. Upon receipt of the request, the library initiates the ISP kernel through NVMe commands.

4.2 ISP Subsystem

The SSD parts of the ISP agent comprise two subsystems: the storage subsystem and the ISP subsystems. The storage subsystem manages standard I/O requests from the host by converting the NVMe command, accessing the data buffer, scheduling NAND flash access, issuing DMAs, and performing garbage collection. Meanwhile, the ISP subsystem manages the ISP kernel and its I/Os. When the storage system recognizes the ISP kernel’s NVMe command and corresponding arguments, it transmits them to the ISP subsystem. Subsequently, the ISP subsystem initiates the kernel.

Figure 7(a) illustrates the ISP subsystem and the ISP kernel. When an I/O request is triggered by the ISP kernel, the ISP subsystem follows a process similar to that of the host OS. It initially divides the I/O into logical blocks by byte-level offset and size. Subsequently, rearrangement information is produced for each partitioned I/O. This information specifies the target destination memory (for write) or source memory (for read), along with the start and end offsets of data within each logical block of the selected file. The ISP subsystem collects rearrangement information, file number, and block-unit file offset along with the I/O size to create a storage command. Afterward, this storage command is routed to the storage subsystem through the inter-core queue. In the storage subsystem, the command is reconstructed into StgCmd requests with LBA information using the F2LBA table. The StgCmd request is a SSD page-level request for accessing the NAND flash module, similar to the slice request described in Section 2.1. The storage system assigns the pre-reserved, address-aligned memory, known as the ISP buffer, to each StgCmd request to access the NAND flash module. The structure of the ISP buffer and the detailed process will be explained in Section 4.3. The storage subsystem updates the rearrangement information using the ISP buffer. Then, the ISP subsystem rearranges the target data between the assigned ISP buffer and the target memory before (for write) or after (for read) accessing the NAND flash module.

Fig. 7.

View Figure

4.3 Storage Subsystem

Figure 7(b) demonstrates how the storage subsystem converts storage commands to StgCmd requests, differing from the transformation of NVMe commands into slice requests. NVMe commands are designed for contiguous logical blocks; therefore, they carry logical block-level information such as start LBA and number of logical blocks (NLB). Thus, the storage subsystem can generate slice requests by simply carving NVMe commands using this information. The storage command’s file-level information may seem continuous initially, but it has a risk of fragmentation when translated into the logical block-level address. As such, a gathering process is required to collect the logical blocks that belong to the same SSD page based on their LBAs. The storage subsystem accomplishes this process with a hash table using LBAs as keys.

As discussed in Section 3.4, the ISP agent introduces the ISP buffer, a pre-reserved address-aligned memory that buffers NAND Flash module accesses requested by the ISP kernel. The ISP buffer is similar to the NAND buffer in that it comprises multiple entries corresponding to SSD pages, each entry containing valid and dirty bits, and the buffer is managed by an LRU list. The primary difference between the ISP buffer and the NAND buffer is that each entry in the ISP buffer has an additional lock flag for each entry. This particular flag indicates that the ISP buffer entry is currently in the process of being rearranged. The storage subsystem verifies whether a SSD page data targeted is unlocked in the ISP buffer before handling StgCmd and slice requests. If the data is locked, then the processing of the request is postponed until the rearrangement process has been completed. This guarantees preservation of data integrity by preventing conflicts or inconsistencies that may arise during the rearrangement.

Algorithm 2 demonstrates how the storage subsystem handles StgCmd requests. To begin, the subsystem accesses the NAND buffer based on the logical address of the targeted SSD page (line 1). If a dirty NAND buffer entry exists, then the subsystem writes the associated data back to the NAND flash (lines 3 and 4). If there is a clean NAND buffer entry and the StgCmd request is a write, then the storage subsystem invalidates the NAND buffer entry (lines 5–8). After lines 1–9 are executed, the current data of the SSD page is stored either in the NAND flash module or the ISP buffer. The storage subsystem handles StgCmd requests similar to the slice requests explained in Algorithm 1, differing only in the usage of the ISP buffer rather than the NAND buffer. The storage system initially accesses the ISP buffer utilizing the target SSD page’s logical address (line 10). If necessary, then it will evict the oldest entry in the ISP buffer (lines 11–17) and access the NAND flash module (lines 18–20). Note that StgCmd requests require the verification of byte-level start and end offsets in the rearrangement information to ensure complete SSD page coverage, unlike slice requests. After completing all operations, the storage subsystem updates the rearrangement information by utilizing the ISP buffer entry (line 26) and locks the ISP buffer entry (line 27) until the rearrangement is finished.

4.4 DMA-enabled Memory

Several applications require retrieving result values to the host instead of writing them to NAND flash. To facilitate this, the ISP agent introduces DMA-enabled memory (DEM), which is a specific memory region of a fixed size within the ISP subsystem that the ISP kernel can access. The host application can request DMA using DEM through the application library. Moreover, DEM also uses a mask to block DMA at the logical block level. The storage subsystem conducts a verification of the mask when the host initiates DMA with the DEM. DMA processes will not execute if the mask is set. The DEM mask can be configured by both the host application and ISP kernel to enable efficient ISP kernel outcome filtering.

4.5 ISP Kernel Optimization: I/O Overlapping and Direct I/O

Figure 8(a) demonstrates that the ISP subsystem remains inactive while storage commands are transmitted and processed. Storage command operations handled in the storage subsystem, such as LBA calculations and NAND flash access, are considered background tasks within the ISP subsystem. Therefore, if the ISP core performs additional computations or I/O operations during this idle period, then it has the potential to enhance the overall performance of the ISP core. Based on this concept, the ISP agent offers a non-blocking I/O interface, enabling the ISP kernel to perform other tasks simultaneously at the same time as processing background I/O. This optimization is referred to as I/O overlapping.

Fig. 8.

View Figure

As explained in Section 3.2.3, the ISP agent buffers access to NAND flash due to its memory address alignment constraints. If the ISP kernel requests I/O while meeting several conditions for accessing the NAND flash, then it can bypass memory buffering for NAND flash access. Specifically, the ISP kernel should request I/O with three conditions to bypass memory buffering: (1) the I/O is of the slice-unit size, (2) the request locates an appropriate file location, and (3) address-aligned ISP kernel memory is utilized. Figure 8(b) demonstrates direct NAND flash access for the ISP kernel I/O. This optimization can accelerate I/O response time by sidestepping rearrangements via memory buffering. We refer to this optimization as Direct I/O. The ISP agent provides memory allocation interfaces that correspond with kernels for this purpose.

Algorithm 3 describes how the storage subsystem handles StgCmd requests with direct I/O. The storage subsystem first assesses whether the StgCmd request matches the direct I/O criteria (line 1). If it matches, then the subsystem verifies whether the latest version of the requested data is present in either the NAND or ISP buffer (lines 2 and 11). Once the existence of the data is confirmed, the subsystem writes it back (lines 4 and 13) and invalidates the corresponding buffer entry (lines 5–9, 14–18). As a result, the newest version of the data is stored in the NAND flash module. The storage subsystem accesses the NAND flash module with the target memory directly without any buffering (lines 20 and 21). After all operations are finished, the storage subsystem updates the rearrangement information to prevent rearrangement in the ISP subsystem.

5.1 Environments

We synthesized a MicroBlaze [37] core on FPGA of Cosmos+ OpenSSD [14] to implement the ISP agent. We ran the storage subsystem on the Arm Cortex-A9 and the ISP subsystem on the MicroBlaze. While the total memory of the Cosmos+ OpenSSD Board is 4 GB, the ISP subsystem only used up to 48 MB memory. Specifically, we allocated 16 MB of memory for each of the ISP buffer, DEM, and ISP kernel. We utilized the Intel i7-4790K [10] as the host CPU with 16 GB of RAM.

5.2 Baselines

We conducted a comparison between the ISP agent, CPU execution, FIFO-ISP, and FTL-based-ISP without hardware support.

FIFO-ISP prevents the ISP kernel from requesting I/Os; instead, it allows the host application to indicate the storage data to be processed. In FIFO-ISP, the SSD loads specific data into the NAND buffer and subsequently executes the ISP kernel with it. This technique offers the advantage of expedited ISP kernel processing due to the absence of SSD-side LBA computation and data rearrangement. However, the amount of data that the ISP kernel can handle is limited by FIFO-ISP, which is confined to the size of the NAND buffer entry. Furthermore, FIFO-ISP limits the offloading capability for specific workloads that may necessitate access to arbitrary positions of the storage data.

FTL-based-ISP allows the ISP kernel to request I/Os, which is then handled by the pre-existing firmware. The firmware caches the storage data requested by the ISP kernel using the NAND buffer, similar to how it handles I/Os and slice requests from the host application. Once the data has been prepared in the NAND buffer, the firmware rearranges the storage data in the buffer and transfers it to the ISP kernel memory. FTL-based-ISP enables the ISP kernel to fully utilize one core for execution, while the ISP agent uses the core for both ISP kernel execution and data rearrangement. Nonetheless, delegating the rearrangement process to the firmware-executing core could interfere with the host’s I/O performance.

We implemented these two approaches, FIFO-ISP and FTL-based-ISP, to accommodate delivering results of the ISP kernel to host memory (storage-to-host) or different addresses in the storage (storage-to-storage). All ISP kernel computations are executed on MicroBlaze, with variations in approach depending on access to storage data.

The storage-to-host FIFO-ISP can be implemented by executing the ISP kernel and DMA based on their results. To achieve this in Algorithm 1, the ISP kernel must be executed before line 15, and DmaWithNandBufEntry() should be called based on its outcome. When implementing storage-to-storage FIFO-ISP, it may be necessary to discard the results of the ISP kernel depending on its output, and recover the input data. Algorithm 4 demonstrates the implementation of storage-to-storage FIFO-ISP. The SSD first verifies if a NAND buffer entry exists for the destination SSD page, removing it if it does (lines 1–7). Next, it prepares the NAND buffer entry for the source SSD page (lines 8–16), which is similar to how slice requests are processed, and performs a writeback to backup the latest data for the source SSD page (line 15). Finally, the SSD launches the ISP kernel with the loaded source data loaded in the NAND buffer entry (line 17). Depending on the result, the corresponding NAND buffer entry is either assigned to the destination SSD page or invalidated (lines 18–21).

To implement FTL-based-ISP, We utilized the ISP agent mechanism including F2LBA tables, StgCmd requests, and DEM. The ISP kernel can request I/Os through these mechanisms in FTL-based-ISP. The difference between FTL-based-ISP and ISP agent lies in how the storage subsystem handles StgCmd requests. Algorithm 5 demonstrates how the storage subsystem handles StgCmd requests in FTL-based-ISP. The storage subsystem uses NAND buffers to buffer data requested by StgCmd. Just as when processing a slice request, the storage subsystem accesses the NAND buffer (line 1), evicts the oldest buffer entry (lines 2–8), and loads the most current data into the NAND buffer entry (lines 9–11). However, in contrast to line 16 of Algorithm 1, the storage subsystem rearranges the data in the NAND buffer entries and transfers it to the target ISP kernel memory (line 16), rather than performing DMA.

5.3 Benchmarks

The evaluation of the ISP agent used three ISP kernels: NDP Aggregation, NDP Filtering, and DB Checkpointing. Tables 1 and 2 present our evaluation, listing the parameters used for each benchmark.

Table 1.

View Table

Table 1. NDP Benchmarks and Their Parameters

Table 2.

View Table

Table 2. MariaDB [21] and Sysbench [31] Configurations

NDP Aggregation and NDP Filtering are both data processing workloads that entail dividing file data into multiple units and performing iterative computations on each unit. NDP Aggregation summarizes integer values in each data unit by utilizing the size of the unit and the percentage of integers within it as input parameters. However, NDP Filtering examines specific data patterns within each data unit and, if detected, transfers the results to the host memory (storage-to-host) or records them to storage result files (storage-to-storage). For NDP Filtering, we utilized the data unit size and the ratio of the pattern present within the data unit as parameters. We evaluated the NDP Aggregation and NDP Filtering benchmarks in both storage-to-host and storage-to-storage scenarios using four methods: CPU Execution, FIFO-ISP, FTL-based-ISP, and ISP agent.

As described in Section 2.2, DB Checkpointing updates dirty DB pages according to the information in the delta files. We used two parameters, the size of the DB page and the number of delta entries (NDE) per SPD as parameters for DB Checkpointing. To demonstrate the impact and effectiveness of the ISP agent on host performance in a practical application, we concurrently measured the performance of the ISP kernels and the host I/O workload with Sysbench [31] and MariaDB [21]. The Sysbench was executed on MariaDB with ISP-assisted DB checkpointing, and OLTP Sysbench performance was measured as host performance. The processing speed of the checkpointing offloaded to SSD by MariaDB was measured as ISP kernel performance. We compared the performance of the ISP agent on both the host and ISP sides to the FTL-based-ISP. In addition, we analyzed the host-side performance by comparing it to the OLTP Sysbench performance on a vanilla DB without ISP.

6.1 ISP Kernel Performances

Figures 9 and 10 show the performance results for the storage-to-storage and storage-to-host benchmarks listed in Table 1. We measured the total time for all workloads that were offloaded to the SSD, which included setup procedures like the formation of the F2LBA table and DMA with DEM. Each bar denotes the efficiency of the ISP kernels in FIFO-ISP, FTL-based-ISP, and ISP agent, encompassing the improved performance of the ISP kernel via I/O overlapping and direct I/O in ISP agent. The labels I and D refer to I/O overlapping optimization and direct I/O optimization, respectively. All performance measurements are normalized to the CPU execution of each benchmark.

Fig. 9.

View Figure

Fig. 10.

View Figure

The figures demonstrate that CPU execution outperforms ISP offloading except for DB checkpointing. Additionally, as the application becomes more compute intensive, the performance of ISP offloading decreases. This is due to the fundamental performance difference of the i7-4790K for CPU execution and the MicroBlaze for ISP offloading. However, ISP offloading achieves greater performance as the intensity of I/O increases, especially in DB checkpointing applications with heavy I/O demands.

In most cases, FIFO-ISP demonstrates the most impressive performance among ISP techniques. As discussed earlier, FIFO-ISP directly computes data in the NAND buffer, avoiding the need to access the F2LBA table or rearrange data. Just as various entries in the NAND buffer can simultaneously access the NAND flash module or execute DMA for slice requests, separate FIFO-ISP kernels can run concurrently for different entries in the NAND buffer. This aspect of FIFO-ISP renders it well-suited for streaming data processing workloads such as NDP Aggregation and NDP Filtering. However, it cannot offload all types of workloads. In particular, FIFO-ISP cannot process data units larger than the 16 KB SSD page size, and it is unable to access any file position necessary for DB checkpointing. Additionally, Figure 9(b) demonstrates that FIFO-ISP was unable to offload storage-to-storage NDP filtering when using a 4 KB data unit size due to the NAND flash module’s inability to perform partial writes to the 16 KB SSD page.

The FTL-based-ISP and the ISP agent effectively offloaded all benchmarks. In general, the unoptimized ISP agent was slower than the FTL-based-ISP, because the FTL-based-ISP allows the cores to concentrate on executing the ISP kernel without data rearrangement. Nevertheless, the optimized ISP agent outperformed the FTL-based-ISP and was similar to the FIFO-ISP. This suggests that the ISP agent’s optimization techniques efficiently handle the structural trade-offs and enhance the effectiveness of ISP offloading.

6.2 I/O Overlapping and Direct I/O

As shown in Figures 9 and 10, the direct I/O optimization fairly improved performance, whereas the I/O overlapping optimization had minor effects. However, when these two optimizations were combined, a significant performance increase was observed. This is because the effectiveness of I/O overlapping relies on the proportion of computations, including data rearrangement, and I/Os. Therefore, if the ISP core is enhanced, then performance can be improved solely through I/O overlapping.

Figure 11(a) demonstrates the impact on I/O overlapping performance when the number of concurrently updated SPDs (NSU) changes. NSU indicates the maximum number of I/Os that can be overlapped simultaneously. For instance, when NSU is set to 16, the ISP kernel parses 16 SPDs and generates non-blocking reads simultaneously in parallel for each DB page indicated by the SPDs. The ISP kernel performs delta updates and writes to the DB pages separately after each non-blocking read for that page. Therefore, I/Os and delta updates from different SPDs can overlap. As shown in Figure 11(a), performance improves as more I/Os overlap, i.e., as NSU increases. This improvement is attributed to SSDs accessing multiple NAND flash modules concurrently through multiple flash channels, leading to an overall faster I/O processing speed as more I/Os are issued concurrently. Figure 11(b) demonstrates the effectiveness of DB checkpointing that uses direct I/O optimization with two different approaches. The first approach, D-Possible, applies direct I/O optimization selectively only when feasible. In detail, the D-possible approach does not perform direct I/O optimization on DB page read/write operations when using a 4 KB DB page size. However, the D-All approach optimizes all I/O, including DB page read/write, by performing them on the entire SSD page where the desired DB page is located. However, D-All requires more memory and leads to additional redundancy for DB page I/O operations. As shown in Figure 11(b), D-All exhibits superior performance compared to D-possible, even with I/O redundancy trade-offs.

Fig. 11.

View Figure

6.3 Effectiveness of ISP Offloading and Interference with Host Application

Figure 12 shows the performance of ISP offloading in real-world scenarios. As discussed earlier, we evaluated the effectiveness of ISP offloading and the impact of the ISP agent on host performance using the real-world application, Sysbench on MariaDB. In this experiment, MariaDB generates standard I/O requests for Sysbench transactions while offloading the DB checkpointing using ISP. Offloading the ISP kernel to the SSD could potentially affect the I/O performance of the host, subsequently impacting Sysbench performance.

Fig. 12.

View Figure

The Sysbench OLTP performance reflects the effectiveness of the host application by indicating the amount of Sysbench transactions processed per second. The processing speed of the offloaded deltas indicates the efficiency of the ISP kernel. Our experiments with MariaDB used three configurations: Vanilla DB (without ISP offloading), FTL-based-ISP offloading, and ISP agent offloading. We evaluated the performance of each configuration and normalized the delta processing speeds using the results obtained from FTL-based-ISP.

As shown by the black solid line in Figure 12, the Sysbench OLTP performance is greatly enhanced by ISP offloading, resulting in \(1.32\times\) and \(1.65\times\) improvements using FTL-based-ISP and ISP agent, respectively. This demonstrates the significant effectiveness of ISP-assisted DB checkpointing in improving overall DB performance. The ISP agent proves especially effective in harnessing ISP offloading, as it mitigates host I/O interference through data rearrangement separation, a functionality not achievable by FTL-based-ISP.

However, as shown by the red dashed line in Figure 12, the ISP agent was 0.894 times slower than FTL-based-ISP on the ISP side. This demonstrates that there is a trade-off between host I/O performance and ISP kernel performance for the ISP agent and FTL-based-ISP. Nevertheless, as depicted on the right side of Figure 12, the ISP agent’s optimization techniques significantly improved ISP kernel performance. The ISP agent increased the performance of the ISP kernel by 1.035\(\times\) through I/O overlapping, 5.044\(\times\) through direct I/O, and up to 15.998\(\times\) through the combination of both techniques, while maintaining a 1.65\(\times\) enhancement in host performance. These results demonstrate that the ISP agent’s optimizations have the potential to considerably enhance the performance of the ISP kernel.

In summary, optimizing the ISP agent allows for significant improvements in ISP kernel performance while maintaining the host performance advantage, despite the trade-off between host I/O performance and ISP kernel performance.

6.4 Insights from Experimental Results

The experimental results indicate the following three conclusions:

First, offloading background tasks to the ISP enhances overall application performance, particularly when managed by the ISP agent. Since the DB checkpointing was previously executed in the background, the ISP-assisted DB currently employs internal SSD resources to operate such background tasks. Figure 12 demonstrates that offloading DB checkpointing to the ISP improves overall DB performance. Among various ISP techniques, the ISP agent provides the best option for offloading background tasks to the ISP and maximizing overall application performance while minimizing interference with host I/O operations.

Second, it is important to evaluate the performance of ISP offloading and CPU execution prior to offloading general foreground tasks to the ISP. Generally, CPU execution was faster than ISP offloading with the exception of DB checkpointing. This is due to the difference in core performance between CPU execution and ISP offloading. The ISP kernel was executed on a 200MHz MicroBlaze for ISP offloading, while CPU execution was performed on a 4.40GHz Intel i7-4790K. Therefore, executing the ISP kernel on higher performance computing cores will likely result in a substantial performance increase.

Third, the ISP agent outperforms most ISP techniques in various situations and is the most efficient solution considering engineering costs. Figures 9 and 10 show that the FIFO-ISP outperforms other ISP technologies in terms of ISP-side performance. Meanwhile, the optimized ISP agent also achieves comparable or superior results. Optimizing ISP kernels using I/O overlapping or direct I/O techniques demands increased engineering costs. However, the ISP agent enables direct offloading of pre-existing host code to the ISP, eliminating the need to rewrite ISP kernels into the streaming programming model required for the FIFO-ISP. Therefore, when engineering costs are considered, the ISP agent proves to be the most cost effective ISP technology.