CUDA virtualization and remoting for gpgpu based acceleration offloading at the edge

In the last decade, GPGPU virtualization and remoting have been among the most important research topics in the field of computer science and engineering due to the rising of cloud computing technologies. Public, private, and hybrid infrastructures need such virtualization tools in order to multiplex and better organize the computing resources. With the advent of novel technologies and paradigms, such as edge computing, code offloading in mobile clouds, deep learning techniques, etc., the need for computing power, especially of specialized hardware such as GPUs, has skyrocketed. Although many GPGPU virtualization tools are available nowadays, in this paper we focus on improving GVirtuS, our solution for GPU virtualization. The contributions in this work focus on the CUDA plug-in, in order to provide updated performance enabling the next generation of GPGPU code offloading applications. Moreover, we present a new GVirtuS implementation characterized by a highly modular approach with a full multithread support. We evaluate and discuss the benchmarks of the new implementation comparing and contrasting the results with the pure CUDA and with the previous version of GVirtuS. The new GVirtuS yielded better results when compared with its previous implementation, closing the gap with the pure CUDA performance and trailblazing the path for the next future improvements.