Dynamic Machine Learning Inference at Mobile Edge

ABSTRACT

This talk highlights the growing integration of AI into mobile and IoT devices, which has the potential to significantly improve user experiences. However, deploying AI algorithms on these edge devices comes with its own set of challenges. The speaker will delve into three dynamic machine learning inference solutions tailored for the mobile edge. First, the talk covers a dynamic inference mechanism that optimizes ML execution on mobile devices. This mechanism efficiently reduces the consumption of computing resources on mobile devices with a slight accuracy reduction by exploiting the spatial and depth redundancy of the input data. Second, the speaker presents an inference distribution method that allows multiple resource-constrained mobile edge devices to collaborate on running a large ML model. This method smartly splits the model and distributes computation among heterogeneous devices, considering varying computing resources and network connectivity. Third, a dynamic inference offloading approach is presented, optimizing the coordination of computation offloading across diverse edge devices. This approach introduces distributed control schemes and asynchronous global coordination to enable flexible and instant offloading control and model selection per user per request. In conclusion, the speaker provides insights into edge AI research, addressing the challenges of resource constraints, connectivity, and real-time performance.

BIOGRAPHY

Tao Han (M’15-SM’20) is an Associate Professor in the Department of Electrical and Computer Engineering at New Jersey Institute of Technology (NJIT) and an IEEE Senior Member. Before joining NJIT, Dr. Han was an Assistant Professor in the Department of Electrical and Computer Engineering at the University of North Carolina at Charlotte. Dr. Han received his Ph.D. in Electrical Engineering from NJIT in 2015 and is the recipient of NSF CAREER Award 2021, Newark College of Engineering Outstanding Dissertation Award 2016, NJIT Hashimoto Prize 2015, and New Jersey Inventors Hall of Fame Graduate Student Award 2014. His papers win IEEE International Conference on Communications (ICC) Best Paper Award 2019 and IEEE Communications Society’s Transmission, Access, and Optical Systems (TAOS) Best Paper Award 2019. His research interest includes mobile edge computing, machine learning, mobile X reality, 5G system, Internet of Things, and smart grid.