ZO-DARTS++: An Efficient and Size-Variable Neural Architecture Search Algorithm
Speaker: Lunchen Xie
Tongji University, Shanghai, China
DEIB - Alpha Room (Bld. 24)
September 2nd, 2024 | 2.00 pm
Contact: Prof. Danilo Ardagna
Research Line: Advanced software architectures and methodologies
Tongji University, Shanghai, China
DEIB - Alpha Room (Bld. 24)
September 2nd, 2024 | 2.00 pm
Contact: Prof. Danilo Ardagna
Research Line: Advanced software architectures and methodologies
Sommario
On September 2nd, 2024 at 2.00 pm the seminar "ZO-DARTS++: An Efficient and Size-Variable Neural Architecture Search Algorithm" will take place at DEIB Alpha Room (Building 24).
The deployment of deep learning models has become very popular in real life. Under the condition of limited hardware resources, the flexible design of a deep learning model that can adapt to the computing power of different platforms draws attention from academia and industry. In this report, we will introduce a new automatic architecture search algorithm ZO-DARTS++. First, by rationally modeling the model architecture design problem, it is transformed into a differentiable bi-level optimization problem. Second, the algorithm uses the zeroth-order approximation method to quickly solve the optimization problem, which has a significant improvement in efficiency, better accuracy and stability compared with traditional methods. Finally, the algorithm designs the model operation with a size-variable search scheme while considering the hardware resource constraints, taking into account both speed and search efficiency. Experiments on a series of public datasets proves that ZO-DARTS++ can automatically and efficiently design deep learning models that adapt to different platforms.
Lunchen Xie received the B.E. degree in software engineering from Xidian University, Xi’an, China, in 2019. He is currently pursuing the Ph.D. degree with the School of Software Engineering, Tongji University, Shanghai, China. He is now a visiting Ph.D. student with the Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milano, Italy, sponsored by Chinese Scholarship Council (CSC). His research interests mainly include federated learning, distributed machine learning, and neural architecture search.
The deployment of deep learning models has become very popular in real life. Under the condition of limited hardware resources, the flexible design of a deep learning model that can adapt to the computing power of different platforms draws attention from academia and industry. In this report, we will introduce a new automatic architecture search algorithm ZO-DARTS++. First, by rationally modeling the model architecture design problem, it is transformed into a differentiable bi-level optimization problem. Second, the algorithm uses the zeroth-order approximation method to quickly solve the optimization problem, which has a significant improvement in efficiency, better accuracy and stability compared with traditional methods. Finally, the algorithm designs the model operation with a size-variable search scheme while considering the hardware resource constraints, taking into account both speed and search efficiency. Experiments on a series of public datasets proves that ZO-DARTS++ can automatically and efficiently design deep learning models that adapt to different platforms.
Lunchen Xie received the B.E. degree in software engineering from Xidian University, Xi’an, China, in 2019. He is currently pursuing the Ph.D. degree with the School of Software Engineering, Tongji University, Shanghai, China. He is now a visiting Ph.D. student with the Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Milano, Italy, sponsored by Chinese Scholarship Council (CSC). His research interests mainly include federated learning, distributed machine learning, and neural architecture search.
