Deep Neural Networks (DNNs) have proven to show superior performance compared to conventional algorithms in the field of Computer Vision (CV) and Natural Language Processing (NLP). However, due to its computational complexity, high energy consumption and memory demand, both academia and industry find it difficult to port DNN-based tasks on traditional CPUs and GPUs. To tackle this problem, software-based model compression methods were proposed to increase energy and memory efficiency of DNNs. Also, engineers developed domain-specific hardware to efficiently accelerate the heavy DNN computation. 

Our lab also studies and develops domain-specific hardware accelerators and model compression techniques for efficient deployment of DNNs. We target acceleration and compression of both traditional Convolution-based Neural Networks (CNNs) and attention-based transformers. Our current research is the optimization of DNN training, where we aim to minimize memory footprint and training latency for effective deployment of DNNs for on-device training (edge computing). We also study model compression techniques for attention-based transformers, such as Vision Transformers (ViT) and Large-Language Models (LLMs).