Enabling Multimodal Sensing, Real-time Onboard Detection and Adaptive Control for Fully Autonomous UAVs

Home

The goal of this NSF CPS research project is to achieve true onboard autonomy in real time for UAVs in the absence of remote control and external navigation aids. Very low power and light weight machine intelligence techniques will be investigated to achieve multi-modal sensing, onboard detection, and adaptive control. Detection, optimization and control problems in an autonomous UAV will be formulated and solved using deep neural networks (DNN) and deep reinforcement learning (DRL). Ultra-low power and high-performance DNNs using the circulant weight matrix and FFT/IFFT operations will be trained and implemented on either GPUs or FPGAs. This unique technique has the potential to reduce computational complexity as well as the storage complexity of the DNNs, and hence enable us to close the loop of sensing, detection, and control in real-time.

Personnel

Faculty

• Dr. Qinru Qiu (Department of EECS, Syracuse University)
• Dr. Amit Sanyal (Department of Mechanical and Aerospace Engineering, Syracuse University)
• Dr. Jian Tang (Department of EECS, Syracuse University)
• Dr. Senem Velipasalar (Department of EECS, Syracuse University)
• Dr. Yanzhi Wang (Department of ECE, Northeastern University)

Students

• Hossein Eslamiat
• Mani Dhullipalla
• Caiwen Ding
• Burak Kakillioglu
• Yilan Li
• Yantao Lu
• Ao Ren

Technology

Publications

DNN Acceleration

1. S. Liao, Z. Li, L. Zhao, Q. Qiu, Y. Wang and B. Yuan, "CircConv: a structured convolution with low complexity," in Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI), 2019.
2. C. Ding, S. Wang, N. Liu, K. Xu, Y. Wang and Y. Liang, "REQ-YOLO: A resource-aware, efficient quantization framework for object detection on FPGAs," in ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA), 2019.
3. A. Ren, T. Zhang, S. Ye, W. Xu, X. Qian, X. Lin and Y. Wang, "ADMM-NN: an algorithm-hardware co-design framework of DNNs using alternating direction methods of multipliers," in Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2019.
4. Z. Li, J. Li, A. Ren, et al., "HEIF: Highly efficient stochastic computing based inference framework for deep neural networks," in IEEE Trans. on Computer-Aided Design (TCAD), 2018.
5. Y. Wang, C. Ding, Z. Li, et al., "Towards ultra-high performance and energy efficiency of deep learning systems: an algorithm-hardware co-optimization framework," in Proc. of AAAI Conference (AAAI), 2018.
6. X. Ma, Y. Zhang, G. Yuan, et al., "An area and energy efficient design of domain-wall memory-based deep convolutional neural networks using stochastic computing," in Proc. of International Symposium on Quality Electronic Design (ISQED), 2018. (Best Paper Nomination)
7. S. Wang, Z. Li, C. Ding, et al., "C-LSTM: Enabling efficient LSTM using structured compression techniques on FPGAs," in Proc. of FPGA, 2018.
8. S. Lin, N. Liu, M. Nazemi, et al., "FFT-based deep learning deployment in embedded systems," in Proc. of Design, Automation and Test in Europe (DATE), 2018.
9. T. Zhang, S. Ye, Y. Zhang, Y. Wang, and M. Fardad, "Systematic weight pruning of DNNs using Alternating Direction Method of Multipliers", in Proc. of International Conference on Learning Representations (ICLR) (Short Paper), 2018.
10. C. Ding, A. Ren, G. Yuan, et al., "Structured weight matrices-based hardware accelerators in deep neural networks: FPGAs and ASICs," in Proc. of Great Lakes Symposium on VLSI (GLS-VLSI), 2018.

Sensing & Detection

1. A. Ahmad, B.Kakillioglu and S. Velipasalar, "3D Capsule Networks for Object Classification from 3D Model Data," to appear in the Proc. of Asilomar Conference on Signals, Systems, and Computers, 2018.
2. B.Kakillioglu, A. Ahmad, and S. Velipasalar, "Object Classification from 3D Volumetric Data with 3D Capsule Networks," to appear in the Proc. of the IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2018.

Flight Control

1. Y. Li, H. Eslamiat, N. Wang, Z. Zhao, A. K. Sanyal and Q. Qiu, "Autonomous Waypoint Planning and Trajectory Generation for Multi-rotor UAVs," in 1st Workshop in Design Automation for CPS and IoT (DESTION), 2019
2. H. Eslamiat, Y. Li, N. Wang, A. K. Sanyal and Q. Qiu, "Autonomous Waypoint Planning, Optimal Trajectory Generation and Nonlinear Tracking Control for Multi-rotor UAVs," in European Control Conference, 2019
3. M. H. Dhullipalla, R. Hamrah, A. Sanyal, "Trajectory Generation on SE(3) with Applications to a Class of Underactuated Vehicles," CDC 2017
4. R. Hamrah, R. Warier, A. Sanyal, "Discrete-time Stable Tracking Control of Underactuated Rigid Body Systems on SE(3)," accepted for IEEE Conference on Decision and Control, 2018.

Code

Coming Soon.

Outreach and Broader Impact Outcomes

Education Activities

• An undergraduate level deep learning course was developed and offered by Dr. Qiu. It introduces basic machine learning methods and implementation skills. [Syllabus]
• A new course was developed for senior graduate students by Dr. Wang with a focus on the computational aspects of deep learning. [Syllabus]
• Two courses, MAE 700 and MAE 725, were taught by Dr. Sanyal to introduce nonlinear control of mechanical systems. [Syllabus - MAE 700, Syllabus - MAE 725]
• Dr. Velipasalar taught courses in image processing where recent machine learning approaches were introduced. [Syllabus]

Outreach Activities

• Dr. Qiu is co-organizing a First Lego League team, the RoboSavages, for students in 6th~8th grade. In a 2018 regional competition, the team received the Robot Design award. The team won the regional championship in 2019 FLL competition.
• Presenting at EE Day