Since the fall of 2017, my friend Michael and I have been developing a self-driving golf cart. We had the ambition of using the knowledge that we learned from the Udacity self-driving nano-degree to make something tangible and meaningful.

Phase 1

During phase one, we used deep learning techniques, such as behavioral cloning, to train neural networks that allow the vehicle to navigate safely. Using the deep learning models, the cart can follow certain roads autonomously, while avoiding obstacles. Here is the list of modules/components that we have been working on. Under each of these topics, you will find specific blog posts about the development process.

Michael and I testing the vehicle in the cold winter.

Phase 2

Michael graduated when phase 1 concluded. After spending a year researching, testing the deep learning models and building the first version of the vehicle hardware, I discovered strengths and weaknesses in the software and hardware. In 2018, I started phase two of the project.

The self-driving car’s hardware setup.

This new phase includes building an all new steering control system, upgrading the on-board computer system, and switching to a stereoscopic vision system. Instead of solely relying on machine learning, I migrated the vehicle software to ROS, and created a ROS navigation system. This new autonomous navigation software is more reliable than deep learning, however, it lacks the ability to generalization in multiple environments that deep learning offers.

So far, this project has been such a wonderful journey. Thanks to my teachers and mentors and friends, I have made decent progress. All the code for the self-driving golf cart is open source on Github. Feel free to check it out! Any contribution from the open source community is very welcomed! Thanks for checking out my self-driving vehicle project.

  1. Deep Learning
    1. Visualizing the Steering Model with Attention Maps
    2. Successfully Tested the Autonomous Steering System for the Self-Driving Golf Cart
    3. Predicting Steering Angle with Deep Learning — Part 2
    4. Predicting Steering Angle with Deep Learning — Part 1
    5. On the Road — Testing the Autonomous Golf Cart
    6. Testing the Steering System with NVIDIA Jetson
    7. Implementing YOLO v3 Object Detection on the Autonomous Vehicle
    8. Testing the All-New Autonomous Steering System – (Part 1)
  2. Semantic Segmentation
    1. The Robustness of the Semantic Segmentation Network
    2. Autonomous Cruise Control System
    3. Understanding the World Through Semantic Segmentation
  3. Driver by Wire System (DBW)
    1. Hacking the Accelerator — Controlling the Speed Using an Arduino
    2. Steering Motor Control System (SMCS)
    3. The Steering Hardware Works!
    4. Engineering the Steering Mechanism
    5. Autonomous Golf Cart Power Steering Using a Linear Actuator (Part 1)
    6. Autonomous Golf Cart Power Steering Using a Linear Actuator (Part 2)
    7. Electronic Braking System – Self-Driving Golf Cart

More blog posts coming soon on the following topics:

  • Object detection
  • Traffic light detection
  • Lane Detection
  • Path planning
  • Localization & maps

After 15 months of development, we still have a long way to go. Along the way, I have learned so much about deep learning, robotics, engineering and much more… I will continue to post the latest development on my blog. If you have any questions, comments or concerns, please contact me at