Aug 30, 2019Two College of Charleston data science students, under the guidance of technology company Logicalis, have completed what they call a competitive learning experience that employs reinforcement learning (RL) computing at the edge to help an autonomous vehicle learn its way around a track.
The team's Internet of Things (IoT)-based system employs Wi-Fi-based connectivity so that software, using Amazon Web Services' (AWS) DeepRacer cloud-based console, can communicate with the vehicle, capturing camera imagery to better understand the vehicle's path and location on roadways, as well as sending instructions to improve performance, such as acceleration and improved path selection. The competitive learning experience they developed enables their own system to out-perform the AWS-based model, the team reports. The student team's system thereby enables a vehicle or other device to learn its environment and respond, based on the data collected and interpreted, at the edge (using a tablet or laptop).
AWS held a DeepRacer inventing competition earlier this year at its AWS re:Invent conference, to challenge developers and data scientists to build autonomous vehicles that employ machine learning. Users can join the AWS DeepRacer League to compete for prizes and a chance to advance to the championship cup.
The DeepRacer is a one-eighteenth scale race car, explains Matthew Funderburg, Logicalis's services architect for IoT analytics, but it comes with the capacity to interact with its own surroundings and accomplish machine learning. Amazon calls it an integrated learning system for users to learn and explore reinforcement learning, and to experiment and build autonomous driving applications.
At Logicalis, Funderburg provides the technology company with solution development, building use cases with customers around the IoT. Initially, Mike Trojecki, Logicalis's IoT and analytics VP, asked Funderburg to mentor the two students, Eliza Starr and Joshua Turner. "I knew this project could be cool for them—and for me as well," he recalls.
Since Logicalis was launched in 2018, it has developed and deployed IoT solutions for health-care, manufacturing, government and education applications. As the company grows, Funderburg says, "We've been working on understanding what customer needs are and building out partnerships with some existing customers."
For the AWS DeepRacer program, Starr and Turner were focused on developing and fine-tuning the algorithm for the autonomous car so that it could use sensor data to detect where a vehicle is located on a track, and to use that data, along with machine learning—sharing that information and receiving prompts from the cloud-based software wirelessly—to train itself to stay on the track at top speed. Turner and Starr are both data sciences majors.
Once the students and the technology company teamed up, Funderburg says, they began exploring. "We wanted to take an end-to-end approach," he states, "and we looked at existing technologies and started filtering out use cases that inspired them." The students' objective, Starr explains, "was to apply the data science process—brainstorming a project, obtaining data, understanding and cleaning data, solving the problem and interpreting the results—in a real-world setting."
The team discovered that they wanted a system with which they could leverage computing at the edge. "That's a big area of focus, machine learning and inference on the edge, without going through the long pipeline," Funderburg says. The group considered several options before building the IoT-based solution that it then employed first on a virtual vehicle.
The group acquired the DeepRacer vehicle and the track beta simulation, then began developing and testing the RL algorithm. The students utilized the simulation of the vehicle, which they named Virtual Car 42 (the number 42 represents the answer to the "ultimate question of life, the universe and everything" in Douglas Adams' novel The Hitchhiker's Guide to the Galaxy and its sequels and adaptations).
They then began putting the simulated vehicle through testing, and ultimately found that the learning algorithms they had developed allowed their virtual vehicle to outperform the AWS DeepRacer, based on the number of laps properly completed. The algorithm identified curves in the road, for instance, and adjusted the vehicle's steering and acceleration to navigate each curve at the highest possible speed.
By July 2019, the group was using the software with the physical car. The vehicle comes with an onboard camera and accelerometer gyroscope that helps them identify the vehicle's location. The students used the AWS Sage Maker machine-learning platform. The AWS Robotmaker robot simulation platform transmits code to the vehicle via Wi-Fi, but this can also be accomplished using a USB stick. The team was then able to control the data flowing to and from the vehicle with an iPad or computer.
The machine-learning algorithms applied to Virtual Car 42 could be provided to other physical devices, such as equipment or vehicles used in commercial or industrial applications. For instance, with IoT connectivity using Wi-Fi or another wireless network, devices could utilize sensors to capture and provide information such as predictive maintenance, image and sound recognition, and surveillance, while also enabling autonomous driving.
The next step could involve marketing, Funderburg says, noting that there are a wide variety of applications for the solution the team has developed. Computing at the edge can serve many IoT deployments, for example, including those in asset tracking, geofencing and other use cases for which sending large amounts of data to a server may pose a security or bandwidth concern.
