Autonomous vehicles are starting to become a real possibility in some parts of industry. Agriculture, transportation and military are some of the examples. The day when we are going to see autonomous vehicles in everyday life for the regular consumer is quickly approaching. Many of the operations that vehicles have to perform are based on sensor information and some AI algorithms. Vehicles need to collect data, plan their trajectory and execute the trajectory. These tasks, especially the last two require non-traditional programming approaches and rely on machine learning techniques, which are part of AI.
There are many tasks for autonomous vehicles that are still presenting significant challenges and require sophisticated approaches. Replacing the cognitive and motor abilities of a human is not easy and will continue to be work in progress for years to come. There are different tasks that AI needs to solve so that we achieve reliable and safe autonomous drive.
The purpose of this two-part series is to illustrate the AI applications that make autonomous vehicles a reality, presenting their challenges and accomplishments. The nature of AI, compared to traditional software, is also explored and the specific challenges for developing, testing and deploying AI technologies in the autonomous vehicle arena is further discussed in Part 2.
1. Analysis of AI in the Autonomous Vehicle
The autonomous vehicle segment is the fastest growing segment in the automotive industry. Artificial Intelligence is indeed the most important and sophisticated component of self driving vehicles (Carmody, Thomas, 2019). A typical autonomous vehicle is shown on figure 1.
Figure 1: Self Driving Vehicle (Lentin, 2017)
The number of sensors with real time data and the need for intelligent processing of the data can be overwhelming. AI is used in the central unit as well as in the multiple electronic control units (ECU) of the modern automobile.
Since AI has been used in many areas, including robotics this is a natural fit of the technology which promises to master autonomy. The promise of the technology is that AI and perception technologies will provide a safer and more deterministic behavior which will lead to benefits such as fuel efficiency, comfort and convenience (Sagar and Nanjundeswaraswamy, 2019).
The challenges in developing AI systems for something as complex as a self driving vehicle are many. The AI has to interact with multitude of sensors and has to use data in real time. Many AI algorithms are computationally intensive and are therefore hard to use with CPUs that have memory and speed restrictions. Modern vehicles are an example of real time systems that have to produce deterministic results in the time domain. This is related to achieving safety while driving the vehicle. Complicated distributed systems like these require a lot of internal communications that are prone to latency which can disturb the decision making of the AI algorithms. In addition there is the issue of power consumption of the software running in the car. The more intensive AI algorithms consume more power, which is an issue, especially for electric vehicles that depend only on the charge of the battery (Carmody, Thomas, 2019).
AI is used for several important tasks in a self driving automobile. One of the main tasks is path planning. That is the navigation system of the vehicle (Sagar and Nanjundeswaraswamy, 2019). Another big task for AI is the interaction with the sensory system and the interpretation of the data coming out of sensors.
It is now apparent that providing a complete solution to replace a human behind the steering wheel is a formidable task (Boucherat, 2019). That is the reason why many manufacturers have started dividing the problem into smaller pieces and solving each one and thus taking smaller steps to achieve complete autonomy. There are always the startups, the disruptive companies in the industry, that try to solve to complete self driving problem and have vowed in the past that will have completely ready vehicles on the roads by 2020. Now it is clear that the reality is much more complex and many of the hurdles come from some of the issues that AI intrinsically has.
As AI develops and becomes better we will get closer and closer to having secure and autonomous transportation. Until then we have to cope with many hours of development and testing and adoption will be determined by how confident consumers are and will depend on market forces. It is all going to happen even if it takes longer than was originally anticipated. The demand and the need are there and the technology is almost there. Adoption may be faster or slower based on regulation and doing it in phases, starting from simpler and more deterministic use cases such as driving in a known environment. The algorithms used can be sufficiently relieved if the vehicle operates in only specific conditions that present fewer unknowns.
2. AI Applications in Vehicles
2.1. Sensor Data Processing
There are numerous sensors providing data for the central computer of the vehicle during operation. Sensors provide information for the road, other vehicles on the road as well as any other impediments that can be noticed similarly to how a human can perceive them (Khayyam et al., 2019). Some of these sensors can provide better perception than the average human, but for this to happen we need smart algorithms that can make sense of the data streams generated in real time.
One of the main tasks is to detect and identify objects ahead and around the vehicle (Sagar and Nanjundeswaraswamy, 2019). Artificial neural networks (ANN) are the algorithms typically used for this task. Another term for this field is deep learning, because a neural network contains many layers that contain many nodes. A deep neural network is shown on figure 2 although the number of nodes and layers in practice can be much higher.
Figure 2: Deep Neural Network Diagram. (Beachler, 2019)
The analysis of video input uses machine learning algorithms and most likely neural networks in order to classify objects. Since we have multiple sensors of different kind it makes sense to have dedicated hardware/software modules for each sensor. This approach allows for parallel processing of data and therefore faster decision making. Each sensor unit can utilize a different AI algorithm and then communicate its results to the other units or the central processing computer.
2.2. Path Planning
Path planning is important in order to optimize the trajectory of the vehicle and to lead to better traffic patterns. This can help reduce delays and avoid congestion on the road (Abduljabbar and Dia, 2019). Planning is a very suitable task for artificial intelligence algorithms. It is a dynamic task that can take into consideration a lot of factors and can solve an optimization problem while executing the path. The following definition of path planning is given: ”path planning for AVs enables self-driving vehicles to find the safest, most convenient, and most economically beneficial routes from point A to point B by using the previous driving experiences which help the AI agent make much more accurate decisions in the future” (Khayyam et al., 2019, p.56).
2.3. Path Execution
After the path is planned the vehicle is able to navigate the road conditions, by detecting objects, pedestrians, bicycles and traffic lights in order to reach the destination (Khayyam et al., 2019). The object detection algorithms are a primary focus of the AI community as they make it possible for human like behavior. The challenges come when different road and weather conditions come into play. Many accidents with testing vehicles happened because the simulation environment is different from the real world conditions and the AI software may react unpredictably when given unknown data.
2.4. Monitoring Vehicle’s Condition
The most promising type of maintenance is predictive maintenance. It can be defined in the following way: “Predictive maintenance employs monitoring and prediction modelling to determine the condition of the machine and to predict what is likely to fail and when it is going to happen” (Prytz, 2014, p.2). It tries to predict future problems, not problems that already exist. In this respect predictive maintenance can save a lot of time and money. Both supervised and unsupervised learning can be used for predictive maintenance. The algorithms can use on-board and off-board data to make decision for predictive maintenance. The machine learning algorithms used for this task are classification algorithms like logistic regression, support vector machines and random forest algorithm.
2.5. Insurance Data Collection
Data logs from a vehicle can contain information about the driver’s behavior and this can be used in the analysis of traffic accidents. These data can be used for processing of claims (Carol, 2017). All this can contribute to decrease in prices for insurance, since the safety is more deterministic and guaranteed. For fully automated cars the liability will shift from the passenger, who is no longer a driver, to the manufacturer. In the semi-autonomous vehicle, we will still most likely have some liability of the driver. Proving these type of cases will rely more and more on smart data captured by the AI system of the vehicle. Data from all sensors generates enormous amounts of information. Saving all the data at every moment may not be practical, but saving snapshots of relevant data seems the right balance of obtaining evidence that could be used for post analysis of a certain traffic event. This approach is similar to how black box information is stored and analyzed after a crash.
The next article in this two-part series discusses artificial intelligence algorithms and challenges for autonomous vehicles.
 Anton Hristozov graduated from the University of Pittsburgh with a Masters in Telecommunications and Information Science. He is currently pursuing a doctoral degree in Technology at Purdue University. His research interests involve embedded systems, Internet of Things, smart sensors and real time systems. He is a Linux enthusiast and enjoys working with cyber physical systems which use sensors and convert energy in different forms. Smart energy and the use of artificial intelligence is a new and interesting direction, given the recent advances in use of machine learning in cyber physical systems. |
