Go up one level Go home

1 How does an AV Work?

The purpose of an AV is to travel from one location to another with little or no input from a human driver. Fundamentally, the AV is asking itself three questions:

  • Where do I need to go?
  • Where am I now?
  • How do I travel safely and make reasonable progress from where I am to where I need to go?

Addressing the first question is relatively trivial. The destination or waypoints might be inputted manually by the passenger of the vehicle, could be sent to the vehicle from a control centre or could be pre-programmed into the vehicle's map.

Once the destination is received, the next step is route planning. Again, this could be inputted or confirmed by a human passenger or control centre or could be pre-programmed, or the vehicle could decide the best route based on routing algorithms.

2 Sensing

In order for the vehicle to localise itself, it needs to be able to sense its environment. It can do this using a combination of externally facing sensors and communication, which may include:

  • Light Detection And Ranging (LIDAR) – pulses of light are fired from the device and the time taken for the pulse to bounce back from a surface is measured.
  • Cameras – light streams through a lens and is converted into pixels as the image is recreated electronically.
  • Infraredcameras/sensors–similar to cameras but using the infrared part of the electromagnetic spectrum.
  • Radio Detection And Ranging (RADAR) – similar to LIDAR, and using longer frequency radio waves, which corresponds to a different part of the electromagnetic spectrum.
  • Ultrasonic sensors – similar to LIDAR and RADAR but using sound waves.
  • Global Navigation Satellite Systems (GNSS) receiver – radio signals transmitted from satellites and received by an aerial on the device. The signal is converted to location coordinates. GNSS includes GPS, GLONASS, Galileo, Beidou and other regional systems
  • Dedicated Short-Range Communication (DSRC) receiver – a wireless communication technology designed to allow communications between vehicles and between vehicles and infrastructure. DSRC technology operates on the 5.9 GHz band of the radio frequency spectrum.
  • Cellular communications receiver – information is transmitted via radio frequency from cellular towers to the vehicle in the same way as it is for mobile telephones.
  • Microphones – captures sound waves electronically and may be useful for detecting emergency vehicle sirens, for example.

In addition to external sources of information, the vehicle can use internal sensors, such as accelerators and gyroscopes to estimate vehicle position, orientation and speed (known as Inertial Navigation System or INS), and odometry sensors, which measure wheel speed to estimate how far a vehicle travels.

Since each source of external and internal information has its own advantages and disadvantages, it is necessary to combine the information by fusing it. This is known as sensor fusion.

An AV may localise by comparing information from its sensors to the detailed map it has stored in its database.

3 Obstacle Avoidance

The examples of commercially used HAVs, such as the Heathrow pods or the Rivium Business Park Shuttle vehicles follow a fixed path. Obstacle avoidance might be limited to stopping when an object is detected in front of the vehicle. This type of system, which could be described as a 'virtual tram' or 'horizontal elevator' is less complex and may have many applications especially where the road infrastructure can be carefully controlled. But for negotiating complex mixed traffic environments on public roads the vehicle will need to be capable of avoiding obstacles through a combination of change of speed and steering. This will require not just detection of where other road users and objects are at the present moment, but prediction of where they need to be in the near future. As discussed in this talk by Chris Urmson, when he was working on the Google Self-Driving car, this could be described as the main difference between a driver assistance system and a fully self-driving system.

4 Machine learning

It may be advantageous, or even essential, for AVs to be able to recognise and classify objects, which can help to predict the future trajectory of that object and also to assess the importance of avoiding a potential collision with it. As a result of thousands of years of evolution, humans are good at pattern recognition and detecting the difference between a child and a plastic bag, however, computers need to learn the difference to avoid applying the brakes or taking other evasive action unnecessarily. Deep learning techniques may be able to assist with this aspect. However, as discussed in this IEEE Spectrum article there is a risk that with over-reliance on machine learning decisions are made by a 'black box' – something humans may not be able to understand.

5 Connectivity

(See also Digital Mapping and Connectivity)

Connected Vehicles (CVs), also known as Cooperative Intelligent Transport Systems (C-ITS) refer to vehicles with increasing levels of connectivity that allow them to communicate with their surrounding environment (including the infrastructure and other vehicles). This could provide information to the driver about road, traffic, and weather conditions, and on routing options and enable a wide range of connectivity services. As stated in the European strategy on Cooperative Intelligent Transport Systems:

"In many respects today's vehicles are already connected devices. However, in the very near future they will also interact directly with each other and with the road infrastructure. This interaction is the domain of Cooperative Intelligent Transport Systems (C-ITS), which will allow road users and traffic managers to share and use information previously not available and to coordinate their actions. This cooperative element is expected to significantly improve road safety, traffic efficiency and comfort of driving, by helping the driver to take the right decisions and adapt to the traffic situation."

The report continues:

"Currently, the most promising hybrid communication mix is a combination of ETSI ITS-G5 and existing cellular networks. This ensures the best possible support for deployment of all Day 1 C-ITS services. It combines low latency of ETSI ITS-G5 for time-critical safety-related C-ITS messages with wide geographical coverage and access to large user groups of existing cellular networks."

Connectivity may not be a requirement of automation, but the two technologies are complementary. As outlined above, a CV can enable a vehicle to see beyond its sensors. It is possible that automation will require certain CV technologies in some situations to improve reliability.

6 Operational Design Domain

As described in the guidance document published by the US Department of Transportation, the Operational Design Domain (ODD) describes the specific conditions under which a given automated driving system or feature is intended to function. The ODD is the definition of where (such as what roadway types and speeds) and when (under what conditions, such as whether it is day or night and weather limits) an AV is designed to operate.

7 Human Machine Interface

The human-machine interface of an AV describes how a human interacts with the vehicle. This might include mechanisms for how the human instructs the vehicle where to go, confirms to the occupant when the vehicle is driving or requests that the human takes over the driving task. Providing an indication of what the vehicle is sensing can provide comfort to the occupant.

It is also important for the vehicle to communicate to external road users, for example, other human drivers, pedestrians and cyclists. This might include providing an indication of when the vehicle is intending to yield to other vehicles or allow a pedestrian to cross. The severity of issues arising from miscommunication can be severe, so this is an area where global standards need to be developed.

8 Advanced Driver Assistance Systems

Advanced Driver Assistance Systems (ADAS) are often considered building blocks to fully automated driving. These include features that have been available for many years, including Adaptive Cruise Control, which controls the vehicle speed to maintain the distance to the vehicle in front, and Lane Keep Assist, which helps keep the vehicle in its lane. A useful list of ADAS enabled vehicles, described in the article as 'almost self-driving', can be viewed here.

9 Challenging Driving Scenarios

Part of the challenge relates to the fact that AVs must be able to negotiate such a wide range of complex driving scenarios. The following table provides an indication of the type of events and scenarios that might need to be considered in preparing a vehicle for automated driving. This table, which is by no means exhaustive, and was taken from the related project by the TSC, titled ‘Taxonomy of Scenarios for Automated Driving'.

Abnormal / Challenging Driving Event
Issues involved
Parked vehicles
How to ensure they are parked and have not momentarily stopped. How to allow for possibility of doors opening?
Disabled (broken down / crashed) vehicles
Passing may lead to compromising other rules of road such as crossing solid white lines
How much space to leave? Different clearances for different types? Pedestrian behaviour can be unpredictable. Should vehicle slow down when passing pedestrians on footway?
Passing cyclists
How much space to leave? Different clearances for different types? Cyclist behaviour can be unpredictable.
Road flooding
Difficult to sense the depth? Could lead to loss of control of vehicle or splashing of pedestrians.
Animals in road (either shepherded or loose)
For smaller animals, it can be difficult to decide whether to pass over animal or attempt to stop or swerve.
Ridden horses
Determining appropriate speed and overtaking strategy.
Negative obstructions such as potholes or road/bridge collapse
Could be difficult to sense.
Load shedding from other vehicles
Action could depend on density/mass of objects being shed, but might not be possible for machine to determine.
Vehicles in process of becoming disabled, e.g. tyre blow out, lorry jack knifing, tall vehicle overturning in wind etc.
Challenging for machine to detect and interpret subtle clues that provide indications.
Traffic calming measures
Speed humps, chicanes, etc. need to be detected and negotiated appropriately.
Fallen power cables/branches in carriageway
Could be challenging to detect with sensors.
Level crossing
Similar to traffic signal but consequences of stopping on rail line could be catastrophic.
Challenging to detect oncoming vehicles
Lane reallocation / rerouting
Temporary lane closure on highway
Road layout may differ from map being referred to by vehicle
Temporary contraflow
Automated driving feature may be designed for highways and not for two-way traffic operation
Lane designations (e.g. bus lanes, high occupancy vehicle lane, hard shoulders)
May need to clarify under what circumstances vehicles can enter.
Adverse weather / environmental conditions
High winds
Loss of control
Snow either falling or on carriageway
Sensor visibility compromised, road markings and kerb lines obscured, loss of vehicle control
Heavy rain
Sensor visibility compromised, road markings obscured, loss of vehicle control
Loss of control
Fog / bright sunshine etc.
Sensor visibility compromised
Road Etiquette
Emergency vehicle in vicinity
How to avoid impeding whilst obeying rules of road
Crossing white lines
Under what circumstances can vehicle do this?
Interpreting gestures from other road users
Challenging to detect and interpret the meaning of hand gestures, flashing of headlights, etc.
Traffic flow arbitration
Police or authorised persons intend to stop AV
How does AV recognise what is an authorised person, and then interpret commands?
Two lanes merge into one
Often involves interaction between human drivers
Merging onto highway
Often involves interaction between human drivers
T junction
Poor lateral field of view from AV
Poor lateral field of view, right turn stale-mate
Temporary speed limits
How to ensure location is communicated to AVs
Temporary traffic signals
How to ensure location is communicated to AVs
Temporary stop-go sign
Can AV interpret?
Giving way to oncoming vehicles on narrow section of road
Often required communication between human drivers to decide who proceeds first
Detecting correct lane allocations, ‘give way to right’ standoff
Zebra crossings
Giving way to waiting pedestrians
Traffic signal failure
Junction reverts to priority based or interaction between human drivers
Examples of Challenging Driving Scenarios for AVs

The reasons for automated driving disengagements cited by Waymo could be indicative of the types of issues that HAVs will always face. The software can be continuously tested and improved, it is uncertain at what point regulators and indeed society decides that it's ready for commercial deployment.

Even if HAVs are considered safe enough, they may struggle to make sufficient progress in some situations. Consider the example of an HAV approaching a give-way line of a busy junction. There may be very few gaps in the main road traffic considered large enough by the automated control system for the vehicle to pull out into. Humans will often apply common sense in this situation and will nudge into the traffic in the hope that another human will let them out. There may be some communication between the two drivers involved, such as a flash of the headlights or a hand gesture. HAVs may simply continue to wait on the minor road for long periods of time, which would be frustrating not just for the occupant, but also for other vehicles stuck behind the HAV.