LiDAR Deer Detection System
An in-vehicle LiDAR deer detection system for drivers of autonomous vehicles in North American regions with large deer populations and collision rates.
Overview
PROJECT BRIEF:
A 10-week graduate school project consisting of research and interface design. This project culminated in a design specification that illustrates the experience and how it could be implemented.
TIMELINE
January – March 2021
10 Weeks
TOOLS
Figma
SKILLS
Literature Review
Technical Research
Ideation
Storyboarding
Interface Specification
WHY I’M SHARING THIS PROJECT:
I really enjoyed researching, storyboarding, and writing a user specification for this automotive interface. It was a solo effort and passion project that tested my ability to conduct and synthesize secondary research, creatively apply an interface technology to solve a real-world problem, and flesh out all the interactions involved with its use.
Introduction
While a student of MHCI+D at the University of Washington, I took a course on User Interface Software and Technology. We were taught a strong conceptual foundation for user interfaces and assigned work to help us explore the boundaries between the role of a designer and an engineer. The course culminated in an interface specification that envisioned a specific application of an interface idea and how it might be implemented. We were asked to communicate the design by creating mockups that illustrate the experience and a specification detailing how it could be implemented.
I chose to focus my efforts on the role of LiDAR (Light Detection and Ranging) in autonomous vehicles. Specifically, I explored how one might apply LiDAR to help prevent collisions between vehicles and deer.
Problem Space
If you’ve driven in any of the states marked in dark grey on the map below, you have faced an increased risk of your car colliding with an animal.
Source: Don't Veer For Deer, State Farm Insurance, October 2020
State Farm Insurance estimates there were approximately 1.5 million vehicle crashes involving deer in the United States during the 1-year period between July 1, 2019 and June 30, 2020. The total cost associated with these wildlife-vehicle collisions is estimated at $8.4 billion annually. This includes the cost of human injury and fatality, wildlife survival and conservation, property damage, law enforcement, emergency service, and road maintenance assistance.
Solutions currently available on the market are effective at providing real-time wildlife tracking information, but they cause drivers to take their eyes off the road.
Concept Summary
This is a conceptual design for an in-vehicle LiDAR deer detection system for drivers of autonomous vehicles in North American regions with large deer populations. The LiDAR Deer Detection (LiDD) system enables forward-facing scanning of vehicle surroundings with a high degree of accuracy. It notifies drivers when a deer is within collision range using a color-coded windshield deer indicator and seat vibration warning. Unlike other solutions, drivers don’t have to take their eyes off the road to receive deer tracking information.
Design Process
Project Phases
I was asked to research and become a subject-matter expert in a specific interface technology. I used this newly acquired knowledge to create a conceptual design and design specification that clearly articulates how to build it.
With this in mind, my process involved the following phases:
- a literature review where I explored the autonomous vehicle space and the technology that powers it;
- a concept definition using what I learned; and
- development of a product proposal via a 2500-word design specification.
Literature Review
I did a literature review of autonomous vehicles, specifically how technologists are addressing trust in autonomous vehicles. I did a review of 15 sources and extracted the following insights:
- Trust is a clear concern when it comes to the future adoption of autonomous vehicles.
- Communicating the vehicle’s understanding of the surrounding environment gives drivers confidence when they need to intervene.
- Calibrating trust between drivers and autonomous vehicles prepares drivers for situations when they need to take control of the vehicle.
These insights indicate that ensuring trust in a vehicle’s automated functionality is going to be key to its successful adoption and use. Proposed solutions should therefore promote trust by accurately and contextually communicating the vehicle’s understanding of the external environment.
LiDAR Research
A key component to trust was the vehicle’s ability to accurately interpret the external environment. Curious to better understand the technology behind autonomous vehicles’ eyes, I chose to explore LiDAR.
A busy street scene captured by Velodyne’s Alpha Puck, a 360 degree horizontal field of view lidar, is seen in this November 2018 image in San Jose, California, U.S. released on February 20, 2019. Courtesy Velodyne/Handout via REUTERS.
LiDAR is a remote sensing technology that uses pulsed lasers to measure the distance between the lazer and surrounding objects. Vehicle sensors create a precise, three-dimensional map of the shape and surface characteristics of the vehicle’s external environment, much like you see in the image above.
LiDAR’s strength is in accurate distance measurement and object detection. It simply can’t be matched by other sensing technologies in the market today. Given its dependence on lasers, however, LiDAR’s accuracy and reliability can be hindered by bad weather, large reflective surfaces, dense forests, and poorly calibrated scanners. Given these strengths and limitations, research indicates it is best to pair LiDAR with complementary technologies like radar and cameras with computer vision to ensure the most accurate and reliable environmental scan and comprehension.
I decided to stay focused on LiDAR because of its strengths and because its weaknesses could be compensated for by other existing technologies.
Ideation
After gaining an understanding of trust in autonomous vehicles and exploring LiDAR, I began ideating and exploring the boundaries in which LiDAR could be used in vehicles and beyond. I developed over 50 ideas.
Ideation Sketches
After another quick round of research, I learned that deer collisions are a problem in the US and LiDAR showed potential to be used as a warning system.
For this reason, I landed on creating a deer detection system using LiDAR.
Concept Development
To illustrate the concept and enable early feedback, I developed a storyboard and shared it with peers. Storyboarding was a great way to give the concept life without having to create a higher-fidelity prototype this early in the process.
LiDD System Concept Storyboard
Initial feedback helped me define, more clearly, details of the concept. It also allowed me to realize that a visual indicator with colors corresponding to how much time the driver has to react wouldn’t be accessible to color-blind populations. At this point, I began to explore seat vibration as a secondary interaction to alert drivers to the potential for a deer collision.
For this reason, I landed on creating a deer detection system using LiDAR.
LiDD System Original Concept Sketch
I then used the storyboard and a few sketches to test the concept with peers, friends, and professors. In this hand-drawn sketch (below), I compared my concept to current solutions that present deer on the vehicle’s dash. This helped me establish whether the concept had more merit than current solutions. Testing also helped me identify questions that needed further clarification.
LiDD System Sketch Comparing New and Existing Solutions
Design Response
Based on what I learned from my initial research and feedback, I began to document the concept’s key functionalities in the form of a design specification.
The design combines LiDAR with a pre-existing deer classification algorithm to scan the road ahead and identify wildlife. When a deer is detected, a real-time augmented reality (AR) windshield display shows the driver where the deer is located.
Visual Warning System
Figure 1: Shows Different Sized and Colored Boxes
Figure 1 shows the different-sized and colored boxes that will be displayed on the driver’s windshield to identify and outline the deer’s rough size and location. The size and box color varies depending on the size of the deer, how close it is to the vehicle, and how much time the driver has to react.
Red, yellow, and green box colors were chosen because they are pretty standard around the world as colors that convey a level of caution and urgency. The colors used here have been modified slightly so they will be eye-catching and easy to see on the dark (nighttime) windshield.
The boxes outline the deer in order to get across the deer’s approximate size and distance away from the vehicle without compromising the driver’s field of view for safety reasons. If there are multiple deer, the system will draw a box around every deer.
Figure 2: Shows a red box around a deer detected on the right-hand side of the road
Figure 2 shows a red box around a deer detected on the right-hand side of the road. The box is red and large when the driver has 2.5 seconds or less to react to avoid a possible collision. The box follows the deer in real-time ensuring the driver knows the position of the deer at all times. The box appears on the windshield so the driver does not have to take their eyes off the road to see the warning.
Vibration Warning System
Figure 3: The Driver's Seat Vibrates When the Deer is Close
Figure 3 is almost the same as Figure 2 except this time, I am showing that the vehicle’s driver seat vibrates when the deer is close. The vibration will increase in intensity as the box colors progress from green to yellow to red and indicate a low, medium, or high urgency to react.
Vibrations are brief and act as a secondary warning system, thereby decreasing the chance that the warning will be overlooked or ignored. It is also an accessibility feature for people who are color blind, serving as an indication of how urgent the need is for them to react.
If there are multiple deer, the system will only invoke the vibration alert for the deer closest to the vehicle.
Visual and Vibration Decision Matrix
Figure 4: LiDD Visual and Vibration Decision Matrix
Assuming an average driver perception-reaction time of 1.5 seconds, box colors and reaction times are suggested in Figure 4. I used reaction time rather than distance to trigger visual and vibration alerts because it centers attention on the perspective of the driver over other measures such as distance.
Reaction times are estimated and depend on many variables such as the speed of the vehicle, the distance the deer is from the vehicle, road conditions, and individual driver reaction times.
These are suggestions. I will need to coordinate with engineering and safety teams to determine the most appropriate time to trigger the box and vibration warnings.
System Confidence and Trust Building
Based on what I learned about trusting in autonomous vehicles, I felt it was important to set priority on system confidence to avoid false alarms which may cause drivers to distrust or ignore the system. While every deer might not be recognized by the system, the driver will be more trusting and responsive to the warnings they receive instead of potentially ignoring them.
I want to avoid all unnecessary distractions. False positives would constitute an unnecessary distraction.
Personal Reflections
- Timing of Interface Technology Research
While I believe it’s ideal to identify a concept that best fits people’s needs before selecting a technology, I found a lot of value in dissecting LiDAR’s key competencies and limitations in advance. This helped me better define what was technically possible and where there may be areas of difficulty while I was exploring concepts.
- Solo Project
This was a solo project and it really drove home the value of working collaboratively on teams. I missed having the day-to-day opportunity to bounce ideas off others, gain different perspectives, and test assumptions. It reaffirmed the value of working in teams on design projects.
- Passion Project
I really enjoyed diving into the complex world of autonomous vehicles and some of the technology that powers them. The interactions in the automotive space are complex, the technologies vast, and the ecosystem even more immense. I hope to continue learning and evolving in design and to make my way into the fascinating automotive space.
- Defining Interactions
While specifying the concept, I particularly enjoyed defining its interactions and considering the many scenarios in which these interactions could fall short or behave unexpectedly or in ways that are unintended.
Next Steps
- Refine Triggers
I would like to speak to an engineering team about the visual and vibration decision matrix to determine if response times are the appropriate metric to trigger the warnings or if some other measure would be more appropriate.
- Explore Confidence Levels
I would like to explore ideal confidence levels with engineers to optimize warning triggers without eroding driver confidence in the system.
- Test Higher Fidelity Prototype
I would love to create a higher fidelity prototype of the detection system and test it to further refine the interactions and get more feedback
- Explore Augmented Reality
I would like to explore the capabilities and limitations of augmented reality in automotive applications.
REFERENCES
[1] https://newsroom.statefarm.com/animal-collision/
[2] Huijser, M. P., McGowen, P. T., Fuller, J., Hardy, A., Kociolek, A., Clevenger, A. P., Smith, D., Ament, R. Wildlife-Vehicle Collision Reduction Study. Report to Congress. U.S. Department of Transportation, Federal Highway Administration, Washington, D.C., 2008. https://westerntransportationinstitute.org/wp-content/uploads/2016/08/4W1096_Report_to_Congress.pdf
[3] Bissonette, John A., Kassar, Christine A., Cook, Lawrence J., “Assessment of Costs Associated With Deer-Vehicle Collisions: Human Death andInjury, Vehicle Damage, and Deer Loss” (2008), Human-Wildlife Interactions. 61. https://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1060&context=hwi
[4] J. Chen, H. Xu, J. Wu, R. Yue, C. Yuan and L. Wang, “Deer Crossing Road Detection With Roadside LiDAR Sensor,” in IEEE Access, vol. 7, pp. 65944-
65954, 2019, doi: 10.1109/ACCESS.2019.2916718. https://ieeexplore.ieee.org/document/8713983
[5] Jernigan, J. D., & Kodaman, M. F. (2001, May). An Investigation of the Utility and Accuracy of the Table of Speed and Stopping Distances Specified in the Code of Virginia (Rep.). Retrieved March 10, 2021, from Virginia Transportation Research Council website: https://rosap.ntl.bts.gov/view/dot/19529