Plus starts mass production of "intelligent trucks" based on L4 technology next year.
“We believe [it] will take billions of real-world miles to prove safety before removing the human driver. While a daunting number, when it comes to people’s lives, we must do what is right.”
This bold statement from the just-released Plus Safety Report puts a stake in the ground, differentiating this AV truck developer from those who are relying on a combination of real-world miles and simulation for validation.
The Report will be a good primer for anyone interested in how an AV truck is developed and validated, highlighting key issues and the approach Plus is taking. For industry veterans, much of the content is likely to echo their own work, but there are some key differences.
This report is by far the most comprehensive information released by Plus to date. Here I’ll provide an overview of the Plus philosophy and some interesting tidbits on their technical approach, as expressed in the Safety Report. (Disclosure: I am an advisor and hold an equity position in Plus.)
Headquartered in Cupertino, CA, Plus was founded in 2016 by Stanford grads David Liu, Shawn Kerrigan, Hao Zheng, and Tim Daly. Like the other truck Automated Driving System (ADS) developers, Plus is focused on automating long-haul trucks operating on limited-access divided highways, where the vast majority of truck freight mileage happens. But that’s not all. Defining their Operational Design Domain (ODD), Plus says “We are training our driving system to handle commercial roads and highways in order for our self-driving trucks to go from one distribution hub onto the highway, down the highway, and then to another distribution hub on the other end.” TuSimple has affirmed this highway-plus-surface-street approach as well, while the other truck ADS developers appear to be highway only, relying on facilities immediately adjacent to the highway to transfer loads to a human driven tractor for the remainder of the journey.
Plus describes doing active testing with “Fortune 500 private shippers” but has thus far declined to name names. They assert that their vehicles can provide both “safety and fuel efficiency gains” to fleets deploying their system.
Mass Production Starting in 2021
On the day of the Safety Report’s release, CEO David Liu spoke at an industry event and noted that Level 4 long-haul trucking “will be commercially ready around 2023-2024.” Regarding deployment strategy, he noted that Plus will “deploy our fully functional and safe driving system to assist drivers; we are starting to mass produce these systems next year. Then, after a few years, we can show our autonomy system is safe without the monitoring of a human driver and we can take the driver out. We believe this is a better and safer approach because you need to build up the billions of real-world miles to statistically prove it’s safe before you completely remove the driver from the truck.”
Shawn Kerrigan, Plus COO and co-founder, elaborated further: “We have targeted go-to-market strategies for each market. In the US, we’re working with OEMs and fleets to incorporate our driving system into automated trucks for fleets. In China, mass production of China’s first intelligent trucks, which we co-developed with FAW, will begin in 2021. These initiatives, and others we have not announced yet, will put thousands of trucks with the Plus automated driving system on the road to enable us to collect billions of real-world miles in the next few years.”
Voluntary Safety Self-Assessment
Formally speaking, the Plus Safety Report is a Voluntary Safety Self-Assessment (VSSA) Letter submitted to the National Highway Traffic Safety Administration (NHTSA). In 2016, the Agency’s AV Policy 2.0 requested VSSAs from ADS developers on a voluntary basis. This has resulted in over 20 submissions from startups working in robo-taxi, robo-shuttles, robo-delivery, and robo-trucking, as well as vehicle manufacturers.
NHTSA requested entities developing ADS to describe their approach across a wide range of topics including system safety, Operational Design Domain, Object and Event Detection and Response, Minimal Risk Condition, validation methods, data recording and sharing, vehicle cybersecurity, consumer education and training, post-crash behavior, and regulations at the federal and state level. There is no Federal approval or comment process; once submitted, VSSA’s are posted on a NHTSA webpage.
Of the AV truck players, Ike, Kodiak, and TuSimple have contributed thus far. Truck ADS developers Aurora and Waymo submitted VSSAs as well, but these were more about their passenger vehicle ADS development.
The VSSA approach has provided a common format to gain insight into a company’s stated values, culture, and overall proficiency. Since the documents are meant to be understandable to non-technical readers, they provide an opportunity to educate policy-makers and the general public.
Two Step Process
Plus is the 24th ADS developer to deliver a VSSA to NHTSA. Their nearly 40-page document opens with a description of their philosophical approach to automated driving, followed by discussions of “how and where the system works,” safety, test and validation methods, and commercialization approach.
The Plus Report describes two discrete steps towards commercialization. First, develop a robust and safe self-driving capability. Second, launch a Level 4 driverless truck product that is significantly more safe than a human driver. I’ll add that safer than a human doesn't mean perfect. Any self-driving system can miss a critical element or make a wrong decision; it’s about reducing the probability of that to a very low level.
No truck ADS developer has deployed driverless services to date. Waymo has done so for their robo-taxi services, and others such as Cruise are now following. Earlier this year, Waymo announced that its autonomous cars have driven tens of billions of miles through computer simulations and 20 million miles on public roads. No one has yet announced having done billions of road miles, or anywhere close to that level.
How can this be accomplished? I put this question to Shawn Kerrigan, who said “ADS developers working to build their own fleet would take many years to have tens of thousands of trucks to accumulate billions of miles. The deployment of Plus technology into commercial truck fleets will enable high annual mileage accumulation. 10,000 trucks can accumulate one billion miles in a year. Plus will have conducted billions of miles of testing in a few years.”
Checking ODD
Plus’s Safety Report details a rigorous methodology in managing ODD. Aligning with industry practice generally, their concept of ODD is “the set of geographic regions, environmental factors, surrounding traffic scenarios, and required maneuvers in which the self-driving system has very high confidence of safe execution.” For any freight-focused ADS company, commercial success means increasing the depth and breadth of the ODD to match the full needs of shippers.
Plus describes their concept of an “ODD Checker” to avoid a situation in which the ADS is operating outside of its ODD. The Report states that “every Plus feature includes a structured ODD definition and a set of tests for determining whether the operating conditions meet the defined ODD,” such that “ODD is managed at various levels within the self-driving software stack.” In this way “each system component can independently signal that it is going out of, or has gone out of its ODD,” so that “the ODD Checker determines which components are critical and transitions the truck’s ODD status as needed.” The Report notes that this goes beyond the current moment, i.e. “this emphasis on monitoring for a valid ODD extends to predicting and reacting to future eventualities.”
Sensors, Redundancy, and Data Fusion
The Plus Safety Report makes some interesting observations about the evolving role of cameras in truck ADS, emphasizing how modern deep learning methods have taken camera-based perception to much higher levels of performance. The Report notes that “deep learning brings an entirely new challenge. How can we prove that such an enormous [AI] model is safe and robust in all cases? This challenge is what underlies our decision at Plus to generate massive test datasets by driving billions of miles with our perception system in a human-centered automated truck.”
The Report also emphasizes the importance of redundancy in sensing. The Plus approach is to maintain high performance for each sensing modality (camera, radar, lidar) on its own. This supports a degree of redundancy such that “the system is theoretically capable of driving with only a single sensing modality.” They are quick to point out however that “on public roads, we always combine all three major sensing modalities to ensure the highest level of safety.”
Validation Takes Many Forms
As truck ADS capability matures, validation has come to the fore. As a key part of their test and validation methods, Plus describes their simulation processes as a) scenario simulation, b) large-scale replay simulation, c) perception simulation, and d) Vehicle-in-the-Loop simulation on a closed course.
In terms of software validation, the Report describes three distinct approaches. For perception and localization, statistical validation is used to prove that what the system says it detects is in fact true. For path planning, mathematical validation using formal mathematical models is employed. For actuation and control, established functional safety processes are used to validate safe system control decisions.
Coming back to their main thesis, the Report states that public road testing “is invaluable and the only way to truly show that our self-driving system is safe and works in the real world. Simulating the real world well enough that it can replace road testing is harder than building a self driving vehicle. The fact that the system works in the simulator does not provide strong statistical evidence in the case of truly unexpected events.” Echoing what has been rightly said by others in their Safety Reports, the Plus Report says “We understand that public road testing comes with the highest degree of responsibility. We do not take this responsibility lightly, and only conduct public road testing for software changes that we deem ready and that are truly necessary.”
Plus has seen many unusual items rolling down the highway, such as this wind turbine blade on a ... [+]
Plus has had their own run-ins with unusual items rolling down the highway. The Report provides examples such as a wind turbine blade on a specialized flatbed truck being followed by an escort vehicle, as well as haulage of a full-sized tree. They note that the ADS needs to be able to handle such extraordinary events as “ordinary.”
The mix of on-road versus virtual validation will continue to be a point of debate for ADS developers. Another is independent testing, which Plus sees as “a cornerstone of our plans for commercial deployment.” They see this as another layer of validity since any bias on the part of the development team is eliminated. Here’s how the Report puts it: “If human drivers have to pass a driving test to obtain a commercial driver’s license to operate a semi-truck, we believe third party validation should be required of self-driving systems as well.”
As announced last summer, Plus is working with the Transportation Research Center (TRC) as their independent testing house. Plus says that the testing will challenge the Plus self-driving system’s ability to handle “complex, realistic driving conditions, with multiple vehicles operating in the vicinity of the Plus truck.” Plus points out that the TRC track facilities “enable self-driving truck operation across various operating conditions, many of which would be considered hazardous to test on public roads.”
Miles And Miles And Miles
2021 is shaping up to be “the year” for various forms of truck ADS system launches. Plus is bringing “intelligent trucks” while others are aiming for launching fully driverless capability “on a limited number of routes.” Plus CEO David Liu sees “tens of thousands” of their pre-L4 trucks on the road in the next few years.
As more details are released in the coming months, it will be fascinating to see how the overall truck AV situation unfolds. At this stage, what we don't know is far greater than what we do know. In particular, we don't know what the powerhouses Daimler and Waymo are contemplating for their commercial driverless truck launches.
As noted above, Plus aims to operate on surface streets taking loads from a distribution center to the highway and then to the final destination. Further, Plus asserts that while simulation is vital for development, full validation must be done on-road at very high mileage levels to reach the commercial launch stage. Apparently, their miles-heavy validation approach will apply to both surface streets and highways. Others truck AV developers have said in their VSSAs they will rely on a combination of miles and simulation to reach a “driverless-ready” level.
This is starting to look like a validation battle. I hear highly respected players in this space making very different statements. I’m not an expert in safety validation specifically, but I suppose there can be more than one legitimate approach. Who will validate the right approach to validation?
