This article was originally published on Motional's Medium page, Minds of Motional, on January 20, 2022
In my post, AVCMDS: Autonomous Vehicle Cybersecurity Manufacturer Disclosure Statement, I introduced the AVCMDS as a way to help AV companies get a snapshot of a supplier’s current cybersecurity capabilities. In this post we’ll consider a way to establish how mature a supplier’s development cybersecurity is.
The method we’ll use is the Capability Maturity Model (CMM). Developed in the mid-1980s for the U.S. Department of Defense, this quantizes maturity into five (really six, when you add the true zero) levels. These are:
Credit: Charles Wilson
We can visualize this (and show relative cost/complexity) as follows:
Figure 1 — Capability Maturity Levels 
We can enhance this diagram with information showing the interplay between the CMM levels and risk, process review lag, and shareholder value.
Figure 2 — Capability Maturity Sweet Spot 
Here we see a “sweet spot,” where perceived shareholder value is high, and both risk and process review lag are low. We can also see a negligence threshold between levels 1 and 2. This gives us an effective floor for desired maturity. At the other end of the spectrum, we can see that beyond level 4 the perceived value and risk level off. This is not to say that it is not desirable to attain level 5, but that one should recognize that there will likely be a high level of internal friction involved given the associated cost and complexity increase.
Covering Your Basis
With an understanding of how we’re going to quantify maturity, we turn to what we will measure. We will use the AVCDL  as the basis for our evaluation. We use the AVCDL because it is built around the development lifecycle rather than any specific certification standard. This is because the AVCDL generalizes the cybersecurity needs of standards such as ISO 21434, ISO 26262, and UNECE WP.29 R155. So long as the AVCDL can be shown to satisfy any arbitrary certification standard, the maturity information will be transferable.
The maturity information is gathered via a simple spreadsheet (AVCDL CMM) .
Credit: Charles Wilson
This spreadsheet covers all AVCDL phase requirements and their associated products. Suppliers are asked to self-report their maturity for each of the phase requirement products. They may also provide notes for each.
We can then visualize the provided responses using a radar diagram.
Trust, but Verify
It’s really important to sanity-check the self-reported maturity values. If a supplier asserts that they are at level 1 (ad hoc activity), ask to see the products. If the supplier asserts that an activity has a maturity of level 2 (documented activity), ask to see the documentation. These are things that will come up in an audit, so suppliers should have no excuse in producing material to support their claims. Additionally, this information will be used in the creation of the cybersecurity interface agreement, so accuracy matters. An activity with a maturity level of 0 would necessitate that either the customer or a third party be used in order to ensure that the activity is properly handled.
How the AVCMDS and AVCDL CMM Differ
Where the AVCMDS gave us a snapshot of cybersecurity technical posture, the AVCDL CMM gives us insight into the cybersecurity process posture. It shows us supplier strengths and weaknesses with respect to supporting their cybersecurity efforts. This is critical in determining whether a supplier is capable of undertaking the activities necessary to ensure the cybersecurity of the element they would provide. It is also a major input into the cybersecurity interface agreement.
Bringing Things Together
In the next post in this series, we’ll consider how to establish the division of responsibilities using a cybersecurity interface agreement. We’ll also show how the AVCMDS and AVCDL CMM inform the completion of that document.
- Capability Maturity Model https://en.wikipedia.org/wiki/Capability_Maturity_Model
- Autonomous Vehicle Cybersecurity Development Lifecycle (AVCDL) https://github.com/nutonomy/AVCDL
- ISO/IEC 21827 — Information technology — Security techniques — Systems Security Engineering — Capability Maturity Model® (SSE-CMM®) https://www.iso.org/standard/44716.html
- Systems Security Engineering Capability Maturity Model — Model Description Document Version 2.0 (1999–01–04) https://apps.dtic.mil/dtic/tr/fulltext/u2/a393329.pdf
- CMM level and CMM Sweet Spot images (CC BY-ND 4.0) https://www.securecontrolsframework.com/sp-cmm
- AVCDL CMM template https://github.com/nutonomy/AVCDL/blob/main/distribution/reference_documents/templates/AVCDL%20vendor%20CMM%20template.xlsx