News

Level Setting Supplier Cybersecurity Maturity

January 20, 2022 Charles Wilson, Senior Principal Engineer, Cybersecurity Cybersecurity

This article was originally published on Motional's Medium page, Minds of Motional, on January 20, 2022

In my post, AVCMDS: Autonomous Vehicle Cybersecurity Manufacturer Disclosure Statement, I introduced the AVCMDS as a way to help AV companies get a snapshot of a supplier’s current cybersecurity capabilities. In this post we’ll consider a way to establish how mature a supplier’s development cybersecurity is.

Measuring Capability

The method we’ll use is the Capability Maturity Model (CMM)[1]. Developed in the mid-1980s for the U.S. Department of Defense, this quantizes maturity into five (really six, when you add the true zero) levels. These are:

A chart of the Capability Maturity Model

Credit: Charles Wilson

We can visualize this (and show relative cost/complexity) as follows:

A chart showing the maturity level compared to cost and complexity

Figure 1 — Capability Maturity Levels [5]

We can enhance this diagram with information showing the interplay between the CMM levels and risk, process review lag, and shareholder value.

A chart showing maturity level, cost and complexity, and shareholder value

Figure 2 — Capability Maturity Sweet Spot [5]

Here we see a “sweet spot,” where perceived shareholder value is high, and both risk and process review lag are low. We can also see a negligence threshold between levels 1 and 2. This gives us an effective floor for desired maturity. At the other end of the spectrum, we can see that beyond level 4 the perceived value and risk level off. This is not to say that it is not desirable to attain level 5, but that one should recognize that there will likely be a high level of internal friction involved given the associated cost and complexity increase.

Covering Your Basis

With an understanding of how we’re going to quantify maturity, we turn to what we will measure. We will use the AVCDL [2] as the basis for our evaluation. We use the AVCDL because it is built around the development lifecycle rather than any specific certification standard. This is because the AVCDL generalizes the cybersecurity needs of standards such as ISO 21434, ISO 26262, and UNECE WP.29 R155. So long as the AVCDL can be shown to satisfy any arbitrary certification standard, the maturity information will be transferable.

Know Thyself

The maturity information is gathered via a simple spreadsheet (AVCDL CMM) [6].

Credit: Charles Wilson

This spreadsheet covers all AVCDL phase requirements and their associated products. Suppliers are asked to self-report their maturity for each of the phase requirement products. They may also provide notes for each.

We can then visualize the provided responses using a radar diagram.

A radar diagram showing vendor capability results

Trust, but Verify

It’s really important to sanity-check the self-reported maturity values. If a supplier asserts that they are at level 1 (ad hoc activity), ask to see the products. If the supplier asserts that an activity has a maturity of level 2 (documented activity), ask to see the documentation. These are things that will come up in an audit, so suppliers should have no excuse in producing material to support their claims. Additionally, this information will be used in the creation of the cybersecurity interface agreement, so accuracy matters. An activity with a maturity level of 0 would necessitate that either the customer or a third party be used in order to ensure that the activity is properly handled.

How the AVCMDS and AVCDL CMM Differ

Where the AVCMDS gave us a snapshot of cybersecurity technical posture, the AVCDL CMM gives us insight into the cybersecurity process posture. It shows us supplier strengths and weaknesses with respect to supporting their cybersecurity efforts. This is critical in determining whether a supplier is capable of undertaking the activities necessary to ensure the cybersecurity of the element they would provide. It is also a major input into the cybersecurity interface agreement.

Bringing Things Together

In the next post in this series, we’ll consider how to establish the division of responsibilities using a cybersecurity interface agreement. We’ll also show how the AVCMDS and AVCDL CMM inform the completion of that document.