Blog Post

Addressing “Order” with Chaos.

Lawrence Lerner • Nov 06, 2023

How the Chaos Monkeys can save us from AI.

On October 30th, President Biden issued an executive order on safe, secure, and trustworthy artificial intelligence.


The order purports to be a direction and standard by which America “leads the way” in AI by “seizing the promise and managing the risks of AI.” What does that mean? Considering the EU has had a committee working on this for two years, it’s an oddly placed fast follow. I called for standards in XXX Businesses, developers, and creators will need help untangling the rhetoric. While agencies are developing the directed standards, what are we to do?


It’s easy to shred this executive order, but leaders look for people to come together during difficult times. Without making this a 20-page white paper, here are the highlights:


There are eight major sections with a total of 26 sub-sections.

  1. New Standards for AI Safety and Security
  2. Protecting Americans’ Privacy
  3. Advancing Equity and Civil Rights
  4. Standing Up for Consumers, Patients, and Students
  5. Supporting Workers
  6. Promoting Innovation and Competition
  7. Advancing American Leadership Abroad
  8. Ensuring Responsible and Effective Government Use of AI


The needs and tests remind me of my Life Sciences and Pharmaceutical consulting practices. There are lessons learned and techniques that can be lifted. More later.


“Require that developers of the most powerful AI systems share their safety test results and other critical information with the U.S. government.

Standards are required to define what is “most powerful,” and what are the lists of tests? NIST (National Institute of Standards and Technology) is the candidate for the US. This section requires “critical information” to be shared and reviewed before commercial release. This is the most contentious of sections and happens to be the first. There is a lot to define and unpack.


Ultimately, guardrails that are well-documented, unambiguous, and consistent with ethical and business practices are required.


“Develop standards, tools, and tests to help ensure that AI systems are safe, secure, and trustworthy.”

Another way to say this is to build and test for “do no harm.”


“Protect Americans’ privacy by prioritizing federal support for accelerating the development and use of privacy-preserving techniques”

Yes. Enuff said.


“Address algorithmic discrimination”


The “Advancing Equity and Civil Rights” section contains several variants of this statement. Ultimately, ensuring the data and algorithms have no biases is essential.


The list goes on with directional guidance. Addressing these will take time, resources, debate, and more. The document feels rushed, but there are solutions.

Enter the Chaos Monkeys 


We love to see people break things. Why? Because it is something we cannot or will not do for ourselves? Who wants to destroy their new $1200 phone to see if it can survive being baked into a loaf of bread?


Is it a form of intellectual speculation and engagement? Yes. We watch the YouTube torture tests that don’t follow the clean-room robotic arm opening and closing a phone 100s of thousands of times. We want to see ourselves in those “safe” fantasies of dropping the phone on a tile floor or the kids knocking a phone into the sink for the third time.


Netflix innovated the idea of the Chaos Army. The idea is based on testing and ensuring massively scalable uptime across hundreds or thousands of devices worldwide.


By integrating principles of drug testing and clinical trials with the philosophy of the Chaos Army, there is an opportunity to test AI and LLMs rigorously and in a proven way.


What torture tests are you willing to, or wish you had put your products in before they failed in the field? 

ABOUT THE AUTHOR


Lawrence


I translate the CEO, Owner, or Board vision and goals into market-making products that generate $100M in new revenue by expanding into geographies, industries, and verticals while adding customers.


As their trusted advisor, leaders engage me to crush their goals and grow, fix, or transition their businesses with a cumulative impact of $1B


👉🏼 Subscribe to Retail industry news, unpacking trends, and timely issues for leaders.

 

Ready to grow, address change, or transition your business? 👉🏼  Let's brainstorm

Share by: