DESIGN ☢ AGAINST ☯ AI  ·  designintech.report 2024  ·  @johnmaeda

DESIGN ☢ AG(AI)NST ☯ AI

DESIGN AG(AI)NST AI

30-minute Rapid Walkthru

Design in Tech Report 2024


  1. Design in Tech Reports: Looking Back To 2015
  2. Let’s Step Back To Take A Look At The Forest
  3. ABCs of AI … At Least For Now
  4. Shiny Things 🐿️✨ Snack In 2 ✏️Flavors🏎️
  5. Makers to Experience Profound, Ongoing Changes
  6. Critical Thinking + Critical Making
  7. Concluding Thoughts About Design Against AI

☉

“This new kind of AI behaves … unpredictably.”
— a colleague

In which quadrants do LLMs
(and humans) perform best?

Aware Unaware
Facts KNOWN
KNOWNS		 Unknown
Knowns
Fictions Known
Unknowns		 Unknown
Unknowns
This is referred to as the Rumsfeld Matrix or Johari Window

🆕 Unknown Known For Design

Expand essence

Extract essence

Conversational dzn ain’t new

Erika Hall (2018)

“It’s the oldest interface. Conversation is how humans interact with one another, and have for millennia.”

LLM AI ain’t new, too

Nicholas Negroponte (1967)

“Imagine a machine that can follow your design methodology and at the same time discern and assimilate your conversational idiosyncrasy. The same machine, after observing your behavior, could build a predictive model of your conversational performance. Such a machine could then reinforce the dialogue by using a predictive model to respond to you in a manner that is in rhythm with your personal behavior and conversational idiosyncrasies.”

⚠️ Dr. Weizenbaum’s 1960s warning

Weizenbaum @MIT invented AI therapist chatbot in 1966
I knew of course that people form all sorts of emotional bonds to machines, for example, to musical instruments, motorcycles, and cars. And I knew from long experience that the strong emotional ties many programmers have to their computers are often formed after only short exposures to their machines.
What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.
This insight led me to attach new importance to questions of the relationship between the individual and the computer, and hence to resolve to think about them.”

“It’s a sad moment when we are actually thinking that being lonely can be fixed by a back-and-forth interaction with something that has no idea about what it is to be born, about what it is to die.” —Sherry Turkle: Surprising Promise and profound perils of AIs that fake empathy*
📱 Lean Forward Polls
This will send you to Poll No. 1 on Typeform

I. Design in Tech Reports: Looking Back To 2015

This year was different with AI

I created workflows with GPT-4 to take all my bookmarked URLs for the year to produce my card system in 1/100th of the time.
How? I used it to write Python code that I would never want to write myself :-)

Sorry my AI @johnmaeda demo doesn’t run online yet

THE 3 KINDS OF DESIGN

🎻
Classical
Design

Bauhaus-ian, traditional recipe with great taste.

🧐
Design
Thinking

Let’s organizations learn how to adopt more risks.

🤖
Computational
Design

All-things-computing and Moore’s Law impacts.

And then it’s 2019

2018 PRE-INFLECTION POINT

📉 Ethics
+ Tech

↓

🥹 Inclusive
Design

#️⃣ Slacks
Everywhere

↓

🏔️ Remote
Work

🤖 Useful AI
Glimmers

↓

🐣 RunwayML
Founded

2019: YEAR BEFORE COVID-19

“Get everyone to work completely online?

Just like the digital natives on #️⃣Slacks?

NAAAAAAAAH …”

This will send you to Poll No. 2 on Typeform

“People’s Regrets When They Are Dying”

When a sentence appears, please read aloud with me …

  • “I wish I had the courage to live a life true to myself,
    not the life others expected of me.”
  • “I wish I hadn’t worked so hard.”
  • “I wish I’d had the courage to express my feelings.”
  • “I wish I had stayed in touch with my friends.”
  • “I wish that I had let myself be happier.”


Source: @LBS “Building personal and organisational resilience with Richard Jolly” @richardjollyLBS
DESIGN AG(AI)NST AI

LAST YEAR: #NO-CG #NO-AI

Hidari: Stop-Motion Samurai Film

2024 SORA WTH WTH WTH

#AI LOTS OF IT
  1. Lets Step Back
    To Take A Look At
    🌳🌲The Forest🌲🌳

LLM = “a blurry JPEG of the Web”

Ted Chiang | Feb 29, 2023 | New Yorker HT Jonathan Hoefler

Ted Chiang (2023)

“Obviously, no one can speak for all writers, but let me make the argument that starting with a blurry copy of unoriginal work isn’t a good way to create original work. If you’re a writer, you will write a lot of unoriginal work before you write something original. And the time and effort expended on that unoriginal work isn’t wasted; on the contrary, I would suggest that it is precisely what enables you to eventually create something original. The hours spent choosing the right word and rearranging sentences to better follow one another are what teach you how meaning is conveyed by prose. Having students write essays isn’t merely a way to test their grasp of the material; it gives them experience in articulating their thoughts. If students never have to write essays that we have all read before, they will never gain the skills needed to write something that we have never read.”

AWESOME LLM Explainers Abound

Sep 11, 2023 | 📰 FT

Nov 1, 2023 | 📰 The Guardian

Newspaper dataviz teams at the FT and The Guardian are rockin’ it 🎉

LLM “SIMPLE PHYSICS” WITH …

“The ‘genius in a room’ mental model”**

  • Token Window Limit: The genius is locked in the room and can only receive ONE note under the door that you and they can use ONLY ONCE. Think postit notes …
  • No Context = Ungrounded: The genius can only use what’s inside their room and what’s on your piece of paper. They guess with the confidence of a tall dude.
  • Completely Stateless: Once they return an answer, they completely forget everything you wrote to them. It’s like the “Ground Hog Day” movie. But with recurring amnesia.


**👆 @jessicashieh Feb 5, 2023

Watch Josh in 2019

“AI is Your New Design Material” by Josh Clark Jun 7, 2019

Or 30 mins w/ John Oliver

“Artificial Intelligence: Last Week Tonight
with John Oliver” Feb 26, 2023

Or go oldskool books …

Joy
Buolamwini

Helen
Armstrong

Abhijit
Bhaduri

Kat
Holmes

Corinne
Schillizi

Annie
Jean-Baptiste

Kevin
Bethune

Kate
Crawford

It
Me

Or COOK w/ ME: COZY AI 🧑‍🍳 KITCHEN

On LinkedIn Learning


In addition to “AI Engineering Essentials” there’s also
“AI UX for Developers” (and Designers too :+)

On YouTube

Special guests include Paola Antonelli, Nancy Duarte, Kevin Bethune, Wendy Johansson, Tim Allen, and more to come!

Or Follow My 88-yr old Mom

Yumi was 😱 upset that her iMessage autocorrects with
weird words. She asked me to TURN IT OFF for her.

Just last week she reversed her 🧑‍⚖️ decree

  1. 🤖 Robots &
    👓 Spatial Computing &
    🎉 Awesome Humans

🤖 Robots are getting smarter, stronger, faster

Tesla Optimus

Optimus is folding laundry here.
Figure Bot 1

Figure Bot 1 learned how to do this in 30 minutes.
Boston Dynamics Atlas

Atlas has got some serious pecs …
Unitree H1

The H1 can run at 3.3 m/sec — the current world record!

👓 Spatial Computing is baaack

“Apple Vision Pro”

—“Apple’s First Spatial Computer”

“What Leaders Need To Know About Spatial Computing”

Harvard Business Review, Nov 2023

“Creating Augmented & Virtual Realities”

—Erin Pangilinan et al, 2019

1993 minus 25 years 👓 …

Nicholas Negroponte (1993)

via 2016 Design in Tech Report — a reminder that change takes time
This will send you to Poll No. 3 on Typeform

Awesome Humans 2024

Sinerider
by Hack Club

Poetry Camera
by Kelin & Ryan

🆕 Puppets.app
by Morphogen

  1. 📑 ABCs of AI …
    At Least For Now

LLMS grew up exponentially

20142016201820202022
Skip-Gram
GloVE
LM-LSTM
Context
Exemplar-CNN
DeepWalk
LINE
CoVE
Shared LSTM
FastText
BiGAN
NAT
Counting
VGAE
node2vec
GraphSage
RoBERTa
GPT-2
BERT
BigBiGAN
DeepCluster
RotNet
DGI
GPT-3
BART
PET
SimCSE
MoCo
SimCLR
BEiT
MAE
GCC
GraphCL
SUGAR
MoCL
GPT-3
UNIFIED-IO
OFA
FLAVA
Gato
BEiT-3
SimCSE
Magenta = Language Models | Blue = Vision Models | Green = Graph Models | Orange = Unified Models

“A Comprehensive Survey on Pretrained Foundation Models:
A History from BERT to ChatGPT” (May 2023)

Keeping up with AI on YouTube looks like …

[A]BC. Models & No. Parameters

Large Models

🌌

“LLMs”


Advantage: Can do some serious reasoning with depth.

Small Models

📦

“SLMs”


Advantage: Can run faster + locally or “on the edge”

A[B]C. Models & Ownership

Open Models

🛣️

“LLaMA” etc


Advantage: Can be relatively easy to modify by anyone.

Closed Models

🔒

“OpenAI” etc


Advantage: Can be protected and unmodified by anyone.

AB[C]. Model Domain(s)

Mono Modal

1️⃣

“gpt-3.5-turbo”,
“LLaMA”, etc


Advantage: Can be specific and focused.

Multi Modal

🦄

“gpt-4-vision”,
“LLaVA”, etc

image + language, sound + language, video + language, …

Advantage: Can benefit from multiple perspectives.

Model ABCs* Quick Review

  1. A 🌌 Large vs 📦 Small Models
  2. B 🛣️ Open vs 🔒 Closed Models
  3. C 1️⃣ MonoModal vs 🦄 Multimodal Models
  4. D 🧬 Embeddings Models vs ⏭️ Completion Models
  5. F 🧑‍⚕️ Full Model Fine-Tuning vs 🧑‍🍳 LoRA Fine-Tuning
  6. E 🛠️ Function Calling Models ← like RAG but way more 💪
  7. G 🥸 AGENTS ← a HOW versus a model


H, I, J, K, … *There’s new terminology that pops up daily on the Internet. I know … exhausting, huh?

Kwame Nyanning: Agentic UX


“Agentic UX interfaces are composed in real-time by the agent within a generative canvas. The generative canvas and the agent are inextricable and functionally rich. Together they are able to consume and compose text, images, audio, and video content.”

—via Kwame Nyanning, LinkedIn


Example: Cozy AI 🧑‍🍳 Kitchen with Nancy Duarte and Agents
This will send you to Poll No. 4 on Typeform

  1. Shiny Things 🐿️✨ Snack
    In 2 Flavors of ✏️Shiny🏎️

✏️ Handcraft 🌿Selection🌿

Rivian Chirp
drams.framer.website

✏️ More Handcraft 🌿Selection🌿

wordsasimage.github.io
SIGGRAPH 2023 Honorable Mention Award
Notable AI Era 🪴 Work (To Me)

🏎️ Speedcraft 🌿Selection🌿

PlayHT
Yohei’s PredictiveChat

I strongly recommend you check out how fast GenAI images can be rendered now.

More 🏎️ Speedcraft 🌿Selection🌿

Check online: something’s faster today than yesterday!

V. 💪 Makers to Experience
Profound, Ongoing Changes
This will send you to Poll No. 5 on Typeform

Some good news, first

1️⃣

Design systems
broadened the adoption of computational design.

2️⃣

AI prompting
is commonplace. It’s a new computational design skill.


Useful in both ☑️Marketing + ☑️Product Loops

More good news?

Marketing

  • AI Copy Creation
  • AI Image Creation
  • AI Podcast Creation
  • AI Video Editing☆
☆ 🆕 capsule.video is amazing!

Product

  • AI Customer Support
  • AI SW Developer Assist
  • AI Product Design Assist
  • AI “Zero UI” Arrival☆
☆ 🆕 Vercel GenAI SDK is amazing!


Making activity in both marketing and product loops
is getting disrupted by AI on a daily basis.

Makers 💪 vs Talkers 🗣️

Maker →

Developer

  • WTH GitHub Copilot
  • WTH GPT Engineer, etc
  • something new …


Is programming dead?
with Stephen Wolfram and Lex Fridman

Maker →

Designer

  • WTH Adobe Firefly
  • WTH Figma AI Plugins
  • something new …


The big design freak out,
by Robert Fabricant in Fast Company

Talker ↑

Prod/Manager

  • WTH Microsoft Copilot
  • WTH Figma + GPTs
  • something new …


Collapsing the product stack,
by Scott Belsky

Keep an eye on 🛠️ Function-Calling

Vercel v0.dev

The user can teleport to their goal instead of having to follow a path or journey.

  1. Critical 🗣️ Thinking
    + Critical 💪 Making

1️⃣ The Standards

  1. Clarity: Example?
  2. Accuracy: Verify how?
  3. Precision: More details?
  4. Relevance: Why useful?
  5. Depth: Complexity?
  6. Breadth: Other views?
  7. Logic: Fit together?
  8. Significance: Key point?
  9. Fairness: Any biases?

2️⃣ Elements of Thought

  • Points of View
  • Concepts
  • Purposes
  • Questions
  • Information
  • Inferences
  • Implications
  • Assumptions


Reference: Critical Thinking by Dr. Richard Paul and Dr. Linda Elder

1️⃣ The Standards

Clarity, Accuracy, Relevance, Logicalness, Breadth, Precision, Significance, Completeness, Fairness, Depth

are applied to …

→

2️⃣ The Elements

Purposes, Questions, Points of view, Information, Inferences, Concepts, Implications, Assumptions

to develop …

→

3️⃣ Intellectual Traits

  • Intellectual Humility
  • Intellectual Autonomy
  • Intellectual Integrity
  • Intellectual Courage
  • Intellectual Perseverance
  • Confidence in Reason
  • Intellectual Empathy
  • Fairmindedness


💪Maker → 🗣️Manager



Reference: Critical Thinking by Dr. Richard Paul and Dr. Linda Elder

TL;DR Of Criticality

Critical thinking takes time.
Critical making takes time.

And time is … $$$$.

Know about Kahneman’s “fast vs slow thinking” and “System I vs II”? Critical thinking is slow, System II thinking.
In the LLM hype world that’s ReACT, Chain of Thought, Tree of Thought, etc.

Therefore, a division of labor

🏭

For-profits are not good at critical thinking. They make interesting new technologies with key benefits and unintended consequences.

🏰

Non-profits are good at critical thinking. They’re not caught in the treadmill of make-grow-make-grow-make-grow, and are often well-informed critics.

🧱 Criticallify↑-urself

On the Opportunities and Risks of Foundation Models
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
Ten Hard Problems in AI We Must Get Right
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence

🎭 To be human is TBD

“Technology challenges us to assert our human values
which means that first of all,
we need to figure out
what they are.”
— @MIT Sherry Turkle

  1. ⌛️ Concluding Thoughts About Design Against AI

Design Humanizes Technology

“Don’t humanize AI”

Then should we stop designing?

I believe design’s future net new AI value is …

creating 👅palpable customer-centric criticality value.

WARNING: Stay resilient to delusion & illusion → 📘 Turkle + Weizenbaum

Get to know the new criticality

ai
×
design

design patterns catalogue

shape
of
ai

designing
with
ai

design
better
podcast

MSFT
HAX
TOOLKIT

Absorb computational design

HOW TO SPEAK MACHINE (2019)

Penguin Random House

3 material properties
+ 3 design shifts

  1. 🔄 Loops: Never tires
  2. 🌌 Large: Cosmic scale
  3. 👶 Living: It’s alive…
  4. → 🤢 Incomplete 🤢
  5. → 👀 Instrumented 👀
  6. → 🔥 Imbalance 🔥

My Parting 2024 Takeaways

Computational Thinking
Is Invaluable

Coding isn’t key — but understanding computation will help.

Learn how to
speak machine.

Work Transformation
Is Coming FAST

Prepare for AI-induced shifts in design and tech careers, continually.

It’s a marathon,
not a sprint.

Be Critical Of AI
(For Customers)

Evaluate AI’s impact critically; consider fairness and inclusivity.

Show customers the benefits of responsible AI.

This info will be up on designintech.report soon

In the meantime, you’re welcome to visit
the Cozy AI 🧑‍🍳 Kitchen to cook AI with real code!

This Cozy AI pocket of the universe is now closed.

👋 Come again!