326. Rethinks: The Secrets to Getting the Information You Need from AI

WEBVTT

00:00:03.534 --> 00:00:06.834
Matt Abrahams: AI can generate
answers in seconds, but the quality

00:00:06.834 --> 00:00:10.364
of those answers depends on the
quality of the conversation.

00:00:10.884 --> 00:00:14.143
I'm Matt Abrahams, and I teach
Strategic Communication at Stanford

00:00:14.143 --> 00:00:15.404
Graduate School of Business.

00:00:15.704 --> 00:00:20.304
Welcome to this Rethinks episode of
Think Fast, Talk Smart, the podcast.

00:00:20.903 --> 00:00:25.213
We're revisiting my conversation with
Jeremy Utley and Kian Gohar to explore

00:00:25.214 --> 00:00:30.163
how conversational AI can become a true
thinking partner, helping individuals

00:00:30.173 --> 00:00:34.643
and teams solve problems more creatively
and communicate more effectively.

00:00:35.104 --> 00:00:39.114
Enjoy this journey into
our communication vault.

00:00:41.074 --> 00:00:45.033
The promise of AI is tremendous.

00:00:45.234 --> 00:00:50.243
Yet most of us, if we're using it
at all, are using it incorrectly.

00:00:50.593 --> 00:00:52.424
It's not about a transaction.

00:00:52.784 --> 00:00:55.863
It's about a conversation and interaction.

00:00:56.913 --> 00:01:00.913
I'm Matt Abrahams, and I teach
Strategic Communication at Stanford

00:01:00.913 --> 00:01:02.203
Graduate School of Business.

00:01:02.813 --> 00:01:06.724
Welcome to Think Fast,
Talk Smart, the podcast.

00:01:08.110 --> 00:01:11.950
Today, I am really excited to chat
with Jeremy Utley and Kian Gohar.

00:01:12.220 --> 00:01:14.950
Jeremy is a repeat guest
to Think Fast, Talk Smart.

00:01:15.180 --> 00:01:19.020
He's an adjunct professor specializing
in creativity and entrepreneurship at

00:01:19.020 --> 00:01:24.050
Stanford, and the author of Idea Flow:
The Only Business Metric That Matters.

00:01:24.800 --> 00:01:28.150
Kian is founder of Geolab
and former executive director

00:01:28.169 --> 00:01:29.719
at Singularity University.

00:01:30.020 --> 00:01:34.199
He's the bestselling author of
Competing in the New World of Work.

00:01:34.489 --> 00:01:36.159
Welcome, Jeremy and Kian.

00:01:36.209 --> 00:01:37.150
Thanks for being here.

00:01:37.530 --> 00:01:38.519
Jeremy Utley: Thanks for having me back.

00:01:38.739 --> 00:01:39.870
Kian Gohar: Such a joy
to be here with you.

00:01:40.259 --> 00:01:40.799
Matt Abrahams: All right.

00:01:40.810 --> 00:01:41.699
Shall we get going?

00:01:42.310 --> 00:01:42.439
Jeremy Utley: Let's do it.

00:01:42.439 --> 00:01:42.889
Kian Gohar: Sounds good.

00:01:43.169 --> 00:01:46.720
Matt Abrahams: To get us started,
Kian, I'm curious, what motivated you

00:01:46.720 --> 00:01:50.829
and Jeremy to look at the impact of
AI on creativity and problem-solving?

00:01:51.360 --> 00:01:56.249
Kian Gohar: So in my business, we coach
teams to achieve complex business goals

00:01:56.249 --> 00:01:58.679
through team transformation practices.

00:01:58.700 --> 00:02:02.210
Sometimes this involves tackling
a difficult innovation project,

00:02:02.210 --> 00:02:05.589
sometimes it's struggling with
change management, but it always

00:02:05.599 --> 00:02:09.079
boils down to this one issue, which
is: How can we solve this problem?

00:02:09.559 --> 00:02:14.249
And that usually entails human
ideation and prioritization.

00:02:14.680 --> 00:02:18.759
And I wanted to see if we can
bring a new technology tool into

00:02:18.769 --> 00:02:20.429
the mix to get better ideas.

00:02:20.830 --> 00:02:24.199
And given that I've been teaching AI
in Silicon Valley to executives for a

00:02:24.199 --> 00:02:26.769
long time, that was always my hunch.

00:02:26.949 --> 00:02:31.109
But it wasn't until ChatGPT became
publicly available that I had the ah-ha,

00:02:31.379 --> 00:02:36.569
and I didn't quite know how it could be
useful to the issue of problem-solving

00:02:36.580 --> 00:02:38.219
on teams, but I had a hunch.

00:02:38.229 --> 00:02:42.530
And so we designed the study together,
uh, to learn and from real world

00:02:42.530 --> 00:02:47.919
practice of how teams might use AI to
get better ideas to solving problems.

00:02:47.919 --> 00:02:51.579
And what we did was we recruited
hundreds of participants from many

00:02:51.580 --> 00:02:57.219
companies in Europe and the US, and
we asked them to identify a particular

00:02:57.360 --> 00:03:00.389
pain point within their organization
or a problem that they needed to

00:03:00.389 --> 00:03:02.430
solve for, something that was real.

00:03:02.519 --> 00:03:06.269
And then we actually gave half of
the participants in each of these

00:03:06.309 --> 00:03:10.930
problem-solving ideation workshops
had access to ChatGPT, and the

00:03:10.930 --> 00:03:12.449
other half didn't have access to it.

00:03:12.449 --> 00:03:14.650
So they were just thinking
on their own as humans.

00:03:14.909 --> 00:03:17.350
And then the other half
had humans plus AI.

00:03:18.150 --> 00:03:22.909
And we went through a problem-solving
ideation exercise, and at the end

00:03:22.910 --> 00:03:26.979
of it, we asked the problem owners
or the executives who shared with

00:03:26.979 --> 00:03:30.899
us the particular pain point to
grade all the various ideas from

00:03:30.959 --> 00:03:35.199
A, being spectacular, all the way
to D, not worth pursuing further.

00:03:35.520 --> 00:03:41.149
And so we ran this blind study to
understand how generative AI can

00:03:41.149 --> 00:03:47.119
facilitate problem-solving, ideation, and
collaboration with real world examples.

00:03:47.840 --> 00:03:51.010
Jeremy Utley: The other thing that we
did was we asked participants before

00:03:51.010 --> 00:03:55.549
and after the session how they felt
about collaboration and problem-solving

00:03:55.560 --> 00:04:00.600
and ideation because we wanted to see
what was the impact of just a standard

00:04:00.600 --> 00:04:03.720
brainstorming session on someone's
attitude towards problem-solving

00:04:03.720 --> 00:04:05.229
and collaboration and innovation.

00:04:05.620 --> 00:04:09.680
And what was the impact of a
session that included generative AI?

00:04:09.680 --> 00:04:12.840
Did it have a differential
impact on participant sentiment?

00:04:13.750 --> 00:04:15.730
Matt Abrahams: And the
research to me is fascinating.

00:04:15.740 --> 00:04:19.500
Jeremy, can you summarize the results
of the research you and Kian did

00:04:19.510 --> 00:04:24.570
and relate it to the quality of idea
generation and the feelings people

00:04:24.580 --> 00:04:27.540
have about the ideas that were created?

00:04:27.870 --> 00:04:32.060
Jeremy Utley: Very broadly speaking,
we found that teams, if they want

00:04:32.060 --> 00:04:36.760
to outperform using AI, they need
to adhere to certain practices.

00:04:37.399 --> 00:04:39.889
And sadly, most teams do not.

00:04:40.269 --> 00:04:44.889
So despite the potential to dramatically
outperform, most teams leave the

00:04:44.889 --> 00:04:48.739
vast majority of their innovation
potential on the table when they use AI.

00:04:48.929 --> 00:04:52.629
When you approach AI like an oracle or
like a search engine just looking for the

00:04:52.629 --> 00:04:57.540
answer, it may feel like magic, but the
data shows you end up underperforming.

00:04:57.890 --> 00:05:01.129
If you want to get world-class
results and outperform, you

00:05:01.129 --> 00:05:02.390
can't approach it like an oracle.

00:05:02.390 --> 00:05:06.620
You need to instead approach AI
like a conversation partner, and

00:05:06.620 --> 00:05:08.659
that doesn't feel nearly as magical.

00:05:08.669 --> 00:05:10.000
It feels more like work.

00:05:10.289 --> 00:05:14.199
And for that reason, the teams
that outperform actually feel worse

00:05:14.440 --> 00:05:15.989
than the teams that underperform.

00:05:16.419 --> 00:05:17.729
It's very counterintuitive.

00:05:18.199 --> 00:05:21.270
Matt Abrahams: I find that absolutely
fascinating, and I also find that

00:05:21.479 --> 00:05:25.979
interestingly, the ability to have
a good conversation with AI is what

00:05:25.979 --> 00:05:30.020
makes the difference for creating
valuable ideas and solving problems.

00:05:30.599 --> 00:05:34.099
Kian, can you go into a little more
depth with the counterintuitive

00:05:34.119 --> 00:05:35.839
findings that Jeremy just shared?

00:05:36.229 --> 00:05:40.850
Why do you think it is that AI-assisted
teams that delivered worse solutions

00:05:40.919 --> 00:05:45.919
actually felt better about their work, and
the AI-assisted teams who delivered better

00:05:45.919 --> 00:05:50.529
outcomes actually felt worse related
to their non-AI-assisted counterparts?

00:05:50.890 --> 00:05:54.770
Kian Gohar: Because rewiring our brains
to act differently takes practice.

00:05:55.030 --> 00:05:59.719
And we professionals have decades
of experience or certain ways of

00:05:59.719 --> 00:06:03.219
doing things, and we're asking
them to work differently now with a

00:06:03.219 --> 00:06:05.249
different workflow, and that's hard.

00:06:05.519 --> 00:06:10.729
It's like going to the gym after not
having exercised for a long time.

00:06:10.769 --> 00:06:14.699
It really sucks, and you don't
particularly enjoy it at first,

00:06:14.949 --> 00:06:16.369
but it has long-term benefits.

00:06:16.750 --> 00:06:21.129
And so we found that teams that were
able to use AI effectively, they

00:06:21.129 --> 00:06:25.239
actually had to rewire their workflow
and how they went about to try to

00:06:25.489 --> 00:06:28.330
incorporate AI as a co-pilot on the team.

00:06:28.749 --> 00:06:32.299
And that's different, and it was
exhausting, and they felt like

00:06:32.309 --> 00:06:35.749
it was just a lot of work, but
ultimately, they had better responses.

00:06:36.260 --> 00:06:37.620
Matt Abrahams: Would you add
something to that, Jeremy?

00:06:38.089 --> 00:06:41.759
Jeremy Utley: Part of the challenge
is, I almost picture a cartoon strip

00:06:41.769 --> 00:06:46.129
with a thought bubble that's like,
"But I was promised superpowers." The

00:06:46.150 --> 00:06:49.939
premise and the promise of AI, I think
it's intimidating, and it maybe strikes

00:06:49.939 --> 00:06:51.409
fear into a lot of people's hearts.

00:06:51.409 --> 00:06:51.999
That's one thing.

00:06:51.999 --> 00:06:53.539
I don't think there's
a reason to be afraid.

00:06:53.540 --> 00:06:56.389
I think there's a lot of reason
for enthusiasm and fun and

00:06:56.389 --> 00:06:58.289
seeking fluency, obviously.

00:06:58.710 --> 00:07:01.869
But I think there's a premise
of this is gonna be like magic.

00:07:02.029 --> 00:07:05.389
But when you start there and you think,
"All I have to do is ask a question, and

00:07:05.389 --> 00:07:06.839
then I get answers," well, guess what?

00:07:07.180 --> 00:07:07.789
You do.

00:07:08.302 --> 00:07:12.602
You ask a question, and you get three
pages of, call it B+ documentation.

00:07:12.632 --> 00:07:17.422
And teams go, "It is kind of
magical. Well, let's go get coffee."

00:07:17.472 --> 00:07:18.672
Like, we're kind of done, right?

00:07:18.981 --> 00:07:23.382
If you're expecting magic, you will
get it, but you won't do that well.

00:07:23.752 --> 00:07:29.532
If you've gotten practice being a
conversation partner to an AI copilot,

00:07:29.792 --> 00:07:33.522
what you realize is the first stuff that
you get out isn't great unless you're

00:07:33.522 --> 00:07:37.712
really thoughtful about framing and
providing context and digging deeper and

00:07:37.712 --> 00:07:41.921
pushing back and asking questions and
treating it much more like a conversation.

00:07:41.921 --> 00:07:45.932
And then the more you treat it like
a conversation, the less magical it

00:07:45.932 --> 00:07:49.762
feels, and the more it feels like AI
is actually getting you to work and

00:07:49.762 --> 00:07:51.792
pulling your best thinking out of you.

00:07:52.032 --> 00:07:55.441
And so I think part of it is just
the mindset people have when they

00:07:55.441 --> 00:07:59.511
hear, "Oh, we get ChatGPT." The
people who go, "Oh, magic," are

00:07:59.581 --> 00:08:01.852
almost always the underperformers.

00:08:02.032 --> 00:08:05.251
The people who roll up their sleeves
and go, "Oh, now we have work to do.

00:08:05.261 --> 00:08:07.842
It's maybe a different kind of work
to do, but I'm excited to do it,"

00:08:08.341 --> 00:08:09.101
those are the people who outperform.

00:08:10.291 --> 00:08:14.351
Kian Gohar: I would also add that
there is a reason why ChatGPT gives

00:08:14.351 --> 00:08:19.512
you generic answers, and it's designed
mathematically to give you the most likely

00:08:19.842 --> 00:08:21.792
answer that it thinks you want to hear.

00:08:21.851 --> 00:08:25.301
And so as a result, the first
few suggestions it offers are

00:08:25.302 --> 00:08:26.651
supposed to be generic and average.

00:08:26.941 --> 00:08:30.751
And so unless you have this conversational
back and forth with it like you would

00:08:30.872 --> 00:08:34.722
with a colleague or a friend to push it
and to give it context, you're just gonna

00:08:34.722 --> 00:08:38.141
get average answers and average ideas,
and that's really not good enough if

00:08:38.141 --> 00:08:39.891
you're trying to solve complex problems.

00:08:40.592 --> 00:08:43.471
Matt Abrahams: It comes down
to mindset shift and work.

00:08:43.541 --> 00:08:44.961
There is no magic.

00:08:44.961 --> 00:08:45.991
It's not for free.

00:08:45.991 --> 00:08:49.812
If you really want to solve problems
well and get creative, you actually

00:08:49.821 --> 00:08:51.471
have to have a conversation.

00:08:52.081 --> 00:08:55.801
So Jeremy, can you share your FIXIT
methodology and describe the different

00:08:55.802 --> 00:08:56.811
components and how they work?

00:08:57.431 --> 00:08:58.121
Jeremy Utley: Absolutely.

00:08:58.122 --> 00:09:04.242
So FIXIT is a five-step methodology
to turbocharge your collaboration.

00:09:05.120 --> 00:09:08.040
F stands for you have a
focused problem, right?

00:09:08.040 --> 00:09:10.080
So you don't want to boil the
ocean, but you want to be very

00:09:10.080 --> 00:09:14.910
focused in the kind of challenge
that you are bringing to ChatGPT.

00:09:15.060 --> 00:09:19.010
So in this example, instead of saying what
recommendations would you make for a human

00:09:19.010 --> 00:09:25.050
interfacing, I would say, "I'm joining a
podcast, and I'd like to give concrete,

00:09:25.060 --> 00:09:30.419
tangible suggestions to professionals who
haven't had much experience with ChatGPT.

00:09:30.660 --> 00:09:35.720
If you had three or four, or maybe I'd
start with ten suggestions for someone

00:09:35.720 --> 00:09:39.350
who's unfamiliar but wants to be more
familiar, start by asking me three

00:09:39.350 --> 00:09:42.280
questions to better understand the kind
of professional I'm describing." Right?

00:09:42.649 --> 00:09:45.879
That's what we mean by F, kind of focused.

00:09:45.890 --> 00:09:47.469
So that's a very focused prompt, right?

00:09:47.840 --> 00:09:50.439
I is ideate individually.

00:09:50.449 --> 00:09:53.579
So before coming to the team, 'cause
remember our study was conducted in the

00:09:53.579 --> 00:09:56.019
context of teams trying to solve problems.

00:09:56.279 --> 00:09:59.789
Before coming to your team and even
before coming to ChatGPT, think

00:09:59.800 --> 00:10:03.040
for yourself about what do you
know and what's your point of view.

00:10:03.040 --> 00:10:08.460
Again, this kind of hones the context
that you bring to the conversation, right?

00:10:08.460 --> 00:10:09.049
It's critical.

00:10:09.290 --> 00:10:12.150
Research has shown that the best way
to brainstorm is to alternate kind of

00:10:12.159 --> 00:10:14.199
individual ideation and group ideation.

00:10:14.500 --> 00:10:16.459
Well, the same is true
actually in the context of

00:10:16.459 --> 00:10:17.650
collaborating with generative AI.

00:10:18.439 --> 00:10:23.259
You want to alternate between individual
personal thinking and thinking

00:10:23.259 --> 00:10:25.259
assisted and amplified by AI, right?

00:10:25.259 --> 00:10:31.889
So that's the I. X is context, so
providing sufficient background context.

00:10:32.180 --> 00:10:35.049
A lot of times we recommend that
folks upload documents, right?

00:10:35.049 --> 00:10:38.340
When we were conducting our study, we
would have a problem owner actually

00:10:38.340 --> 00:10:42.879
do a dossier that we would upload
to ChatGPT to provide context.

00:10:43.070 --> 00:10:45.619
If you don't know what your
context is, here's an amazing hack.

00:10:46.049 --> 00:10:48.519
Ask ChatGPT to ask you for the context.

00:10:48.809 --> 00:10:52.849
So as an example, I've got a friend who's
negotiating a lease trying to build a gym,

00:10:53.129 --> 00:10:58.099
and he's trying to come to an agreement
with the property owner, and there's

00:10:58.099 --> 00:10:59.840
a gap between where they need to be.

00:11:00.209 --> 00:11:03.270
And I said, "Hey, why don't you
ask ChatGPT for help?" He said,

00:11:03.270 --> 00:11:03.849
"Well, how would I do that?"

00:11:03.849 --> 00:11:08.740
I said, "Well, have ChatGPT interview you
about your objectives and then interview

00:11:08.740 --> 00:11:12.130
you about the counterparty's objectives
and then make some recommendations."

00:11:12.130 --> 00:11:17.239
Well, that interview effectively becomes
a means by which you can provide context.

00:11:17.469 --> 00:11:21.120
So the X is make sure that you're
giving ChatGPT context on the problem.

00:11:21.806 --> 00:11:26.736
The second I is interactive
iterative conversation, right?

00:11:26.736 --> 00:11:31.226
So you're never just taking the first
response that ChatGPT gives you.

00:11:31.476 --> 00:11:34.136
You wanna be having a back
and forth and a dialogue.

00:11:34.445 --> 00:11:39.076
So for example, if Matt and Kian and I
are going to try to title this episode,

00:11:39.335 --> 00:11:43.606
we might say to ChatGPT, "Hey, we'd
love ten titles for an episode." And

00:11:43.606 --> 00:11:45.516
then ChatGPT comes back with ten.

00:11:45.855 --> 00:11:49.135
Almost everyone's default is to
think, "Which of these ten do I like

00:11:49.155 --> 00:11:52.416
best?" What we'd recommend is just
immediately say, "I'd like another

00:11:52.416 --> 00:11:56.446
ten." And then read all twenty and
say, "Here are the ones that I like.

00:11:56.475 --> 00:11:59.585
Would you give me ten more like
these?" Chances are you're not gonna

00:11:59.585 --> 00:12:02.456
get as good. And then if you think,
"Wait, why did I like number two?

00:12:02.495 --> 00:12:06.015
Oh, there was a funny alliteration,
and number seven had a funny pun,

00:12:06.495 --> 00:12:10.386
and number nine made reference
to…" Okay, use those as design

00:12:10.405 --> 00:12:12.186
principles for the next ten, right?

00:12:12.326 --> 00:12:14.945
That's an iterative back
and forth that yields.

00:12:15.155 --> 00:12:19.705
We'd probably find that the fourth
tranche of ten title suggestions

00:12:19.705 --> 00:12:23.075
would be radically exponentially
better than the first ten, right?

00:12:23.306 --> 00:12:24.975
But it's by that going back and forth.

00:12:25.466 --> 00:12:27.796
F-I-X-I-T.

00:12:28.166 --> 00:12:30.386
T is for team incubation.

00:12:30.905 --> 00:12:35.216
So it's important to then bring the
ideas that you've generated individually

00:12:35.236 --> 00:12:40.006
and with ChatGPT or whatever LLM you're
using back to your team, and then

00:12:40.166 --> 00:12:42.825
importantly, commission some experiments.

00:12:42.835 --> 00:12:46.495
So this is where this work dovetails
with traditional innovation

00:12:46.495 --> 00:12:51.015
methodology, but you never just wanna
select one idea and move forward.

00:12:51.245 --> 00:12:54.696
It's impossible for an LLM or for
a human to a priori know which

00:12:54.696 --> 00:12:56.285
solution is gonna be the best fit.

00:12:56.655 --> 00:13:01.395
So as a team, you wanna have a practice
and a process around incubating or

00:13:01.395 --> 00:13:05.505
low-resolution prototyping a handful of
the high-potential solutions that you've

00:13:05.505 --> 00:13:09.945
generated in order to determine which
one actually solves the problem the best.

00:13:10.575 --> 00:13:14.165
Matt Abrahams: This methodology,
upon hearing it, makes a lot

00:13:14.166 --> 00:13:18.415
of intuitive sense, but I can
definitely see the effort involved.

00:13:18.785 --> 00:13:22.226
I like that it involves individual
work and collaborative work, not

00:13:22.226 --> 00:13:26.325
just with the ChatGPT LLM, but
also with others on the team.

00:13:26.885 --> 00:13:31.605
Kian, not surprisingly, I'd like
to dive deeper into the fourth

00:13:31.615 --> 00:13:35.495
step that Jeremy introduced us
to: interactive conversations.

00:13:35.925 --> 00:13:39.506
We've spent a lot of time on
this podcast talking about how

00:13:39.506 --> 00:13:43.215
to have better conversations, but
of course, focusing on humans.

00:13:43.645 --> 00:13:48.765
What specific advice can you provide
to us for improving our conversations

00:13:48.765 --> 00:13:54.406
and our communication with LLMs to
make sure we maximize the potential

00:13:54.606 --> 00:13:59.225
goodness that can come from
collaborating with a tool like ChatGPT?

00:14:00.014 --> 00:14:00.334
Kian Gohar: Yeah.

00:14:00.334 --> 00:14:06.244
So the first thing I'd say is to make sure
you've downloaded a LLM app on your phone

00:14:06.604 --> 00:14:12.374
and interact with an LLM on the phone
instead of doing it on the web browser.

00:14:12.724 --> 00:14:16.454
If you don't see a app on your
phone, it's very unlikely that

00:14:16.454 --> 00:14:18.713
you'll use it on a consistent basis.

00:14:18.793 --> 00:14:21.743
And the more you use it, the more
familiar you become with it, the

00:14:21.744 --> 00:14:24.464
more likely it'll actually be part
of your workflow, whether it's

00:14:24.473 --> 00:14:26.263
individually or whether it's as a team.

00:14:26.674 --> 00:14:27.824
So that's really the first step.

00:14:27.873 --> 00:14:30.143
Download one of these apps on your phone.

00:14:30.633 --> 00:14:34.744
And part of that is because we have
historically been, for the last

00:14:34.744 --> 00:14:38.804
twenty-plus years in the internet era
and the browser era, we see a text box,

00:14:38.804 --> 00:14:40.403
and we know exactly what to do with it.

00:14:40.453 --> 00:14:43.524
We type in a particular
word or particular question.

00:14:43.874 --> 00:14:47.683
And now that we have, uh, large
language models, the user interfaces

00:14:47.683 --> 00:14:50.514
look pretty much exactly the same,
like the Google search engine box.

00:14:50.933 --> 00:14:54.783
And it's oftentimes difficult to figure
out the right kind of question to ask

00:14:54.784 --> 00:14:58.204
it or to word it in the right way to
get the right answer to suggestions.

00:14:58.553 --> 00:15:02.463
And so we actually think it's a lot
better if you start interacting with

00:15:02.753 --> 00:15:06.773
these large language models through
conversation, through spoken audio,

00:15:06.774 --> 00:15:10.233
rather than just trying to type it
into a search engine box and trying

00:15:10.234 --> 00:15:11.643
to figure out the exact right prompt.

00:15:12.084 --> 00:15:17.343
The second thing is that you should
be uploading your prompts with audio

00:15:17.603 --> 00:15:21.363
messages, voice messages, and talk to
it just like you would talk to a friend.

00:15:21.624 --> 00:15:25.394
So whether you're talking to a friend
about a particular problem or talking to

00:15:25.394 --> 00:15:29.264
a colleague about a particular problem,
just record it literally on the phone

00:15:29.283 --> 00:15:33.233
on audio to text, and then the large
language model will transcribe that

00:15:33.273 --> 00:15:35.053
and then offer out some suggestions.

00:15:35.363 --> 00:15:39.013
And then the third thing I'd say is
think about using different models.

00:15:39.194 --> 00:15:43.023
There are several different kinds of large
language models that you could use, from

00:15:43.023 --> 00:15:49.143
ChatGPT to Claude to Bing and to others,
and they all have their different flavors

00:15:49.143 --> 00:15:50.843
and different kinds of personalities.

00:15:51.203 --> 00:15:55.404
And you might want to use one of those
models for a particular exercise or

00:15:55.404 --> 00:15:58.863
activity, and then maybe you should go
talk to another model, just like you

00:15:58.863 --> 00:16:01.673
would talk to two or three different
friends and get their opinions or

00:16:01.713 --> 00:16:04.293
two or three different colleagues and
get their opinions, because they're

00:16:04.293 --> 00:16:07.164
gonna give you different kinds of
nuance and different kinds of answers.

00:16:07.573 --> 00:16:09.193
Matt Abrahams: I find that
really helpful advice.

00:16:09.193 --> 00:16:12.903
I mean, we approach the communication
with technology based on the

00:16:12.903 --> 00:16:14.313
interface that we have with it.

00:16:14.323 --> 00:16:17.983
And if we move to our phones,
which we're used to communicating

00:16:17.984 --> 00:16:21.793
with very differently than we
are, let's say, a browser and a

00:16:21.793 --> 00:16:23.783
text box, it can really change it.

00:16:23.883 --> 00:16:27.884
And it strikes me that it's not just
the actual interaction interface, but

00:16:27.884 --> 00:16:30.424
it's also the curiosity we bring to it.

00:16:30.913 --> 00:16:34.504
When I go to enter information
into a search engine, I

00:16:34.504 --> 00:16:35.744
just want to get the answer.

00:16:35.783 --> 00:16:38.933
I don't necessarily see it as
a conversation and a dialogue.

00:16:38.933 --> 00:16:43.953
So that curiosity that I bring with
the follow-up questions or the doubting

00:16:43.963 --> 00:16:47.284
and the exploration and expansion,
I think, is really important.

00:16:47.674 --> 00:16:51.553
And I like the advice to check
with different LLMs, just like you

00:16:51.553 --> 00:16:52.953
would check with different friends.

00:16:52.954 --> 00:16:54.793
It gives you different ideas and inputs

00:16:55.406 --> 00:16:58.795
Jeremy Utley: You can even feed different
LLMs answers into one another, right?

00:16:58.795 --> 00:16:59.486
Matt Abrahams: Oh my goodness.

00:16:59.536 --> 00:17:03.325
We can facilitate and broker
a conversation amongst LLMs.

00:17:03.945 --> 00:17:04.515
Jeremy Utley: I, absolutely.

00:17:04.515 --> 00:17:07.816
I mean, I plug in ChatGPT's answer into
Claude and say, "What do you think of

00:17:07.816 --> 00:17:09.386
this?" all the time, and vice versa.

00:17:09.435 --> 00:17:11.505
Take Claude's answer and
plug it into ChatGPT.

00:17:11.735 --> 00:17:13.295
Ask ChatGPT what it thinks of it's own.

00:17:13.295 --> 00:17:17.935
So this is, this is why Googling is
such a bad metaphor for what we're

00:17:17.935 --> 00:17:21.356
talking about here, because you'd
never query Google about a Google query

00:17:21.475 --> 00:17:23.245
to, to get meta for a second, right?

00:17:23.345 --> 00:17:28.025
But after a sales call, I hop
on my ChatGPT voice app while

00:17:28.025 --> 00:17:29.165
I'm stretching for a run.

00:17:29.316 --> 00:17:32.445
Instead of sitting at the
screen, I'm stretching for a run.

00:17:32.625 --> 00:17:35.505
I say, "Hey, I just talked
with Matt and Kion about our

00:17:35.535 --> 00:17:37.346
research, and a couple follow-ups.

00:17:37.375 --> 00:17:40.775
Would you craft a quick memo that I
could send to the team just thanking

00:17:40.775 --> 00:17:43.216
them for their time today and how much
I enjoyed the conversation," right?

00:17:43.316 --> 00:17:44.205
Well, it's gonna do it.

00:17:44.395 --> 00:17:48.755
Well, then my next thing is I just look
at it and I say, "If I were to upload this

00:17:48.795 --> 00:17:53.655
memo to you, and I were to ask you for
advice on how to make sure that they read

00:17:53.656 --> 00:17:57.345
it and respond, what three changes would
you recommend?" And then immediately,

00:17:57.455 --> 00:17:58.515
I had this happen the other day.

00:17:58.515 --> 00:18:03.345
I was doing a voice vomit on a
post-sales call, and I had ChatGPT

00:18:03.345 --> 00:18:06.846
write a memo, and then I asked ChatGPT,
"How would you criticize this memo?"

00:18:07.095 --> 00:18:11.325
And ChatGPT said, "It's far too
long for today's busy professionals.

00:18:11.325 --> 00:18:14.556
No one's gonna read this memo." And I
said, "Would you please go ahead and

00:18:14.556 --> 00:18:17.075
shorten it to a point where you think
that people will actually read it?" You

00:18:17.075 --> 00:18:21.525
can ask ChatGPT to evaluate its own work,
and it will do so dispassionately, right?

00:18:21.806 --> 00:18:25.285
And that's the fun, but that requires
iteration and a back and forth.

00:18:26.245 --> 00:18:28.815
Matt Abrahams: In many ways, the
advice that you all are giving are

00:18:28.815 --> 00:18:33.386
advice that we give to people when
they are engaged in empathetic

00:18:33.395 --> 00:18:35.965
dialogue in human-to-human interaction.

00:18:35.965 --> 00:18:40.385
It's about curiosity, questioning,
about challenging in certain

00:18:40.385 --> 00:18:44.385
ways, and it changes the metaphor
completely, as you mentioned, Jeremy.

00:18:45.358 --> 00:18:49.697
So before we end, I like to ask
my guests a series of questions.

00:18:49.697 --> 00:18:52.717
Two are similar to everybody,
and one that's very unique.

00:18:52.977 --> 00:18:58.197
So Jeremy, for those who haven't
started with ChatGPT and large language

00:18:58.217 --> 00:19:00.068
models, where should they start?

00:19:00.808 --> 00:19:03.248
Jeremy Utley: I think for a lot of people,
if, if they say, "Where should I start?"

00:19:03.487 --> 00:19:08.798
We've talked to so many professionals who
say, "I've been meaning to try ChatGPT.

00:19:08.907 --> 00:19:13.507
I've been meaning to try generative
AI." And that to us is a shame.

00:19:13.827 --> 00:19:17.577
Every single listener to this
podcast could have at least ten

00:19:17.577 --> 00:19:19.377
hours of ChatGPT under their belt.

00:19:19.777 --> 00:19:22.588
And if you find yourself going, "Oh
man, I'm behind," well, don't worry.

00:19:22.657 --> 00:19:23.887
You can get up to speed quickly.

00:19:24.277 --> 00:19:26.058
Here's a simple place to start.

00:19:26.407 --> 00:19:29.217
This is something that every
single listener can do right now.

00:19:29.858 --> 00:19:36.057
Think of an emotional human decision
you're trying to make in your life,

00:19:36.588 --> 00:19:40.237
something that you would ordinarily
ask a partner or a friend or a

00:19:40.237 --> 00:19:43.538
spouse or a colleague about, okay?

00:19:44.077 --> 00:19:45.177
Think about what that is.

00:19:45.637 --> 00:19:50.627
Go to ChatGPT and say, "Hey, I'm trying
to make this decision. Will you please

00:19:50.707 --> 00:19:57.898
ask me three or four questions before
giving me your recommendation?" If

00:19:57.987 --> 00:20:02.618
every single listener will do that
one simple activity, they're gonna

00:20:02.618 --> 00:20:04.798
have what we call a personal epiphany.

00:20:05.614 --> 00:20:08.184
That's gonna have cascading impact.

00:20:08.264 --> 00:20:12.474
All of a sudden they're gonna start
thinking, "Could ChatGPT do this?

00:20:13.144 --> 00:20:15.273
Could I ask ChatGPT about that?" Right?

00:20:15.314 --> 00:20:19.044
But it must be emotional, it must be
deeply personal, it must be the kind

00:20:19.044 --> 00:20:21.613
of thing that you would ordinarily
ask another human being about.

00:20:22.054 --> 00:20:24.864
Get ChatGPT to ask you three or
four questions about it before

00:20:24.864 --> 00:20:28.174
giving you advice, and start
the snowball rolling that way.

00:20:28.844 --> 00:20:30.413
Matt Abrahams: Question
number two and three.

00:20:30.464 --> 00:20:34.344
Kian, I'm gonna ask you because Jeremy
has previously answered these on an

00:20:34.403 --> 00:20:38.974
earlier episode, and I encourage everybody
to listen to that episode where Jeremy

00:20:38.974 --> 00:20:40.804
and I talk about his book, Idea Flow.

00:20:41.103 --> 00:20:44.494
So Kian, who's a communicator
that you admire and why?

00:20:44.524 --> 00:20:46.383
And you can't say ChatGPT.

00:20:47.193 --> 00:20:48.164
Kian Gohar: I'm gonna
give you two answers.

00:20:48.483 --> 00:20:53.704
One is Peggy Noonan, who is a columnist
for The Wall Street Journal, and I

00:20:53.713 --> 00:20:59.423
so deeply admire her elegant prose
and her personal anecdotes that make

00:20:59.603 --> 00:21:02.853
complex topics readily understandable.

00:21:03.283 --> 00:21:09.743
She was a speechwriter for George Bush
Senior, and she developed the phrases

00:21:09.753 --> 00:21:14.683
that we became very familiar with during
his presidency, like "Kindler, gentler

00:21:14.684 --> 00:21:18.244
nation," or "Read my lips, no new taxes."

00:21:18.764 --> 00:21:25.114
And the politics and the policy aside,
the ability to translate big ideas

00:21:25.154 --> 00:21:31.433
into a few short words is just so
masterful that I always enjoy reading

00:21:31.434 --> 00:21:35.763
her articles in The Wall Street Journal,
regardless of the politics or the policy.

00:21:36.133 --> 00:21:39.614
And the second person I'd
recommend is Sam Horn.

00:21:40.063 --> 00:21:45.003
She is an author of a book called
Tongue Fu, and she is a complete

00:21:45.014 --> 00:21:51.953
master at teaching which words to
lose and which words to use to convey

00:21:51.953 --> 00:21:56.413
meaning without tripping over yourself
and creating unintended arguments.

00:21:56.753 --> 00:22:00.183
And these two are role models of
how I think about communication.

00:22:00.972 --> 00:22:01.612
Matt Abrahams: Thank you.

00:22:01.812 --> 00:22:05.832
Both of those recommendations, I
hear you talking about words and the

00:22:05.832 --> 00:22:08.032
ideas that those words bring about.

00:22:08.062 --> 00:22:11.001
Both of those are individuals
who, who have mastered that craft.

00:22:11.232 --> 00:22:13.531
Kian Gohar: Because words are
really conversations that are the

00:22:13.532 --> 00:22:17.612
seeds of innovation and how we
think about solving big problems.

00:22:17.672 --> 00:22:20.862
And if we don't get the words,
then we won't be able to actually

00:22:20.872 --> 00:22:24.072
get to the meaning and connection
of what we're trying to solve for.

00:22:24.391 --> 00:22:25.952
Matt Abrahams: Final
question for you, Kian.

00:22:26.272 --> 00:22:30.702
What are the first three ingredients that
go into a successful communication recipe?

00:22:31.342 --> 00:22:34.631
Kian Gohar: The first ingredient is
something that might seem counterfactual

00:22:34.872 --> 00:22:38.591
when we're talking about a communication
recipe, and that is listening.

00:22:39.021 --> 00:22:42.561
Listening to the environment, listening
to the context, and listening to what

00:22:42.901 --> 00:22:45.841
your counterparty might be thinking.

00:22:46.131 --> 00:22:50.411
And so trying to understand the context
by listening is super important.

00:22:50.922 --> 00:22:53.901
The second one is to know your audience.

00:22:54.361 --> 00:22:55.491
Who are you speaking to?

00:22:55.501 --> 00:22:57.282
Who's on the other side of the table?

00:22:57.291 --> 00:22:58.072
Who's in the room?

00:22:58.512 --> 00:23:01.361
What are their priorities,
and what are their goals?

00:23:01.401 --> 00:23:04.532
And so that you can think about
crafting your message in a

00:23:04.532 --> 00:23:06.232
way that resonates with them.

00:23:06.732 --> 00:23:09.871
And the third thing is
to know your end goal.

00:23:10.522 --> 00:23:14.981
How do you want the audience of
your intended communication to feel

00:23:15.281 --> 00:23:16.692
after the communication is over?

00:23:17.061 --> 00:23:21.591
Whether that's in person, whether that's
in a meeting or whether that's online.

00:23:21.831 --> 00:23:25.631
How do you want the audience
receiving the communication to feel?

00:23:26.001 --> 00:23:30.901
And then you work backwards from
that to structure the flow and the

00:23:30.971 --> 00:23:35.582
necessary ingredients to make your
communication easily relatable,

00:23:35.861 --> 00:23:37.282
understandable, and to land.

00:23:38.101 --> 00:23:41.741
Matt Abrahams: We have certainly
heard the notion of knowing your

00:23:41.741 --> 00:23:43.282
audience and being critical.

00:23:43.302 --> 00:23:47.361
I really appreciate and like
this idea of backward mapping.

00:23:47.881 --> 00:23:52.981
Start from the outcome, and then
we have to build and do and say

00:23:52.991 --> 00:23:55.161
and emote to get to that outcome.

00:23:55.491 --> 00:23:56.342
Thank you for that.

00:23:56.701 --> 00:24:00.261
And thank you both, Kian and
Jeremy, for the conversation.

00:24:00.671 --> 00:24:04.781
I find your work fascinating and
super helpful as generative AI

00:24:04.791 --> 00:24:07.671
becomes more and more commonplace.

00:24:08.451 --> 00:24:11.751
Further, I love how the concepts
of effective communication and

00:24:11.751 --> 00:24:17.011
conversation and critical thinking
can help our partnering with AI to be

00:24:17.052 --> 00:24:19.861
better, more creative problem solvers.

00:24:20.181 --> 00:24:24.541
To learn more about the results
from Kian and Jeremy's work, go to

00:24:26.802 --> 00:24:30.441
howtofixit.ai and check
out their article in HBR.

00:24:30.651 --> 00:24:31.721
Thank you so much.

00:24:32.132 --> 00:24:33.022
Jeremy Utley: Thanks for having us.

00:24:35.062 --> 00:24:37.162
Matt Abrahams: Thank you for joining
us for another episode of Think

00:24:37.162 --> 00:24:39.172
Fast, Talk Smart, the podcast.

00:24:39.652 --> 00:24:41.851
It was great learning
from Jeremy and Kian.

00:24:42.081 --> 00:24:45.912
Among the many things they shared, the
one that stands out most to me is that

00:24:45.962 --> 00:24:51.472
AI is most powerful when we treat it as
a collaborative thinking partner rather

00:24:51.472 --> 00:24:54.072
than simply a tool for finding answers.

00:24:54.591 --> 00:24:58.192
To learn more, please be sure to dive
into our library of past episodes

00:24:58.211 --> 00:24:59.942
wherever you get your podcasts.

00:25:00.221 --> 00:25:04.261
Our back catalog is full of helpful
communication tips and tools to

00:25:04.262 --> 00:25:06.402
help you be a better communicator.

00:25:06.942 --> 00:25:11.752
This episode was produced by Katherine
Reed, Ryan Campos, and me, Matt Abrahams.

00:25:12.031 --> 00:25:16.301
Our music is from Floyd Wonder, with
special thanks to Podium Podcast Company.

00:25:16.691 --> 00:25:19.911
Please find us on YouTube and
wherever you get your podcasts.

00:25:20.161 --> 00:25:22.401
Be sure to subscribe and rate us.

00:25:22.691 --> 00:25:25.151
Also, follow us on LinkedIn,
TikTok, and Instagram.