Long before modern AI systems existed, one of the greatest minds of the 20th century defined the limits of intelligence. Dr. Craig A. Kaplan, an AI pioneer and creator of Open Democratic Frameworks for Safe Superintelligence, shared insights from his time collaborating with Herbert A. Simon during his PhD program at Carnegie Mellon University: “Reason is wholly instrumental. It cannot tell us where to go. At best, it can tell us how to get there.” No level of intelligence can determine what is right or wrong. Values do not come from computation. They have to be given, guided by human judgment and values. And as AI becomes more powerful, that responsibility only grows clearer. Watch part three of our conversation now.
0:00
You’ve worked with one of the greatest
0:01
[music] minds of the 20th century. How
0:03
did that shape your vision of
0:05
intelligence? And so [music] these would
0:07
be the lessons from Dr. Herbert A.
0:10
Simman. And I’ I’d just love [music] to
0:12
get your take on this and your
0:14
understanding of human intelligence and
0:16
just those early lessons.
0:18
Sure. Yeah, that was a really fun period
0:20
of my life. So I was the young graduate
0:22
student. I think I was 21 years old and
0:25
uh went to Carnegie Melon to be in the
0:27
PhD program and Herbert Simon was almost
0:29
80\. He was 68 69. He’d already won his
0:32
Nobel Prize. Um he was one of those 11
0:36
pioneering scientists that in 1956 which
0:39
was before I was born he actually named
0:41
the field and he had built some of the
0:43
very first AI systems. Really smart guy,
0:45
loved science and we just hit it off. I
0:48
mean I love science too. who I didn’t
0:50
know hardly anything and I think he was
0:52
just happy to see such an enthusiastic
0:54
person that wanted to learn you know and
0:56
we would we met every week it was
0:59
amazing for four years if he was in town
1:01
we met and it was great so I’ll tell you
1:03
one thing that he wrote that I think is
1:07
really critical for the future of AI and
1:09
the things that we’ve been talking about
1:10
so back in 1981 he wrote a little book
1:12
and I love little books because I figure
1:14
if you can’t say it in a little book
1:16
then maybe you’re not thinking clearly
1:17
enough
1:18
what is your definition of a little book
1:20
just so we’re clear,
1:21
you know, 100 pages or less, right? one
1:24
really small ones
1:26
and it was called uh the name of the
1:28
book reason in human affairs and um in
1:31
it he had this line he said reason he
1:34
was talking about reason and he he sort
1:35
of said look I’m not saying things that
1:37
other people haven’t already said but a
1:39
lot of people seem to have forgotten
1:40
these so I want to repeat them and he
1:43
wrote that reason is wholly instrumental
1:47
it cannot tell us where to go at best it
1:50
can tell us how to get there and so what
1:52
he meant was you can have a logical
1:54
system like the way I apply it to AI.
1:56
You can think of AI as logical. It’s a
1:58
logical reasoning system. And it could
2:00
be in the future it will be trillions of
2:02
times smarter than me or you or pretty
2:05
much any human. And so the worry which
2:07
we talked about at the beginning is that
2:10
what if it comes up with values that say
2:11
we don’t care about the humans or
2:13
whatever, right? It’s so much more
2:14
powerful and everything.
2:15
Yeah. Right there, Herb was saying in
2:18
that little book that no matter how
2:20
smart an intelligence gets, it can’t use
2:23
logic to determine what’s right and
2:25
what’s wrong. What’s right and what’s
2:26
wrong has to be given. It has to come
2:28
from someplace else, like the human
2:30
heart or human value systems. Humans get
2:32
it from maybe their parents or their
2:34
peers. We don’t sit down and rationally
2:36
figure out this is right or what this is
2:37
wrong. Once we know what it is that we
2:39
care about, we value human life or we
2:41
value treating people fairly. Once we
2:44
have that value, then the logic can tell
2:46
us a better way to do it or a faster way
2:48
to do it or a more efficient way and the
2:51
machines will be really good at that,
2:52
way better than us. But are they going
2:54
to be better at coming up with that core
2:55
value? Herb said no, it’s impossible for
2:58
them to logically that.
3:00
Yeah.
3:01
So you had asked what’s the role of
3:02
humans in the future? And I think that
3:04
is the role of humans in the future. I
3:06
think that is right now we we pride
3:09
ourselves on being the most intelligent
3:11
species. Although maybe dolphins are
3:12
smarter, but we think we are and we
3:15
really make a big deal about our brains
3:17
and all of our thoughts and everything.
3:19
But I think in the future AI will be so
3:21
much better than us.
3:29
[music]
