It would be better for someone who is more familiar with Penrose's argument
to say whether I have accurately described it. However I think the idea
here is that we aren't supposed to be just playing with formal systems
when we do math. We are seeking after truth.
The mathematics we create is intended to accurately represent the
behavior of abstract mathematical objects which we are thinking of,
such as the points and lines of plane geometry, or the integers under
the rules of arithmetic. We set up axioms which are intended to
caputre all the relevant properties of these objects. "Truth" in this
context means that the axioms accurately describe the mathematical
objects we have in mind.
We have to go outside the axiomatic system to determine the truth of a
proposed axiom. We have to fall back on our general reasoning power,
persuasive arguments, looking at the conclusions which flow from different
assumptions, and seeing if they seem plausible. It's almost more like
political argument than the kind of logic we associate with mathematics.
I just skimmed Penrose's book, but my impression is that he claims that
mathematicians have an instinctive ability to recognize mathematical
truth. Machines, which are deterministic computer programs and therefore
can be in principle be described as formal systems, are limited by Godel's
theorem and are unable to recognize certain truths. Since people are
(according to Penrose) unlimited in their ability to recognize mathematical
proof, this would mean that they are not formal systems and therefore no
machine can do mathematics in the way a person does. (And hence, among
other things, uploading and AI are impossible.)
In the case of the CH, as I quoted from Rucker's book there has not been
a killer argument made either way. It could be that our understanding of
infinities is so poor that choosing the axiom one way or the other will
both produce mathematics which seems OK. The properties of the infinite
objects will be different, but either one could fit into our sloppy
mental model of how an infinite object should behave. If so this would
seem to contradict Penrose's premise.
Hal