I have been a reader of this forum for some time, but the serendipity of this post inspired me to create an account. Just yesterday I read an paper titled "Erro, Ergo Sum" in the
Philosophy Pathways journal which hit on this exact subject. The author takes a more extreme stance than The Runaway Species to hypothesize that error making isn't just a necessity for creative endeavors, but the defining characteristic of a conscious mind.
Although errors vary greatly in nature and degree, this discussion will focus on the
most fundamental manners in which we might err, which will be referred to as perceptual
errors. These occur when sensory information is either ambiguous or misinterpreted, and
suffers an encoding error from which we derive an inaccurate picture of reality. The
theory proposed is a framework in which consciousness, cognition and free will may
have emerged from a single evolutionary adaptation to safeguard against these perceptual
errors.
I have been thinking on it all weekend, and it is really weird when you hear a new idea and it seems to start popping up all over the place. I watched David Eagleman's PBS series on The Brain, and now it looks like this book will move to the front of the queue when it arrives.
To your point regarding error making as an obstacle for a strong AI, I agree it is an obstacle and one of the most profound differences between current machines and human minds. It seems to go part and parcel with the enormous undercurrent of hidden motivations (evolutionary pressures, social dynamics, biological drives, ect.) that there is no obvious way to program into a computer in order to replicate or approximate human thinking.