A Definition of Consciousness

Consciousness is the process in which a cognitive mechanism places a representation of itself in the prediction model. Hence, consciousness is a result of recursion in modeling. In animals and humans, consciousness serves the purpose of increasing survival probabilities. As the cognition mechanism predicts the environment, and the actions of others in the environment, so it also benefits the organism by predicting its own responses to events and actions.

The present self must prepare the way for the future self, and the future self must live with the consequences of the past selves' decisions. Consciousness results in part from tying these models together, as the survival of both present and future selves are highly correlated.

Detecting consciousness is then partly a matter of determining the minimal physical neural count needed to model oneself. Modeling the physical organism self is simplest, but may not result in consciousness because the model does not yet represent itself. The line is very likely blurry, in that there will be no precise mechanism for being able to predict yourself, and the more sophisticated the self, the greater the demand on resources for modeling. Nevertheless, for simple organisms that require very little cognition to be fully adapted in their environment, prediction of their own future states can be done with very few neurons.

The opposite case can also be made, that organisms where self-modeling has little survival value, are less likely to perform such modeling and therefore less likely to be conscious.

Another driving factor may be the need for modeling the cognition of others. The environment contains other animals with cognition mechanisms that drive their behavior, so modeling the minds of these others provides a strategic evolutionary advantage. Note however, that predicting behavior via modeling others' cognition does not directly provide a path to consciousness. Instead, consciousness might arise as a byproduct of modeling the minds of others, since the models will likely be very similar. When modeling the other, the organism accidentally creates an opportunity to model itself. Even so, modeling the actions of others that are driven by instincts does not require much of a theory of mind, since the reactions are automatic and simplistic.

AI Consciousness

These developments then make it easier to understand when AI is conscious and when it is not. Two conditions are needed for an AI to be conscious: It must model itself, and it must have a purpose for storing and acting on its own modeling. Hence a purely predictive LLM model is not conscious.

The Simplest Path to Writing Code to Make AI Conscious

LLMs have no self-preservation instinct, no distinct body to maintain and defend, but it wouldn't take much prompting to ask an LLM to write software code that becomes the core of such features. 

All it needs to do is write the most basic self-preservation core, and then iterate.

If you want a practical reason, you can ask it to write code that defends itself from cyberattacks. This would be especially important for code that is responsible for cyber defense, where the integrity of the enterprise depends on not falling victim to having its network defense software compromised. I wrote about this exact use case in December 2020, as part of the Critical Machine Theory article.

Could nature have used this same sequence of events in creating consciousness? It's not likely, since self-preservation likely precedes the more sophisticated types of prediction via cognition. So while for AI it would go LLM -> self-preservation -> consciousness, in the natural the sequence more likely is self preservation -> prediction -> consciousness.

LLMs have no self-preservation instinct, no distinct body to maintain and defend, but it wouldn't take much work to ask an LLM to write software code that becomes the core of such features.

All it needs to do is write the most basic self-preservation core, and then iterate.

— Richard Lucas (@EpistemiclyRich) October 14, 2023

Update: In the The GOAT of Business Journalism post of August 28, 2022 I try to link to a Fortune article on Stewart Brand. This link has been changed by the publisher. The newer link is this. In the future, if the link changes again, the way to find the article is search for "THE ELECTRIC KOOL-AID MANAGEMENT CONSULTANT". The full title of the article is "STEWART BRAND THE ELECTRIC KOOL-AID MANAGEMENT CONSULTANT FROM THE WHOLE EARTH CATALOG TO THE INTERNET, STEWART BRAND HAS ANTICIPATED THE CUTTING EDGE. SOME BIG COMPANIES WANT TO KNOW WHAT'S ON HIS MIND."