Theory Arena. Do Consciousness Theories Choose Their Own Evidence?

Four families of theories compete to explain consciousness. Global workspace theories say an experience is information broadcast across the brain. Integrated information theories tie it to how much a system’s causes are integrated. Higher order theories need the brain to represent its own states. First order and predictive processing theories place it in the sensory areas themselves, through local recurrent activity or prediction.

Itay Yaron, Lucia Melloni, Michael Pitts and Liad Mudrik built the ConTraSt database to ask how these theories are tested. They coded experiments that read their result in light of one of the four families, and found that the theory an experiment supported could be predicted from its methods alone. This console replays their open data, now over 500 experiments, and runs that test again.

What the console shows

• Chapter 1, Arena. The literature from 2001 on, one year at a time. Each experiment drops into the column of every family it was read as supporting, above the line, or challenging, below it. An experiment read against two families appears twice. The buttons show only experiments designed to test a theory, only those interpreted after the fact, or only those that mention a theory.
• Chapter 2, Methods. For each family, how often its supporting experiments used a method: no report, a change of state such as sleep or anaesthesia rather than a change of content, and each recording technique. The shares form as the years play.
• Chapter 3, Guess the theory. A naive Bayes classifier sees only the methods of an experiment and guesses which family it supported. Each experiment is guessed by a model fitted on all the others.

What this replay finds

The numbers in this section come from the console’s data file, the ConTraSt snapshot of the build date.

• The methods give the theory away. Among the some experiments that support one family and challenge none, the guess from the methods alone reached a balanced accuracy of this value, where chance is this value. None of 1000 guesses on shuffled labels did as well.
• Not only through each theory’s own measures. Some measures belong to one theory, such as phi for integrated information. With every measure left out, so that only the technique, the paradigm, the task, the sample and the design remain, the balanced accuracy was still this value.
• Higher order theories are left out of the guess. Only a few experiments support them alone, too few to learn from.
• Experiments designed to test a theory mostly support it. Of the some experiments a global workspace theory drove, some were read as supporting it and some as challenging it. For integrated information the counts were some and some of some, and for first order theories some and some of some.

A guess that works means each family is tested with its own kind of experiment. Sleep and complexity measures go with integrated information. Oddball tasks and masking go with the global workspace. Figure and ground, and brain stimulation with TMS, go with first order theories. When each theory is tested on its own ground, its supporters rarely meet a result that could refute it. This is the problem that adversarial collaborations such as COGITATE were set up to fix, with two theories committing in advance to opposite predictions for the same experiment.

What the console shows and does not show

• The coding is ConTraSt’s. Whether an experiment supports or challenges a family is how its authors read their result, as coded by the database team. This console does not judge the experiments.
• The guess is simpler than the paper’s. The paper used its own analysis. This console uses a naive Bayes on presence and absence of each method that at least 8 experiments share, with leave one out and 1000 label permutations, so its numbers are this site’s.
• A snapshot. The database grows. The build takes a new snapshot when it runs again.

Why this belongs on this site

This project studies consciousness as an emergent property of physical systems, and it borrows from more than one of these families. Its agent has a global workspace that broadcasts what wins a competition, and the project has instruments for phi and for perturbational complexity, which come from the integrated information side. The lesson of this database applies to the project too. A measure chosen because a theory favours it will tend to favour that theory, so the project’s own tests need predictions that could fail. The sister consoles show some of those measures on real brains, the brain’s echo to a pulse and sleep and dreams.

Sources

• Yaron, I., Melloni, L., Pitts, M. and Mudrik, L. (2022). The ConTraSt database for analysing and comparing empirical studies of consciousness theories. Nature Human Behaviour 6, 593 to 604. doi:10.1038/s41562-021-01284-5
• Data: the ConTraSt database, contrastdb.tau.ac.il, licence CC BY-SA 4.0. The file this console loads, arena.json, is derived from it and shared under the same licence.