Stimulus Ladder. Where Our Agent Loses What It Sees

The agent in this project sees a stimulus, a coloured shape, and has to remember it through a delay. Before it can remember the stimulus, the identity of the stimulus has to survive the agent’s own perception. This console asks where it survives and where it is lost.

At each stage of perception, a decoder is trained to read the shape and the colour from the agent’s internal state. Where the decoder reads them, the identity is there. Where it falls to chance, the identity is gone. The results come from the project’s research repository, and each version of the agent shown here backs a published verdict.

The four stages

• Eye map. The output of the agent’s retinotopic encoder, its first map of what it sees.
• World-model latent. The state of the agent’s recurrent world model after it takes in the eye map.
• Final capsule level. The last level of a capsule network that groups the latent into parts and wholes.
• What the workspace reads. The 256 numbers that the agent’s global workspace and its policy receive.

The task is delayed match to sample. While the sample is on screen, a decoder tries to read its shape and its colour, each one of 6, from each stage. It is trained on some trials and tested on others, 280 trials in all. A linear decoder and a stronger one, 80 principal components followed by a small neural network, are both shown. Chance is 1 in 6.

What the four versions show

• Versions 1 and 2, a discrete latent. The eye map holds the identity almost perfectly. The world-model latent holds none of it, before training and after 100 episodes of it. Every objective the project tried at this point, up to direct supervision with the true labels, left the latent at chance.
• Version 3, a continuous latent. Replacing the discrete latent with a continuous one of the same shape lets the identity through, with no objective that asks for it. The loss moves up the ladder, to the final capsule level. Chapter 2 shows exactly where, the last routing step.
• Version 4, the workspace reads every capsule level. For the first time, the identity reaches what the workspace reads.
• It did not learn the task. Chapter 3 shows the task reward. It stays flat in every version. Making the identity available did not by itself make the agent use it. That is a separate problem, of learning across the delay, and it is still open.

What the console shows and does not show

• Decodable is not used. A decoder reading the identity shows that the information is there. It does not show that the agent uses it.
• One seed for version 4. The continuous latent was repeated on three seeds. The capsule fix is one seed, and it is off by default in the code.
• The rewards are not comparable across versions. The reward scales differ from run to run, so each reward line is on its own scale. Compare the start and the end of one line, not two lines.
• The example stimulus is a drawing. It fades by how far the decoders sit above chance. It is not a recorded trial.

Why this belongs on this site

This project tests whether consciousness can emerge in a built system, and a global workspace can only broadcast what reaches it. For most of the project’s history, the identity of what the agent saw did not reach its workspace. This console shows that finding as the project measured it, including the part that is still not solved.

Sources

• Verdict, the discrete latent before and after training: collapse_locus_trained_2026_06_21
• Verdict, the continuous latent, the capsule levels and the capsule fix: b1_continuous_latent_2026_07
• The earlier first measurement: perception_decodability_2026_06_09