Is perception not under any cognitive control?
Higher cognition (e.g., motivations, desires, linguistic representations) has ostensibly been demonstrated to exert top-down influences on what we perceive. The authors of this paper refute this claim (of which hundreds of papers back up), citing that alternative explanations are possible. Here I will dissect whether their listed pitfalls adequately substantiate the claim that there is no current evidence for top-down effects of cognition on visual perception.
This is a particularly rich paper because of the vastness of examples the authors provide, and it has prompted rebuttals and comments from various authors, all of which are included in the current version of the article along with the author comments.
Just to lay out what top-down factors influencing perception may entail, here are two examples. Note that the authors define “top-down” as a sort of cognitive penetrability – it is not enough that input is changed (e.g., closing eyes changes perception), the process of input to output has to be changed.
- After completing a racial categorization task, Levin & Banaji (2006) reported that the perceived lightness of faces was altered.
- A grayscale banana appears tinged with yellow (Hansen et al., 2006).
This idea of cognition on perception is personally a huge theoretical question for me, and I will state that I went into reading this article with a bias contrary to the authors’ claims. For instance, if cognition does not influence perception, then it seems neuroimaging would be mostly irrelevant to investigations of higher-level cognition on perception (such explorations would only shed light on how input gets altered). And I personally (before reading this article) found the idea that the brain is a black box in which cognition is independent to visual output as strangely obstinate. I will try my best to objectively assess this article, however. Most content in the following sections are simply summarizing the article (with a few personal comments thrown in at times). The last section, however, reflects my personal concluding opinions.
Optical Illusions
As an example of how cognition cannot influence perception, consider most optical illusions:
No matter how well we understand the processes underlying visual perception, we will always fall for these illusions (e.g., the above left column will always look a lighter shade of gray than the right column). My desire to make the two columns appear the same shade of gray is completely irrelevant. Vision being “smart” in the sense of distorted output (e.g., context effects), does not imply cognitive penetrability.
The authors are claiming that all of visual perception is akin to these optical illusions: no matter the internal desire, motivation, or linguistic representation of the input, the percept will remain the same.
Comments:
- What about optical illusions that you can control to an extent? For instance, the white/gold or blue/black dress illusion, or the clockwise or counterclockwise spinning ballerina illusion? The authors relate that this may simply reflect where one is attending on an object (i.e., necker cube), but this is less obvious in the aforementioned illusions.
Re-entrant Processing: Irrelevant?
In section 2.2, the authors mention that it is well-established that lower-level regions may be modulated by higher-level regions. However, they state this as irrelevant for cognitive penetrability.
It is almost never clear which mental process a descending pathway is descending to (or if that descending pathway is influencing the input or the processing of whatever it descends to).
The reasoning is essentially that these re-entrant processes are often ambiguous and occur “whether you like it or not” (the same reasoning was used against cross-modal effects).
Comments:
- Is the same output occurring “whether you like it or not” a suitable litmus test for cognitive penetrability? At first I thought that our past experiences may alter neural processing and lead to changes in perception. While occurring whether we like it or not, it is still something we have control over. But now I believe this line of argument doesn’t hold much weight – as the authors explain, experience may simply reflect increased sensitivity to certain visual input. The question then is, if knowledge interacts with perception before input is received, could this be interpreted as cognitive penetrability? The answer is no according to the definition of the authors, but I could see the definition itself being somewhat fuzzy.
Pitfalls
El Greco Fallacy
The authors state that many of the described studies succumb from an overly confirmatory research strategy, where they fail to test whether the observed effect fails to occur in certain cases. They explain this via the “El Greco fallacy”, which I think is a great way of thinking. It is an art-historical reasoning error where it was thought that El Greco’s painting appeared “stretched” because he perceived the world as stretched. The problem is that his canvas should have equally been stretched out, canceling out the distortions.
For instance, in a study claiming that reflecting on unethical actions makes the world appear darker, the original effect was obtained using a numerical scale to rate the brightness of the room. They replicated this with gray color patches and the effect held up. The problem is that the gray patches themselves should have been perceived as darker as well, cancelling out the effect. The fact that the effect did not cancel out points to alternative explanations.
Perception vs. Judgment
Another pitfall the authors mention is that perception effects are not always considered against simple judgment effects. I.e., does someone literally perceive the world differently or are they just responding in a way that makes it seem like they perceive the world differently. Put more concretely, one study reported that increased throwing effort made a dart board appear farther away. However, when researchers told participants that the darts were flawed, participants no longer demonstrated the same effect. Perhaps the effect of effort on distance judgments was more to do with how subjects thought about the distance than how it truly looked. Thinking that the darts were flawed maybe led participants to attribute bad performance to faulty darts rather than distance from the dartboard.
Another example was how conservatives ranked darker photos of Obama as more representative than liberals who showed the opposite trend. If they showed Obama with a red horns or a halo, the effect would probably persist. But that does not mean the participants literally see Obama as having horns or a halo.
They state it would be better if these studies used performance-based measures in which subjects success tied directly to how well they perceive the stimuli (e.g., visual search task).
Demand and Response Bias (Embodied Cognition)
The authors state that the social nature of psychology experiments may lead to reports being contaminated by task demands. For instance, one famous study reported that wearing a heavy backpack altered slope judgments (making hills look steeper). Perhaps subjects modified their responses to suit experimental circumstances, especially given the somewhat obvious expt. manipulation. When experimenters gave participants a valid excuse for wearing the backpack (to hold monitoring equipment during climbing), the effect disappeared.
Experimenters should use careful debriefing to ask the subject what they thought the experiment purpose was, or whether they used any strategy. They can also employ nontransparent experimental manipulations.
Low-level Differences
One problem with changing the stimuli employed and using this as an experimental manipulation for top-down effects is that it may be confounded by low-level differences being actually responsible for perceptual differences across conditions.
For instance, one paper claimed that African-American faces look darker than Caucasian faces, even when matched for mean luminance. This paper was widely regarded because the difference in perceived lightness is immediately apparent when looking at the images. The problem is that mean luminance does not account for visual cues for lighting (e.g., consider the optical illusion above). The black face appears to be under illumination whereas the white face does not. The black face also has a darker jawline and the white face has darker eyes. One therefore cannot conclude it is race that is inducing differences in perception. In fact, when blurring the faces (eliminating race information), the originally black face still appears darker. This all convinced me that the original race study was flawed.
Peripheral Attention Effects
The authors state that attention is similar to closing ones eyes. Sure it is under volitional control, but it changes the input for perceptual processing. They state that top-down effects must operate not solely through attention, and that many studies fail to consider how attention may be focused differently depending on the task (wherein nontransparent manipulations can test whether attention is actually the reason for effects). Attention should be manipulated directly or attentional load should be increased to examine whether peripheral attentional explanations explain the observed effect.
Comments:
- If attention can (for example) lead objects to perceptually appear larger, then doesn’t that practically mean that we have conscious control over perception? Another example, if playing scary music can change our interpretation of an ambiguous figure, then doesn’t that mean our cognition influences perception (for instance, if we assume it takes experience/knowledge to interpret that music as being scary).
Memory & Recognition
Top-down effects on perception cannot be studied by how we recognize stimuli (as perception is what we see not what we remember). E.g., assigning linguistic labels to simple shapes improves RT in recognition tasks and, when briefly presented, morally relevant words are easier to identify that morally irrelevant words. These are not demonstrations of cognitive penetrability. A recognition task involves memory retrieval, and of course memory retrieval may be sped up if given a retrieval cue such as linguistic labelling (e.g., if you know that a fuzzy image is labeled as a kangaroo, you will have an easier/faster time perceiving what animal the image is). In the moral words study, the morally relevant words could have primed each other whereas the morally irrelevant words were completely irrelevant to each other (i.e., semantic priming). This is not improvement in visual processing but improvement in memory retrieval. When replicated with fashion terms instead of morally relevant terms, the effect persisted (and it seems ridiculous to conclude that fashion words are inherently easier to see).
Some Condensed Counterarguments (from Open Peer Commentary)
Gary Lupyan: Firestone and Scholl assume that attention (or memory, or recognition processes) changes input to a putatively modular visual system. Perception cannot be assumed to be a modular, independent system.
- Authors: Attention may be a genuine alternative explanation just as long as attention sometimes changes input to later visual processing, because then such attentional effects must be actively ruled out by careful experimental tests of the sorts sketched in our article.
Steve Most: Failures to see what is right in front of us when our attention is otherwise occupied are examples of cognition penetrating perception. Attention guides perception at multiple levels of processing, and it is likely incorrect to relegate its role only to gating of input into early vision.
-
Authors: You are studying selective attention, an input-level effect that does not violate the encapsulation of seeing from thinking. If we assume that attention may guide perception at multiple levels of processing, that also means that it does, to some extent, guide early perceptual input and that is what needs to be empirically tested. You say categorization of a stimulus can change the likelihood that we will see it in the first place, but that study manipulated categorization by altering orientation, a low-level difference that can really matter. Better to use a truly ambiguous stimulus (e.g., B vs. 13).
-
Me: One of the points that Steve Most mentioned was wanting better fences separating attention and perception. We need “a better map of where they think that fence stands”. I do not think F&S adequately outline these fences in their responses. That said, how does one outline such neighborly systems?
Howe & Carter: Perception is obviously affected by top-down cognition. People actually see the mental images as opposed to simply being made aware of them.
- Authors: Don’t agree with that last sentence. And even if this were the case, so what? Even if mental imagery and perception share characteristics, they differ in vividness, speed, effortfulness, etc. These differences allow for distinguishing imagery from perception.
Witzel, Olkkonen, & Gegenfurtner: Remembered color of an object alters its perceived color, such that subjects who must set a banana’s color to be gray in fact make it a bit blue (Hansen et al., 2006).
-
Authors: Even if gray bananas truly look yellow, that does not imply cognitive impenetrability: it could be a form of perceptual learning (thus altering visual sensitivity during input). Also note that this study fails to work as a subjectively appreciable demo despite huge effect sizes. A gray banana on its own does not look yellow. Perhaps a response bias is involved: subjects task was to adjust banana’s color to look gray, so it is possible subjects are overcorrecting. They see a banana and make sure that all of yellow, specifically, is gone. They pay less attention to reducing blueness, and thus a blue banana is left. It is also possible that the gray standard subjects have in mind changes dependent on object identity, rather than perceived color of the object. Further, they suspect that after adjusting a banana to look gray (but in fact be blue), subjects who see this blueish banana next to an objectively gray patch, or alternatively an objectively blue patch that matches the banana blue, will lead subjects to realize that the banana is the same color as the blue patch and not the gray patch.
-
Me: How is it possible that gray bananas looking yellow does not imply cognitive impenetrability? A grayscale image appearing as not gray due to knowledge of the object (and not due to things like attentional demands or visual lighting cues) should be conclusive! Why does it matter if knowing that the image is of a banana leads to altered visual sensitivity to certain color frequencies? This harken back to my complaint that altered input may still signal cognitive penetrability. That said, I agree with their concern about response bias and with their proposed additional experiment, but not with the first sentence of their response.
Suggestions and Overall Comments
Firestone and Scholl repeatedly state that knowing what perception is and how it is connected to the rest of the mind has deep theoretical foundations and motivations, but they never practically explain what these deep motivations for understanding are. I agree with their statement, but I do not think they explained these motivations clearly.
I applaud the Open Peer Commentary section and the repeated empirical tests of prior studies using their six pitfalls. I also like the writing style for the article – very enjoyable to read.
I love whac-a-mole as much as the next visual scientist, and it is a good analogy for section R4, but it seems to be a bit in poor taste to attach a photo of the authors enjoying a game of whac-a-mole given that this is the same section where they address the concerns of frustrated academics whose research has supposedly been invalidated.
I suppose my major complaint involves the refusal to acknowledge altered perceptual input as indicative of cognitive penetrability, such as in cases of attention. Note this is perhaps a theoretical misunderstanding, and I may be wrong due to lack of knowledge. It is clear that altered perceptual input is not always akin to the act of closing or moving ones eyes. The authors agree, such as in cases of feature-based attention, but they say this is analogous to seeing through tinted lenses – attention is a selective process that modulates early perceptual filters. My argument is that perceptual filtering, if specific enough, is analogous to cognition affecting perception. To give an extreme example, if you are able to consciously adopt an attentional filter that makes every object you see look like a horse by altering sensitivity to specific visual input, then damn I think you have control over your perception! I know this is a ridiculous overextension, but sometimes it is useful to take something to the extremes to see if conclusions intuitively hold. I am not sure that attention is “unquestionably perceptual” and not cognitive, as cognition (e.g., task demands) may influence attentional processes that thereby influence perceptual input.
Overall, I agree heavily with the article. It outlines very well the arguments against cognition influencing perception. Despite my preconceived notions and the vast array of literature ostensibly supporting cognitive penetrability, I do side with the claims made by Firestone and Scholl.