CS319 - Cognitive Modeling, Questions and Answers

These are questions that students had about cognitive science in a previous version of this course, together with my answers (provided on 10-26-99).

Some of these answers are a matter of opinion rather than fact, so keep in mind that all answers reflect my own biases.

• What is one implemented cognitive model that has generally been accepted by cognitive scientists? Such a model might show us how the concepts we have covered might actually work.
  • There is a scientific journal, called Cognitive Science, that has been around since 1977. Pretty much every article presented in this journal represents a "cognitive model that has generally been accepted by cognitive scientists". We will cover some of these models soon, I promise! But even the work we have looked at so far is respected, we just haven't been able to go into as much detail as many of you might like. Simon produced much of the foundational work in the field (as well as more recent research), even including models of how people solve the "Tower of Hanoi" :-). Anderson's enormous amount of work with the ACT systems is also well respected. And Johnson-Laird is one of the creators of the notion of "mental models", which many cognitive scientists have incorporated into their work.
• Why are there so many hypotheses in cognitive science, but so few that are accepted as "correct"?
• Are cognitive scientists really getting anywhere in determining what is going on inside a human's mind to produce "thinking"?
• Are there any incontrovertible statements that cognitive scientists can make?
  • I am definitely partly to blame for the fact that you folks are asking these questions. If it seems like we just don't know much about cognitive science, that's partly because I am being very careful in class not to make any claims of the nature "this system is right and that system is wrong". Such claims would be scientifically imprudent. But the fact is that there are a number of things we know about cognition with a high degree of certainty. One of the lessons I would like to get across is that the various approaches to cognitive science that we look at are all actually quite similar in many ways, and build upon a large body of accepted facts. Simon hinted at some of these facts in the paper we read:
    • We know that people process symbols, and we know much about how they process symbols.
    • We know people think and reason by using selective, heuristic search plus recognition of familiar cues.
    • We know how people solve problems and create problem-solving strategies.
    • We know how people learn language.
    • We know how scientists discover new things.
    • There exist computer programs that do all of these things in psychologically plausible ways.
    There are many other psychological regularities that have been identified with the help of cognitive science and cognitive models. As some further examples, Kurt VanLehn (1989) concisely summarizes what we know about how people solve problems and learn to solve problems:
    • People reduce their verbalizations of task rules as they become more experienced with practice.
    • Improvement occurs quickly in knowledge-lean domains.
    • There is a power-law relationship between the speed of performance on perceptual-motor skills (and possibly problem-solving skills) and the number of practice trials.
    • Problem isomorphs do not become more difficult simply by changing surface features of the problems.
    • Other representation changes can make problem isomorphs substantially more difficult.
    • There is asymmetric transfer between tasks when one task subsumes another.
    • Negative transfer is rare.
    • "Set" effects can lead to negative transfer.
    • Spontaneous noticing of a potential analogy is rare.
    • Spontaneous noticing is based on superficial features.
    Again, there are many existing computer programs that explain these facts about human problem solving and learning.
• We've talked about many types of cognitive processes (such as deduction, using production rules, manipulating frames), but how would they actually get implemented in a computer system?
  • We will spend much of the rest of the semester looking at case studies that will hopefully answer this question in part. We have already looked at one way of using frames, and one way to use production rules. Hopefully these projects give you some clues about how we translate cognitive skills into computer programs. The answers to the following to questions may also approach an answer to this question.
• What areas besides cognitive architecture can we explore?
• What experimental evidence do we have to support or reject the models we've looked at?
• What models have been developed based on experiments on human behavior or research in neuroscience? What is the exact connection between the data and the model implementations?
  • There are vast numbers of computer models that have been inspired by and/or matched against experimental data from human behavior. Here is a summary of some of the examples I hope to cover by the end of the course:
    • People make particular types of errors when solving subtraction problems.
    • People can retrieve and successfully use analogies in certain types of situations, but not others.
    • People learn and store concepts in memory in ways that lead to specific types of errors and timing results.
    • College students learning physics rely heavily on the use of analogy, but perform better when they can actually explain the subject matter to themselves.
    • Children independently invent particular types of problem-solving strategies
    • There is a specific power-law relationship between performance and practice on many skills.
    These are facts uncovered by psychological studies, and we will examine computer programs that explain why these facts exist.
• How are cognitive architectures implemented?
  • Cognitive architectures are implemented in a computer language, just like any other computer program. They are implemented in a variety of languages, such as Lisp, C, and Prolog. Some architectures are implemented to a level of detail that it is useful to consider the architecture itself to be a programming language (where the "program" might be a set of rules, or a semantic network, for example). What sets cognitive architectures apart from other computer programs is that they normally have conceptual components representing sensors, short-term memory, long-term memory, and motor systems. In addition, they usually have built-in mechanisms for representing propositions, producing propositions, perceiving the world, acting in the world, and learning. Finally, any cognitive architecture comes with an implicit argument for how the behavior of humans (or other intelligent creatures) maps onto the behavior of specific models implemented within the architecture.
• Can you make more clear how we would distinguish particular memory representations using experiments?
  • There is a large body of research in psychology that studies just this problem. We have lightly touched on just a bit of it. Much of this research involves timing how long it takes people to perform certain tasks (tasks that we assume involve retrieval from memory). These tasks usually take on the order of a couple hundred milliseconds, and we attempt to distinguish between alternative representations by making predictions about retrieval times. It is important to remember, however, that when we talk about "memory representation", we are talking about a symbol-level description of how knowledge is stored in memory. When doing experiments on people, we are measuring a physical implementation of memory. It is very possible that there is more than one symbol-level description that would adequately describe our physical memories, so it may be impossible to distinguish between some types of representations.
• What is the point of intelligent computer systems? Are they meant to model the mind or to perform a particular task or both?
  • That depends on who you ask. Most cognitive scientists build systems specifically to model the mind or particular functions of the mind. Many AI scientists are interested in building systems that perform tasks intelligently, regardless of how they model the mind. Some people assume that most systems they build could well serve both purposes. In this class, I wish to focus for the most part on systems that are specifically intended to model the mind. You'll sometimes hear me call such systems "psychologically plausible".
• How does heuristic search guarantee or produce a "good enough" answer?
  • Heuristic search doesn't necessarily guarantee a "good enough" answer. Whether you can make any guarantees depends on the type of problem you are working on. In general, the idea of heuristic search is that it will try (on average) to give you a "good enough" (on average) answer. However, it may fail to find a good answer, or it may fail to find any answer at all, depending on the task and the specific type of heuristic search being used.
• What is the difference between introspectionism and behaviorism?
  • Introspectionism is the approach to psychology where we collect data about what people are thinking simply by asking them what they are thinking. There is much criticism of this approach, particularly because it does not seem like a very objective type of experimentation. However, it turns out that there are a variety of cognitive tasks for which introspection appears to be an entirely adequate way of collecting data (and perhaps the best way we currently have available, until there are vast improvements in our ability to image the brain and interpret those images).
  • Behaviorism is the branch of psychology that attempts to go to the other extreme in terms of experimentation, and ensure that all psychological experiments are completely objective and repeatable. This is a laudable goal that all scientists should strive for. In practice, however, such an approach limits the types of questions we can currently ask, because much data about how people think is inherently subjective (at least for now).
• What are frames?
  • Frames are a particular type of knowledge representation. That is, they are one symbol-level way of describing how knowledge might be stored in the mind. In a frame representation, every concept labels a "frame", and the frame contains a number of attributes, each with a value. As we discussed in class, frames are very similar to "schemas" and "scripts" (although some people might claim there are important differences).
• Can you explain the "systems response" to the Chinese Room argument?
  • Searle claims that the man in the Chinese Room clearly does not understand Chinese. By analogy (an incorrectly mapped analogy, in my opinion), he argues that a computer program would also not understand Chinese. The "systems" response is that a computer program can't do anything unless there is something running the computer program. That might be a computer, or it might be a brain. In any event, only the combination (the entire system) of the program and computer would understand Chinese. By analogy, although the man in the room doesn't understand Chinese, and the rule book doesn't understand Chinese, the combination of the two does "understand" Chinese.
• How do symbols get processed?
  • Symbols get processed by symbol processors :-). Seriously, though, you can think of a symbol as any data structure that you might store in a computer program. You write a program to manipulate the data structure. We say that program processes the symbols that represent whatever it is you are trying to compute. The "classical view" of cognitive science simply assumes that minds process symbols in the same way that computer programs process symbols. The only differences are at the physical level, where one implements symbols as patterns of firing neurons, and the other represents them as patterns of binary digits.
• How and where do production rules get stored?
  • There are many different ways to interpret this questions. In a computer program, production rules get stored in "long-term memory" as part of the "program" that is going to run in order to "think". You can imagine writing a computer program that consists of hundreds of if-then statements inside a "while" loop. The program simply runs forever, executing whichever if-then statements happen to be satisfied. You can consider each if-then statement to be a production rule. Modern production-rule systems actually store the rules in a more complex data structure, so that they can match and execute more efficiently. Inside the human mind, there are no "real" production rules. Remember that productions rules are a symbol-level description of something that goes on inside the mind. Production rules represent the processes of firing neurons that chain together in order to produce "symbols" (which also might not "really" exist) and to generate behavior.
• Is it safe to say that production rules do the work of a semantic network?
  • Yes and no. A collection of productions rules can be viewed as if it were a semantic network. Each rule links one set of concepts to another set of concepts. However, production rules are very content specific. Each individual rule only fires when its precise conditions are met. In contrast, the general process of spreading activation is very content independent. Spreading activation will spread through links and nodes in a network with no care at all about what each of those nodes and links actually stand for. In a way, this makes the spreading activation theory more attractive (if it works).
• Would it be useful to view a semantic network as actually being like a network of neurons? Or is it really more useful as an abstraction?
  • It depends on who you ask. For the most part, semantic networks are considered not to be much like networks of neurons at all; they are really just an abstraction. Connectionist networks (also sometimes called "neural networks") look a lot like semantic networks in many ways, but they also are usually viewed as abstractions. There are some researchers, however, who are really trying to model behavior at the level of individual neurons (or at least small collections of neurons), and for them the networks are much closer to a direct physical mapping.
• Can you explain more about the different levels of cognitive analysis? How does a system implement things at the knowledge level? How can you leave levels out?
  • Remember that the different levels of analysis are just different ways of looking at the same thing. Depending on what you care about, it usually makes sense to use different levels of description at different times. As an analogy, biologists look at life with one level of description, chemists at another level, quantum physicists at another level, and priests at another level. Similarly, you might say that psychologists are concerned with looking at the knowledge level of behavior, cognitive scientists look at symbol-level descriptions, and computer scientists and neuroscientists look at two very different types of implementations of behavior. Now a neuroscientist might say that a cognitive scientist can't ignore what's happening in the neurons, and a physicist might say that a priest can't ignore the spiritual implications of the behavior of quarks. But in general we can advance science by looking at things from a variety of "semi-self-contained" perspectives.
• We have sometimes said that goals are not part of the cognitive architecture, but we have also seen that goals are a crucial part of cognitive architecture. How does this make sense?
  • This is a good question that probably arises from some sloppiness in my own terminology. We can certainly say that the capacity to have internal goals must be part of the cognitive architecture. However, it is not so clear that any particular or specific goals are part of the architecture. For example, everybody presumably has the cognitive capacity to have a goal to jump out of an airplane, but only a subset of us would ever actually have that goal (which would arise from our own set of experiences and environmental cues). On the other hand, there may be some goals that we would consider to be part of the cognitive architecture, such as the goals to "survive" and "procreate".
• The idea of propositions is unclear. Are they just one way of representing knowledge, or do we assume that all systems implement propositions in one way or another?
  • We have to be careful with this question, because different people use the word "proposition" for different things. For example, in logic a "proposition" is a "sentence" or a string of symbols that form an expression that is either true or false. You may think of this as a "symbol-level" use of the word. In class, I am trying to use the word "proposition" as more of a "knowledge-level" term. I consider a proposition to be any "piece of knowledge" we might make a statement of fact about. I am doing this on purpose, so we can consider any thinking thing to use propositions, even if we don't know how those propositions get represented inside the mind. But this allows me to make the claim that any successful cognitive architecture must include a model of working memory that holds the things we call "propositions".
• How is search defined?
  • For our purposes, search is a process of considering and pursuing alternative courses of action, while maintaining some memory of the alternatives at each "choice point". We usually view search as the process of finding paths through a set of possible states connected together by possible actions (or we often call the actions operators). Cognitive systems can be viewed as doing a variety of forms of search, such as:
    • Searching for the "thing to do next"
    • Searching for the right way to interpret the current situation
    • Searching for the right way to achieve goals
    • Searching for the right goals to use in the current situation
    • Searching for the best way to improve performance (learning)
• What are the parameters for a search, and how are they chosen?
  • Search usually involves the following pieces:
    • A language (knowledge representation) to describe different states the system might get into.
    • A set of operators that the system can use to get from one state to another
    • One particular state from which to start the search (called the initial state)
    • One or more states at which we may end the search (call goal states
    • An evaluation method for determining (or guessing) how far any given state is from a goal state (in order to help find the shortest or best path to a goal)
• What is the best search technique?
  • Being the "best" depends on many things. Do you want a search that guarantees it will find a path to a goal state? Does it have to find the shortest path? Does it have it find the "best" path? Does it have to find the path quickly? You can't usually get everything you want from a search, so you have to make tradeoffs. The "best" search technique will depend on the tradeoffs you are willing to make. The mind has developed in a way that makes certain kinds of tradeoffs for different kinds of search. Part of what we are doing in cognitive science is attempting to specify what those tradeoffs and conditions are.
• When learning from examples, why do the examples need to be "easy to interpret"?
  • Because two have the hardest parts of learning are knowing when to learn and what to learn. If it is not obvious that a particular example has something useful for your system to learn, you might not choose to learn anything about it, and miss an opportunity. If you know an example has something useful to tell you, but it's too difficult to figure out what, you might again fail to learn (or maybe learn the wrong thing). So the most effective learning will occur when it is easy to interpret when and how to use each example.
• How do we build an artificial system that can implement cognition in the same way a human brain does?
  • What do you mean by "in the same way"? Do you mean that the artificial system has to have neurons and biological circuits? Do you mean that the artificial system has to have digital subsystems that act like neurons and biological circuits? Do you mean that the artificial system has to "process symbols" (whatever those are) the way the brain does? Or do you mean that the artificial system has to generate behavior that looks like it was generated by a human brain? My personal belief is that every one of these is in the realm of possibility, but the "how" depends on which question you are interested in.
• What goes on in the connection between the physical and cognitive bands?
  • There may be some confusion in terminology here. Are you asking about the connection between Newell's biological and cognitive bands, or about the connection between the implementation (physical) and formal (symbolic) levels of analysis. In any event, Newell (1990) attempted to provide a more clear rationale for different levels of analysis, by mapping them to similar "bands" of timing in intelligent behavior. The biological band includes "significant events" that occur once every .0001 to .01 seconds. It would be difficult to model this time-scale of intelligent behavior without precisely specifying the physical implementation of the behavior, because this is as fast as organelles, neurons, and neural circuits can process information. The cognitive band pays less attention to the firing of individual neurons and collections of neurons, and instead focuses on "significant events" that occur every .1 to 10 seconds. These are the types of processes we generally attempt to model with symbol-processing systems, such as computers. At the biological band time-scale, it is meaningless to talk about symbols, because no such entities get processed by the things that are processing that quickly. The slower processes (the cognitive processes) are slow enough that we can look at the bigger patterns of information they manipulate and usefully call them "symbols".
• Are we nothing but conditional statements in a god's AI program?
• Do I make my own decisions day in and day out, or are they being dictated by a database and given to me by a higher being?
  • This is probably more than one question. Some cognitive scientists might claim that all of your actions are "dictated by a database", but I think there would be more disagreement over where that database came from. Some might say it came from a "higher being", and some might say it was an inevitable product of evolution. Others would certainly allow that you can "make your own decisions", even though at a lower level of processing your actions my be "dictated by a database".
• Is there a database out there that has all of my actions since my creation?
  • Perhaps in your brain? I have seen no convincing evidence that people do not store somewhere in their brains everything that has ever happened to them. This is a hard thing to determine by experiment, because it is difficult to distinguish between information that has disappeared from memory completely, and information that is still there but we have lost access to.
• If we had unlimited processing and memory power, could we simulate human being perfectly?
  • I don't personally feel that you even need unlimited processing and memory power to do that. Humans don't have unlimited processing and memory power. Why would we need it to simulate humans? And yes, I believe we will eventually be able to simulate humans.
• Is there a better way to organize the topics in cognitive science? Or is everything just tied into everything else?
  • There's almost certainly a better way. I welcome and encourage your comments (REALLY) so I can improve this course for the remainder of the semester and for the next time I teach it.
• What is the most successful cognitive model that has ever been designed?
• Whose theories on how the mind works are currently most popular (Simon's? Anderson's?)?
  • Newell and Simon (1972) can be considered to be two of the primary founders of the "classical view" of cognitive science, so we might consider their theories to be the most popular. However, in Anderson's terminology, the "classical view" might more properly be termed a "framework" than a "theory", even though it does make some specific (although high-level) claims about how humans behave. Anderson's ACT (1993) systems may well be the most popular, especially among psychologists. There are only a small number of systems that strive to be "Unified Theories of Cognition"; ACT (Anderson, 1993) and Soar (Newell, 1990) are probably the most famous and/or popular of those. Many people develop "isolated" cognitive models that focus only on particular pieces of cognition. So although these people learn lessons that others may build on, it's not quite the fact that they have complete theories that others borrow from. Finally, the general "connectionist framework" (again, Anderson would probably not call this a "theory") is used by quite a lot of people.
• Is any architecture that can run on a computer considered to be a physical symbol system? Why?
  • In general, all computer systems are assumed to be "physical symbol systems", because they represent symbols digitally at the physical level. One of the controversial (to some) claims is that humans and other thinking beings are also physical symbol systems. Because computer systems are physical symbol systems, most computer programs (including cognitive architectures) would also be physical symbol systems. However, one might argue that "connectionist" architectures are not physical symbol systems, because the digital data they manipulate does not correspond to what some people would call "symbols".
• I have gotten the feeling that "logic systems" only fail if the person making use of them does something wrong. Is this correct?
  • I'm not sure what you mean be "fail". In general, "logic systems" (like production-rule systems or Prolog) use formal methods to manipulate symbols, using the "rules of logic". Thus, these systems never "fail", because they will only compute the logical consequences of what you feed into them. If such a system failed to follow the rules of its own logic, we might hesitate to call it a "logic system". However, there are certainly systems that don't follow the rules of logic in general, but can exhibit logical-looking behavior (some might claim this is how the human mind works). It is usually the case that you might see what you would call "failure" if you feed inconsistent data or rules into a logic system. In this case, you might say that the person using the system did "something wrong". On the other hand, to model some kinds of psychological data, it seems necessary to allow a system to hold inconsistent beliefs at times. In these cases, perhaps the logic system would "fail" because it should fail (if it is meant to match the data).
• Are the architectures we've seen designed to model individual capabilities, or could one of them potentially be "the" cognitive system that grows and learns with no bounds?
  • By far, most of the work in cognitive science focuses on individual capabilities, or small collections of capabilities. There are some researchers who attempt to bring a wide variety of research efforts (on a wide variety of capabilities) under the umbrella of a single "theory of cognition". Probably the two most successful attempts at this are ACT-R (Anderson, 1993) and Soar (Newell, 1990). Even these two architectures will likely require some fine tuning and adjustment before they can become "the cognitive system that grows and learns with no bounds". But I consider them both to be large steps in the right direction (even though I have qualms about the theory behind each of them).
• What information has neurophysiology provided about the functioning of the brain?
  • Well, that's a pretty vague question. But that's okay, because my knowledge of neurophysiology is even more vague. Fortunately, there are others around who do research in this area, so I will try to ask some of them for some answers to this question.
• Has anyone tried to design a model that gets input from the world via sound and sight? Touch? Smell? Taste?
  • Much of this work is more computationally oriented than "cognitive sciencey", partly because there are still a lot of questions about how exactly the senses tie into and interact with cognition (at least from the "classical" point of view in cognitive science). However, there as been substantial work on computer vision, and also a fair amount on computer hearing. Some robots have tactile sensor to allow some notion of "touch", and I have heard of (but am not at all familiar with) projects to make artificial noses and taste buds to help sample chemicals in the air.
• Where do you (or others) think cognitive science will be in 10, 20, 30 years?
  • I try not to think about anything much farther than about next Tuesday. However, I'm willing to speculate a little. I think the primary use of cognitive science will be to create "simulated people" for a variety of types of problems and uses. As the years progress, the "simulated people" we create will become better and better, and they will become usable in more and more areas. Here is a sampling of the way "simulated people" will be used (remember this is all speculation on my part):
    • To design and test new training and education methods
    • To build virtual environments for education and entertainment
    • To create cheap actors for movies
    • To serve as "synthetic friends" for people
    • To test new designs for systems and devices that people have to interact with
    • To predict how people will behave in new types of physical and social situations
    • To test new marketing and advertising strategies
    • To create, test, and provide new ways for people to communicate and interact with each other

References

• Anderson, J. R. (1993). Rules of the Mind. Hillsdale, NJ: Lawrence Erlbaum.
• Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
• VanLehn, K. (1989). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.), Foundations of cognitive science. Cambridge, MA: MIT Press.