Chris goes on:
"If we were writing artificial intelligence for a robot, we would have to write all kinds of computer vision and analysis of the images to figure out what it was seeing. But because [the Halo characters] live in a simulated world, the characters can directly perceive that world. We chose to do that through simulated senses, because that way, the characters perceive the world in a way that players can reason about, because the player understands how their senses work in that world. We made simulated senses [for the AI].
So we have simulated vision, hearing and also a little bit of tactile knowledge. Where the player has five senses to deal with, and they're well developed, the AI in Halo really primarily only responds to visual input and sound input. That's because the two ways the player generally makes himself known to the AI is: The AI sees the player or they make a noise, like shooting someone."
"So we have simulated vision and sound. We take that information about what the AI can see right now and we turn that into a memory structure. So [the AI] is able to say, 'At this moment in time, I'm able to see [a woman] sitting right there. I see a woman wearing a black sweater right there.' You then turn that into more processed information by saying, 'I have a memory there that is [a woman] who is in this particular state at this time.' If [that woman] were to walk out of the room and go around that corner, I wouldn't see her anymore. But the AI would keep a memory of that character and the fact that they last saw her there, and when she left, she was moving in that direction.
So they have memory models that are what they know about the world. They take that memory model and turn it into more specific combat information. For instance, if I have memory of seven characters in this room and one of them is my friend and the other six are my enemies, I will have the knowledge that says, 'I am in a battle in close quarters with an overwhelming force. There are six enemies and only two friends in here.' That is an example of more processed information that is used to create a more persistent knowledge about what is the state of the battle or what is the state of the world and how should I react to it?"