[KongchangAI]
· 2 min read· 1,220 words

Switching Control of Agent Avatars in VR: Users' Perceived Agency Does Not Update with the Actual Controller

Switching Control of Agent Avatars in VR: Users' Perceived Agency Does Not Update with the Actual Controller

VR experiment finds that after control switches, users' perceived agency of a virtual human remains anchored to the initial label, while social evaluations change with actual behavior.

An arXiv study used a 24-participant within-subjects VR experiment to examine how users' perceptions change when control of a mixed-control virtual human switches. The four conditions were always computer-controlled, computer-to-human, human-to-computer, and always human-controlled, with participants acting as interviewers. Results showed perceived agency was strongly influenced by the controller type announced at the start, with no evidence of updating after a switch, indicating the enduring role of prior beliefs and first impressions. In contrast, social evaluations (liking, social presence) were higher when human control appeared later and covered more of the interaction, showing greater sensitivity to actual behavior. The study reveals a dissociation between perceived agency and social evaluation and offers design implications: switching the actual controller alone is insufficient for accurate perception, while improving interactive behavior quality can still enhance the social experience.

Research Background: Perceived Agency in Mixed-Control Virtual Humans

In virtual reality (VR), whether a Virtual Human (VH) is controlled by a real person or by a computer program profoundly affects how users interpret and respond to its behavior. With the emergence of mixed VH systems, control can transfer from a human operator to a computer program during an interaction, or vice versa. But a key question remains unanswered: when control switches, does users' perception of the VH's "agency" update accordingly, or does it stay anchored to the impression formed initially?

A research paper published on arXiv, Perceived Agency During Agent-Avatar Control Transitions in Virtual Reality, conducted an experiment to address this question. The team designed a within-subjects VR experiment with 24 participants and four conditions: always computer-controlled, computer-to-human, human-to-computer, and always human-controlled. Participants played the role of an interviewer and asked the VH a fixed set of questions. Before each interaction, participants were told whether the VH was controlled by a human or a computer—this information matched the actual controller at the start of the interaction, but participants did not know that control might switch during the interaction.

Research source screenshot

Perceived Agency is a core concept in social cognition and virtual human interaction research, referring to users' judgment of whether an entity's (here, a virtual human's) behavior is driven by its own intentions or by an external program. In the human–agent interaction field, it is often discussed alongside concepts such as "perceived autonomy" and "perceived intelligence," but it emphasizes inference about "who is in control"—a causal attribution. Perceived agency differs from objective control: the former is the user's subjective attribution, while the latter is the actual source of control at the system level. Prior studies have shown that people's judgments of agency rely heavily on cues such as behavioral naturalness, response latency, and language style, and are easily anchored by prior expectations. In mixed VH systems, control can transfer during an interaction, making perceived agency a dynamic but potentially lagging cognitive variable—precisely the focus of this study.

Core Finding: Initial Beliefs Dominate Perceived Agency

The results showed that perceived agency was strongly influenced by the type of controller presented at the start of the interaction, and there was no evidence that perceived agency updated after control switched. In other words, even if the VH's actual controller changed midway, users still tended to judge its agency according to the information they were initially given.

This finding reveals the enduring role of prior beliefs and first impressions in shaping perceived agency. Users appear to internalize the initial label of "whether this VH is controlled by a human or a computer" as a stable cognitive framework, and subsequent behavioral changes are insufficient to break that framework. This runs counter to the intuition of many mixed-system designers, who might assume that users can detect the transfer of control and adjust their judgments accordingly.

Social Evaluations Are More Sensitive to Actual Behavioral Changes

Unlike perceived agency, the social evaluation dimensions (including liking and social presence) showed a different pattern. The study found that liking and social presence were generally higher when human control appeared later and covered a larger proportion of the interaction. This means social evaluations are more sensitive to the VH's actual behavior during the interaction, rather than relying solely on the initial label.

This dissociation is worth pondering: perceived agency is a relatively "cognitively fixed" judgment, whereas social evaluations are based more on the dynamic accumulation of interaction experience. Users may insist at a rational level that "it was human-controlled at the beginning," yet at an emotional and social level adjust their feelings according to the actual quality of the interaction.

Social Presence is a classic construct in VR and remote communication research, referring to the degree to which users feel that the other party is a "genuinely present social other" when interacting with a mediated entity. It typically includes dimensions such as joint attention, emotional connection, and spatial co-presence, and is related to but distinct from evaluative measures such as liking and trust. Social presence is considered a key indicator of the social quality of virtual humans and is strongly affected by nonverbal cues (eye contact, gestures, responsiveness) and interaction fluency. In this study, social presence was used as one of the social evaluation dimensions; the results showed higher social presence when human control appeared later and covered a larger proportion, indicating that users are dynamically sensitive to the social signals of "real human participation"—a contrast with the cognitive fixation of perceived agency.

Implications for Mixed Virtual Human Design

This study has direct implications for the design of mixed VH systems. If designers want users to accurately perceive control switches, actually switching the controller is not enough—users' initial beliefs will continue to override subsequent perception. Therefore, it may be necessary to provide explicit cues or feedback during the interaction to signal control changes, or to rethink the interaction design of control transitions.

On the other hand, the sensitivity of social evaluations to actual behavior suggests to designers that even if perceived agency is hard to change, improving the quality of interactive behavior can still enhance users' social experience. For application scenarios that require flexible switching between human and machine control (such as telepresence, virtual assistants, and social VR), this finding offers important design guidance.

Limitations and Future Directions

The study used a within-subjects design with a sample of 24 participants, and the experimental task was an interview scenario with fixed questions. The timing and proportion of control switches were key variables in the experiment, but the study did not discuss in detail how different switch timings differentially affect perceived agency. Future research could explore: whether perceived agency eventually updates over longer interactions; whether different types of control-switch cues (visual, behavioral, linguistic) can promote perceptual updating; and whether users' acceptance of control switches differs across application scenarios.

Overall, this study reveals an easily overlooked cognitive phenomenon in mixed virtual human interaction: users' judgments of "who is in control" are more stubbornly anchored to first impressions than we imagine, while social-level feelings are more flexible.

One-line Summary

A VR experiment found that after control switches, users' perceived agency of a virtual human remains anchored to the initial label, while social evaluations change with actual behavior.

Paragraph Summary

A study published on arXiv used a within-subjects VR experiment with 24 participants to examine how users' perceptions change when control of a mixed-control virtual human switches. The experiment set four conditions—always computer-controlled, computer-to-human, human-to-computer, and always human-controlled—with participants playing the role of an interviewer interacting with the virtual human. Results showed that perceived agency was strongly influenced by the type of controller announced at the start of the interaction, with no evidence that perception updated after control switched, indicating the enduring role of prior beliefs and first impressions. In contrast, social evaluation dimensions (liking, social presence) were rated higher in conditions where human control appeared later and covered a larger proportion, indicating greater sensitivity to actual behavioral changes. The study reveals a dissociation between perceived agency and social evaluation, and offers implications for mixed virtual human design: merely switching the actual controller is not enough for users to perceive it accurately—explicit cues are needed; yet improving the quality of interactive behavior can still enhance the social experience.

#VR#virtual human#perceived agency#control transition#social presence#mixed control#human-agent interaction
Share:

Related articles