Ting Liao Receives NSF CAREER Award to Study Trust in Human-AI Engineering Design
To trust or not to trust AI: that is a question people face today and one that Ting Liao, an assistant professor of systems engineering at the Stevens Institute of Technology, has had on her mind for more than a decade.
In a Gallup survey spanning 37 countries, only about 36% of adults said they trust AI completely or a lot to provide accurate information. Those who use AI more frequently tend to have greater confidence in its accuracy than those who rarely or never use it, according to the survey.
For Liao, as AI becomes more integrated into engineering and design, trust becomes a pressing issue. “Too little and too much trust can create problems,” she said.
Liao recently received a Faculty Early Career Development (CAREER) Program award from the U.S. National Science Foundation (NSF) for the project, “Modeling Trust Dynamics and System Adaptation for Human-AI Collaboration in Engineering Design.” The project will examine trust in human-AI collaboration and develop AI systems that can adapt to users during the engineering design process.
Her work brings together engineering design, AI, experimental psychology, and data science to address a central question: How can people determine when and how much to rely on AI?
Her research focuses on what she describes as trust calibration, helping users find an appropriate balance between rejecting AI outputs and accepting them too readily. The goal is not necessarily to increase trust, she explained, but to help people recognize when AI deserves their confidence and when its recommendations merit another look.
Designing AI as a collaborator
The question of trust becomes more complicated as AI moves from being a tool engineers use to an active participant in the design process. Liao explained that this shift changes the traditional relationship between designer and technology, raising questions about how human and machine priorities fit together.
As part of human-centered design, “we need to care about the human, but now we also need to consider what machines care about, so that humans and machines work together optimally,” she said.
Liao’s research approaches that problem from two directions.
“First, we need to design AI. We want it to be user-friendly. We want it to be transparent. We want it to be collaborative,” Liao said. “The second part is AI also becomes a collaborator, like an active participant that we design with.”
But with this new collaboration comes competing priorities.
“It can become challenging because human values may conflict with machine values, such as consistency and efficiency,” Liao said. “So, we need to find an intersection between human values and machine values.”
Liao’s background in behavioral psychology and data science gives her tools for examining that relationship. Psychology helps explain how people perceive AI recommendations and make decisions, while data science can identify patterns in how they respond during an interaction.
A decade of studying human-AI trust
Liao’s interest in human-AI trust predates today’s generative AI boom. It goes back to her days as a doctoral student at Stanford University, where she developed a dissertation focused on how people interact with intelligent systems.
She noted how much of the discussion at the time centered on autonomous vehicles and questions such as how much control people should retain when machines make decisions. Liao also experimented with early conversational agents such as Alexa, whose capabilities were far more limited than those of today’s foundation models.
As AI has advanced, Liao observed the shift from under-trust to over-trust.
Liao explained that earlier technologies often faced skepticism because users were uncertain about what they could do. Today, advanced AI systems can create an opposite problem: users may accept their recommendations without sufficiently questioning them.
Liao explained that trust has become increasingly relevant as AI has moved into everyday work and decision-making. “This wasn’t traditional engineering work or system engineering work like 10 years ago, but now people are recognizing the importance of this work,” she said.
“My prior work focuses on understanding trust under the influence of many different factors,” Liao said. That work included her NSF Engineering Research Initiation project, “Developing a Trust-supporting Design Framework with Affect for Human-AI Collaboration,” which examined how different characteristics of AI systems influence human trust. It also produced the AI Design Assistant, or AIDA, an AI-supported engineering design tool used to study factors including performance, empathy, appearance, feedback, and trust.
“This CAREER project mainly focuses on the next step,” she said. Rather than only examining the factors that shape trust, Liao is working toward what she calls “trust-aware AI”: systems that can assess a user’s response in real time and adapt to the task, context, and individual interaction.
Measuring trust in real time
Surveys are commonly used to study trust, but “what people report does not always correspond with how they behave,” noted Liao. She added that surveys also capture trust only at particular moments, even though a person’s response to an AI system can shift throughout an interaction.
Her research incorporates physiological and behavioral signals that can be measured as people work with AI. The team plans to examine patterns that could indicate states such as hesitation, confidence, or information overload.
Eye-gaze patterns, for example, may help researchers recognize when someone is struggling to decide. Other behaviors could suggest that a user has become overwhelmed by information and is simply accepting or rejecting an AI recommendation rather than carefully considering it.
Identifying these states could help a system determine when additional guidance may be useful and when it should step back.
“The goal is to help users understand when they should trust it and when they should not,” Liao said. “What is the right time to trust it, and how much should you trust it?”
Even when users choose to rely on AI, she added, important decisions may still need a “sanity check.”
Preparing students to work with AI
Stevens students participating in the research will learn how to work with AI systems more critically. Graduate students will help develop the systems and analyze research data. Liao also envisions bringing adaptive AI technologies into the classroom, where students are already using AI tools for coursework and will likely encounter them throughout their careers.
Because AI capabilities can vary substantially from one task or context to another, Liao wants students to develop the judgment to know when a tool is appropriate and how much confidence to place in its output.
“These tools hopefully can train them to be AI-ready for their future work,” she said.
Learn more about academic programs and research in the Department of Systems Engineering:
