Towards Collaborative Sensemaking with AI in Visual Analytics

Human and Robot Hand Interaction.

Department of Electrical and Computer Engineering

Location: Burchard Hall, Room 104

Speaker: Adam Coscia, Assistant Professor, Stevens Institute of Technology

ABSTRACT

From national security to ​healthcare and robotics, the world's best analysts, engineers, and practitioners rely on analysis tools to help them gather, organize, and make critical decisions with data. Today, those tools increasingly automate analysis using AI. If AI can retrieve, summarize, and explain our data for us, how should we rethink the way we build analysis tools for experts?

In this talk, I present my research vision for collaborative sensemaking with AI in visual analytics, exploring how visualizations can serve as a bridge between human expertise and AI capabilities. Rather than replacing humans with fully automated systems, my work investigates how humans and AI can operate as collaborative partners in the sensemaking process by balancing machine automation with human judgment. I demonstrate three different systems that make human-AI collaboration interactive and visual. (1) Goal tracking: OnGoal presents a goal-tracking framework that can visualize whether an LLM maintains a user's goals across long conversations. (2) Fact-checking: VisPile uses LLMs and knowledge graphs to help intelligence analysts organize their documents, saving time reading while grounding AI-generated evidence in sources and the KG. (3) Human-machine teaming: Retrace helps operators debrief with robot teammates by annotating deterministic visualizations of robot logs with generative LLM responses. Across these systems, I argue the value of visualization lies in its ability to act as a two-way communication channel between people and machines. I conclude by outlining future research directions for human-AI collaboration in visual analytics.

BIOGRAPHY

Adam Coscia.

Adam Coscia is a Tenure-Track Assistant Professor in the School of Computing at Stevens Institute of Technology, where he directs the Stevens Visualization Lab. His research helps people make sense of complex data through the design of interactive systems for data visualization and analysis. His tools are used by analysts, educators, engineers, and scientists in domains including education, intelligence analysis, autonomous robotics, space science, ocean science, and graphic design. His work has been generously supported by the NSF AI-ALOE Institute, NSA, MIT Lincoln Lab, NASA JPL, Adobe Research, the NC State Laboratory for Analytic Sciences, and the Monterey Bay Aquarium Research Institute. His work appears at premier HCI and visualization venues, including ACM CHI, UIST, IUI, and IEEE VIS and TVCG, and has received a best paper award and three best poster awards. He was also a finalist for the Foley Scholars Award and a recipient of Georgia Tech's President's Fellowship. He received his Ph.D. in Human-Centered Computing from the Georgia Institute of Technology and holds a B.S. in Physics from Stevens.

Discrimination notice: Persons of all identities are invited to and included in this group. Stevens does not discriminate against any person on the basis of sex, race, religion, disability, sexual orientation, gender expression, or any other basis prohibited by law.

Photo and video notice: At any time, photography or videography may be occurring on Stevens’ campus. Resulting footage may include the image or likeness of event attendees. Such footage is Stevens’ property and may be used for Stevens’ commercial and/or noncommercial purposes. By registering for and/or attending this event, you consent and waive any claim against Stevens related to such use in any media. See Stevens' Privacy Policy for more information.