"Hey Dashboard!": Supporting Voice, Text, and Pointing Modalities in Dashboard Onboarding using Large Language Models | TU Wien

Information

Publication Type: Conference Paper
Workgroup(s)/Project(s): not specified
Date: 2026
ISBN: 979-8-4007-2278-3
Publisher: Association for Computing Machinery
Location: Barcelona
Lecturer: Vaishali Dhanoa
Event: ACM CHI Conference on Human Factors in Computing Systems (CHI '26)
DOI: 10.1145/3772318.3791766
Booktitle: CHI '26: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
Pages: 15
Conference date: 13. April 2026 – 17. April 2026
Pages: 1 – 15
Keywords: dashboards, large language models, multimodal interactions, onboarding, visualization, visualization literacy

Abstract

Visualization dashboards are regularly used for data exploration and analysis, but their complex interactions and interlinked views often require time-consuming onboarding sessions from dashboard authors. Preparing these onboarding materials is labor-intensive and requires manual updates when dashboards change. Recent advances in multimodal interaction powered by large language models (LLMs) provide ways to support self-guided onboarding. We present Diana (Dashboard Interactive Assistant for Navigation and Analysis), a multimodal dashboard assistant that helps users for navigation and guided analysis through chat, audio, and mouse-based interactions. Users can choose any interaction modality or a combination of them to onboard themselves on the dashboard. Each modality highlights relevant dashboard features to support user orientation. Unlike typical LLM systems that rely solely on text-based chat, Diana combines multiple modalities to provide explanations directly in the dashboard interface. We conducted a comparative qualitative user study to understand the use of different modalities for different types of onboarding tasks and their complexities.

Additional Files and Images

No additional files or images.

Weblinks

BibTeX

@inproceedings{dhanoa-2026-hds,
  title =      ""Hey Dashboard!": Supporting Voice, Text, and Pointing
               Modalities in Dashboard Onboarding using Large Language
               Models",
  author =     "Vaishali Dhanoa and Gabriela Molina Le\'{o}n and Eve Hoggan
               and Eduard Gr\"{o}ller and Marc Streit and Niklas Elmqvist",
  year =       "2026",
  abstract =   "Visualization dashboards are regularly used for data
               exploration and analysis, but their complex interactions and
               interlinked views often require time-consuming onboarding
               sessions from dashboard authors. Preparing these onboarding
               materials is labor-intensive and requires manual updates
               when dashboards change. Recent advances in multimodal
               interaction powered by large language models (LLMs) provide
               ways to support self-guided onboarding. We present Diana
               (Dashboard Interactive Assistant for Navigation and
               Analysis), a multimodal dashboard assistant that helps users
               for navigation and guided analysis through chat, audio, and
               mouse-based interactions. Users can choose any interaction
               modality or a combination of them to onboard themselves on
               the dashboard. Each modality highlights relevant dashboard
               features to support user orientation. Unlike typical LLM
               systems that rely solely on text-based chat, Diana combines
               multiple modalities to provide explanations directly in the
               dashboard interface. We conducted a comparative qualitative
               user study to understand the use of different modalities for
               different types of onboarding tasks and their complexities.",
  isbn =       "979-8-4007-2278-3",
  publisher =  "Association for Computing Machinery",
  location =   "Barcelona",
  event =      "ACM CHI Conference on Human Factors in Computing Systems
               (CHI '26)",
  doi =        "10.1145/3772318.3791766",
  booktitle =  "CHI '26: Proceedings of the 2026 CHI Conference on Human
               Factors in Computing Systems",
  pages =      "15",
  pages =      "1--15",
  keywords =   "dashboards, large language models, multimodal interactions,
               onboarding, visualization, visualization literacy",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2026/dhanoa-2026-hds/",
}