Maath Musleh
, Davide Ceneda, Henry Ehlers, Renata Raidou
ConAn: Measuring and Evaluating User Confidence in Visual Data Analysis Under Uncertainty
Computer Graphics Forum, 44(1):e15272, February 2025.
Information
- Publication Type: Journal Paper with Conference Talk
- Workgroup(s)/Project(s):
- Date: February 2025
- Journal: Computer Graphics Forum
- Volume: 44
- Open Access: yes
- Number: 1
- Article Number: e15272
- ISSN: 1467-8659
- Event: EuroVis 2025
- DOI: 10.1111/cgf.15272
- Pages: 18
- Publisher: WILEY
- Keywords: decision making, uncertainty, user confidence, visual analytics, guided visual data analysis
Abstract
User confidence plays an important role in guided visual data analysis scenarios, especially when uncertainty is involved in the analytical process. However, measuring confidence in practical scenarios remains an open challenge, as previous work relies primarily on self-reporting methods. In this work, we propose a quantitative approach to measure user confidence—as opposed to trust—in an analytical scenario. We do so by exploiting the respective user interaction provenance graph and examining the impact of guidance using a set of network metrics. We assess the usefulness of our proposed metrics through a user study that correlates results obtained from self-reported confidence assessments and our metrics—both with and without guidance. The results suggest that our metrics improve the evaluation of user confidence compared to available approaches. In particular, we found a correlation between self-reported confidence and some of the proposed provenance network metrics. The quantitative results, though, do not show a statistically significant impact of the guidance on user confidence. An additional descriptive analysis suggests that guidance could impact users' confidence and that the qualitative analysis of the provenance network topology can provide a comprehensive view of changes in user confidence. Our results indicate that our proposed metrics and the provenance network graph representation support the evaluation of user confidence and, subsequently, the effective development of guidance in VA.Additional Files and Images
Weblinks
BibTeX
@article{musleh-2024-conan,
title = "ConAn: Measuring and Evaluating User Confidence in Visual
Data Analysis Under Uncertainty",
author = "Maath Musleh and Davide Ceneda and Henry Ehlers and Renata
Raidou",
year = "2025",
abstract = "User confidence plays an important role in guided visual
data analysis scenarios, especially when uncertainty is
involved in the analytical process. However, measuring
confidence in practical scenarios remains an open challenge,
as previous work relies primarily on self-reporting methods.
In this work, we propose a quantitative approach to measure
user confidence—as opposed to trust—in an analytical
scenario. We do so by exploiting the respective user
interaction provenance graph and examining the impact of
guidance using a set of network metrics. We assess the
usefulness of our proposed metrics through a user study that
correlates results obtained from self-reported confidence
assessments and our metrics—both with and without
guidance. The results suggest that our metrics improve the
evaluation of user confidence compared to available
approaches. In particular, we found a correlation between
self-reported confidence and some of the proposed provenance
network metrics. The quantitative results, though, do not
show a statistically significant impact of the guidance on
user confidence. An additional descriptive analysis suggests
that guidance could impact users' confidence and that the
qualitative analysis of the provenance network topology can
provide a comprehensive view of changes in user confidence.
Our results indicate that our proposed metrics and the
provenance network graph representation support the
evaluation of user confidence and, subsequently, the
effective development of guidance in VA.",
month = feb,
journal = "Computer Graphics Forum",
volume = "44",
number = "1",
articleno = "e15272",
issn = "1467-8659",
doi = "10.1111/cgf.15272",
pages = "18",
publisher = "WILEY",
keywords = "decision making, uncertainty, user confidence, visual
analytics, guided visual data analysis",
URL = "https://www.cg.tuwien.ac.at/research/publications/2025/musleh-2024-conan/",
}