The semantic interaction process seeks to elicit a user's mental model as they interact with and query visualizations during a sense-making activity. Semantic interaction enables the development of computational models that capture user intent and anticipate user actions. Deep learning is proving to be highly effective for learning complex functions and is, therefore, a compelling tool for encoding a user's mental model. In this paper, we show that deep contrastive learning significantly enhances semantic interaction in visual analytics systems. Our approach does so by allowing users to explore alternative arrangements of their data while simultaneously training a parametric algorithm to learn their evolving mental model. As an example of the efficacy of our approach, we deployed our model in Z-Explorer, a visual analytics extension to the widely used Zotero document management system. The user study demonstrates that this flexible approach effectively captures users' mental data models without explicit hyperparameter tuning or even requiring prior machine learning expertise.
IEEE Comput Graph Appl. 2014 Jul-Aug;34(4):8-15
[PMID:
25051565]
Nature. 2015 May 28;521(7553):436-44
[PMID:
26017442]
IEEE Trans Vis Comput Graph. 2018 Aug 20;:
[PMID:
30136978]
Bioinformatics. 2021 Apr 05;:
[PMID:
33822891]
IEEE Comput Graph Appl. 2013 Jul-Aug;33(4):6-13
[PMID:
24808054]
Nat Biotechnol. 2018 Dec 03;:
[PMID:
30531897]
PLoS One. 2013;8(3):e50474
[PMID:
23555552]
IEEE Trans Vis Comput Graph. 2017 Jan;23(1):241-250
[PMID:
27875141]
IEEE Trans Vis Comput Graph. 2013 Mar;19(3):367-78
[PMID:
22665724]
Methods Mol Biol. 2021;2190:73-94
[PMID:
32804361]