Guanhua Zhang

Institute for Visualisation and Interactive Systems, University of Stuttgart

+49 711 685 60152 Pfaffenwaldring 5a, 70569 Stuttgart, Germany University of Stuttgart, SimTech Building, Room 01.033 Google Scholar

Biography

Guanhua Zhang is a PhD student in the Collaborative Artificial Intelligence group since September 2020 and in the International Max Planck Research School for Intelligent Systems (IMPRS-IS) since February 2021. She holds a Bachelor’s degree in Computer Science and Technology from Beijing University of Posts and Telecommunications and a Master’s degree in Computer Science and Technology from Tsinghua University. Her research interests include representation learning, user behaviour and intention modelling, and affective computing.

Teaching

2024

Machine Perception and Learning (Teaching Assistant)Master

2023

Machine Perception and Learning (Tutor)Master

2022

Machine Perception and Learning (Tutor)Master

2021

Machine Learning and Computer Vision for HCI (Fachpraktikum)Master

2020

Machine Learning Approaches to Understanding User Behavior (Hauptseminar)Master

---

Supervision

2024

Cross-Device Dataset Collection (M.Sc.) Kaiyang Zhu

2023

Generalised Encoding for Mouse and Keyboard Data (B.Sc.) Manuel Schwartz

 

Keyboard behaviour representation learning (HiWi Project) Jize Zhang

2022

Mouse behaviour representation learning (HiWi Project) Yun Ye

 

Gaze2Vec: Learning Embeddings from Camera-based Eye Tracking Data (M.Sc.) Nithyasree Pushpalatha

 

Intent Prediction from Gaze during Text Formatting (HiWi Project) Xuefeng Hou

2021

Multimodal Personality Synthesis on 3D Avatars (B.Sc.) Jiaxuan He

 

Hyperdimensional Computing-based Prediction of Personality Traits Using Multimodal Data (InfoTech Project) Nithyasree Pushpalatha

 

Group-Level Personality Analysis and Detection (M.Sc.) Anandavalli Veerasubramanian

---

Publications

(Jiao et al., 2024; Zhang et al., 2024; Jiao et al., 2024; Zhang et al., 2024; Zhang et al., 2023; Zhang et al., 2022; Müller et al., 2021; Zhang et al., 2021; Zhao et al., 2020; Du et al., 2020; Zhang et al., 2019)

1. 

   DisMouse: Disentangling Information from Mouse Movement Data

   Guanhua Zhang, Zhiming Hu, Andreas Bulling

   Proc. ACM Symposium on User Interface Software and Technology (UIST), pp. 1–13, 2024.

   Abstract Links BibTeX Project

   Mouse movement data contain rich information about users, performed tasks, and user interfaces, but separating the respective components remains challenging and unexplored. As a first step to address this challenge, we propose DisMouse – the first method to disentangle user-specific and user-independent information and stochastic variations from mouse movement data. At the core of our method is an autoencoder trained in a semi-supervised fashion, consisting of a self-supervised denoising diffusion process and a supervised contrastive user identification module. Through evaluations on three datasets, we show that DisMouse 1) captures complementary information of mouse input, hence providing an interpretable framework for modelling mouse movements, 2) can be used to produce refined features, thus enabling various applications such as personalised and variable mouse data generation, and 3) generalises across different datasets. Taken together, our results underline the significant potential of disentangled representation learning for explainable, controllable, and generalised mouse behaviour modelling.

   doi: https://doi.org/10.1145/3654777.3676411

   Paper: zhang24_uist.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/DisMouse.git

   @inproceedings{zhang24_uist, title = {DisMouse: Disentangling Information from Mouse Movement Data}, author = {Zhang, Guanhua and Hu, Zhiming and Bulling, Andreas}, year = {2024}, pages = {1--13}, booktitle = {Proc. ACM Symposium on User Interface Software and Technology (UIST)}, doi = {https://doi.org/10.1145/3654777.3676411} }
2. 

   Mouse2Vec: Learning Reusable Semantic Representations of Mouse Behaviour

   Guanhua Zhang, Zhiming Hu, Mihai Bâce, Andreas Bulling

   Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1–17, 2024.

   Abstract Links BibTeX Project

   The mouse is a pervasive input device used for a wide range of interactive applications. However, computational modelling of mouse behaviour typically requires time-consuming design and extraction of handcrafted features, or approaches that are application-specific. We instead propose Mouse2Vec – a novel self-supervised method designed to learn semantic representations of mouse behaviour that are reusable across users and applications. Mouse2Vec uses a Transformer-based encoder-decoder architecture, which is specifically geared for mouse data: During pretraining, the encoder learns an embedding of input mouse trajectories while the decoder reconstructs the input and simultaneously detects mouse click events. We show that the representations learned by our method can identify interpretable mouse behaviour clusters and retrieve similar mouse trajectories. We also demonstrate on three sample downstream tasks that the representations can be practically used to augment mouse data for training supervised methods and serve as an effective feature extractor.

   doi: 10.1145/3613904.3642141

   Paper: zhang24_chi.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/Mouse2Vec

   @inproceedings{zhang24_chi, title = {Mouse2Vec: Learning Reusable Semantic Representations of Mouse Behaviour}, author = {Zhang, Guanhua and Hu, Zhiming and B{\^a}ce, Mihai and Bulling, Andreas}, year = {2024}, pages = {1--17}, booktitle = {Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)}, doi = {10.1145/3613904.3642141} }

1. 

   DiffEyeSyn: Diffusion-based User-specific Eye Movement Synthesis

   Chuhan Jiao, Guanhua Zhang, Zhiming Hu, Andreas Bulling

   arXiv:2409.01240, pp. 1–22, 2024.

   Abstract Links BibTeX Project

   High-frequency components in eye gaze data contain user-specific information promising for various applications, but existing gaze modelling methods focus on low frequencies of typically not more than 30 Hz. We present DiffEyeSyn – the first computational method to synthesise high-frequency gaze data, including eye movement characteristics specific to individual users. The key idea is to consider the high-frequency, user-specific information as a special type of noise in eye movement data. This perspective reshapes eye movement synthesis into the task of injecting this user-specific noise into any given eye movement sequence. We formulate this injection task as a conditional diffusion process in which the synthesis is conditioned on user-specific embeddings extracted from the gaze data using pre-trained models for user authentication. We propose user identity guidance – a novel loss function that allows our model to preserve user identity while generating human-like eye movements in the spatial domain. Experiment results on two public high-frequency eye movement biometric datasets show that our synthetic eye movements are indistinguishable from real human eye movements. Furthermore, we demonstrate that DiffEyeSyn can be used to synthesise eye gaze data at scale and for different downstream tasks, such as gaze data imputation and gaze data super-resolution. As such, our work lays the methodological foundations for personalised eye movement synthesis that has significant application potential, such as for character animation, eye movement biometrics, or gaze-based activity and context recognition.

   Paper Access: https://arxiv.org/abs/2409.01240

   @techreport{jiao24_arxiv_2, title = {DiffEyeSyn: Diffusion-based User-specific Eye Movement Synthesis}, author = {Jiao, Chuhan and Zhang, Guanhua and Hu, Zhiming and Bulling, Andreas}, year = {2024}, pages = {1--22}, url = {https://arxiv.org/abs/2409.01240} }
2. 

   DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images

   Chuhan Jiao, Yao Wang, Guanhua Zhang, Mihai Bâce, Zhiming Hu, Andreas Bulling

   arXiv:2403.17477, pp. 1–13, 2024.

   Abstract Links BibTeX Project

   We present DiffGaze, a novel method for generating realistic and diverse continuous human gaze sequences on 360° images based on a conditional score-based denoising diffusion model. Generating human gaze on 360° images is important for various human-computer interaction and computer graphics applications, e.g. for creating large-scale eye tracking datasets or for realistic animation of virtual humans. However, existing methods are limited to predicting discrete fixation sequences or aggregated saliency maps, thereby neglecting crucial parts of natural gaze behaviour. Our method uses features extracted from 360° images as condition and uses two transformers to model the temporal and spatial dependencies of continuous human gaze. We evaluate DiffGaze on two 360° image benchmarks for gaze sequence generation as well as scanpath prediction and saliency prediction. Our evaluations show that DiffGaze outperforms state-of-the-art methods on all tasks on both benchmarks. We also report a 21-participant user study showing that our method generates gaze sequences that are indistinguishable from real human sequences. Taken together, our evaluations not only demonstrate the effectiveness of DiffGaze but also point towards a new generation of methods that faithfully model the rich spatial and temporal nature of natural human gaze behaviour.

   Paper Access: https://arxiv.org/abs/2403.17477

   @techreport{jiao24_arxiv, title = {DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images}, author = {Jiao, Chuhan and Wang, Yao and Zhang, Guanhua and B{\^a}ce, Mihai and Hu, Zhiming and Bulling, Andreas}, year = {2024}, pages = {1--13}, url = {https://arxiv.org/abs/2403.17477} }

1. 

   Exploring Natural Language Processing Methods for Interactive Behaviour Modelling

   Guanhua Zhang, Matteo Bortoletto, Zhiming Hu, Lei Shi, Mihai Bâce, Andreas Bulling

   Proc. IFIP TC13 Conference on Human-Computer Interaction (INTERACT), pp. 1–22, 2023.

   Abstract Links BibTeX Project Best Student Paper Nomination

   Analysing and modelling interactive behaviour is an important topic in human-computer interaction (HCI) and a key requirement for the development of intelligent interactive systems. Interactive behaviour has a sequential (actions happen one after another) and hierarchical (a sequence of actions forms an activity driven by interaction goals) structure, which may be similar to the structure of natural language. Designed based on such a structure, natural language processing (NLP) methods have achieved groundbreaking success in various downstream tasks. However, few works linked interactive behaviour with natural language. In this paper, we explore the similarity between interactive behaviour and natural language by applying an NLP method, byte pair encoding (BPE), to encode mouse and keyboard behaviour. We then analyse the vocabulary, i.e., the set of action sequences, learnt by BPE, as well as use the vocabulary to encode the input behaviour for interactive task recognition. An existing dataset collected in constrained lab settings and our novel out-of-the-lab dataset were used for evaluation. Results show that this natural language-inspired approach not only learns action sequences that reflect specific interaction goals, but also achieves higher F1 scores on task recognition than other methods. Our work reveals the similarity between interactive behaviour and natural language, and presents the potential of applying the new pack of methods that leverage insights from NLP to model interactive behaviour in HCI.

   Paper: zhang23_interact.pdf

   @inproceedings{zhang23_interact, title = {Exploring Natural Language Processing Methods for Interactive Behaviour Modelling}, author = {Zhang, Guanhua and Bortoletto, Matteo and Hu, Zhiming and Shi, Lei and B{\^a}ce, Mihai and Bulling, Andreas}, booktitle = {Proc. IFIP TC13 Conference on Human-Computer Interaction (INTERACT)}, pages = {1--22}, year = {2023}, publisher = {Springer} }

1. 

   Predicting Next Actions and Latent Intents during Text Formatting

   Guanhua Zhang, Susanne Hindennach, Jan Leusmann, Felix Bühler, Benedict Steuerlein, Sven Mayer, Mihai Bâce, Andreas Bulling

   Proc. the CHI Workshop Computational Approaches for Understanding, Generating, and Adapting User Interfaces, pp. 1–6, 2022.

   Abstract Links BibTeX Project

   In this work we investigate the challenging task of predicting user intents from mouse and keyboard input as well as gaze behaviour. In contrast to prior work we study intent prediction at two different resolutions on the behavioural timeline: predicting future input actions as well as latent intents to achieve a high-level interaction goal. Results from a user study (N=15) on a sample text formatting task show that the sequence of prior actions is more informative for intent prediction than gaze. Only using the action sequence, we can predict the next action and the high-level intent with an accuracy of 66% and 96%, respectively. In contrast, accuracy when using features extracted from gaze behaviour was significantly lower, at 41% and 46%. This finding is important for the development of future anticipatory user interfaces that aim to proactively adapt to user intents and interaction goals.

   Paper: zhang22_caugaui.pdf

   @inproceedings{zhang22_caugaui, author = {Zhang, Guanhua and Hindennach, Susanne and Leusmann, Jan and Bühler, Felix and Steuerlein, Benedict and Mayer, Sven and Bâce, Mihai and Bulling, Andreas}, title = {Predicting Next Actions and Latent Intents during Text Formatting}, booktitle = {Proc. the CHI Workshop Computational Approaches for Understanding, Generating, and Adapting User Interfaces}, year = {2022}, pages = {1--6} }

1. SparseDGCNN: Recognizing Emotion from Multichannel EEG Signals

   Guanhua Zhang, Minjing Yu, Yong-Jin Liu, Guozhen Zhao, Dan Zhang

   IEEE Transactions on Affective Computing (TAFFC), , pp. 1–12, 2021.

   Abstract Links BibTeX Project

   Emotion recognition from EEG signals has attracted much attention in affective computing. Recently, a novel dynamic graph convolutional neural network (DGCNN) model was proposed, which simultaneously optimized the network parameters and a weighted graph G characterizing the strength of functional relation between each pair of two electrodes in the EEG recording equipment. In this paper, we propose a sparse DGCNN model which improves the DGCNN by imposing a sparseness constraint on G. Our work is based on an important observation: the tomography study reveals that different brain regions sampled by EEG electrodes may be related to different functions of the brain and then the functional relations among electrodes are possibly highly localized and sparse. However, introducing sparseness constraint into the graph G makes the loss function of sparse DGCNN non-differentiable at some singular points. To ensure that the training process of sparse DGCNN converges, we apply the forward-backward splitting method. To evaluate the performance of sparse DGCNN, we compare it with four representative recognition methods as well as different features and spectral bands. The results show that (1) sparse DGCNN has consistently better accuracy than representative methods and has a good scalability, and (2) DE, PSD and ASM features on γ bands convey most discriminative emotional information, and fusion of separate features and frequency bands can improve recognition performance.

   doi: 10.1109/TAFFC.2021.3051332

   @article{zhang21_taffc, title = {SparseDGCNN: Recognizing Emotion from Multichannel EEG Signals}, author = {{Zhang}, Guanhua and {Yu}, Minjing and {Liu}, Yong-Jin and {Zhao}, Guozhen and {Zhang}, Dan}, journal = {IEEE Transactions on Affective Computing (TAFFC)}, doi = {10.1109/TAFFC.2021.3051332}, year = {2021}, pages = {1--12} }

1. 

   MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation

   Philipp Müller, Dominik Schiller, Dominike Thomas, Guanhua Zhang, Michael Dietz, Patrick Gebhard, Elisabeth André, Andreas Bulling

   Proc. ACM Multimedia (MM), pp. 4878–4882, 2021.

   Abstract Links BibTeX Project

   Artificial mediators are promising to support human group conversations but at present their abilities are limited by insufficient progress in group behaviour analysis. The MultiMediate challenge addresses, for the first time, two fundamental group behaviour analysis tasks in well-defined conditions: eye contact detection and next speaker prediction. For training and evaluation, MultiMediate makes use of the MPIIGroupInteraction dataset consisting of 22 three- to four-person discussions as well as of an unpublished test set of six additional discussions. This paper describes the MultiMediate challenge and presents the challenge dataset including novel fine-grained speaking annotations that were collected for the purpose of MultiMediate. Furthermore, we present baseline approaches and ablation studies for both challenge tasks.

   doi: 10.1145/3474085.3479219

   Paper: mueller21_mm.pdf

   @inproceedings{mueller21_mm, title = {MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation}, author = {M{\"{u}}ller, Philipp and Schiller, Dominik and Thomas, Dominike and Zhang, Guanhua and Dietz, Michael and Gebhard, Patrick and André, Elisabeth and Bulling, Andreas}, year = {2021}, pages = {4878--4882}, doi = {10.1145/3474085.3479219}, booktitle = {Proc. ACM Multimedia (MM)} }

1. Multi-target Positive Emotion Recognition from EEG Signals

   Guozhen Zhao, Yulin Zhang, Guanhua Zhang, Dan Zhang, Yong-Jin Liu

   IEEE Transactions on Affective Computing (TAFFC), , pp. 1-13, 2020.

   Abstract Links BibTeX Project

   Compared with the widely studied negative emotions in which different classes are easy to distinguish, nowadays less attention is paid to the recognition of positive emotions that are not fully independent. In this paper, we propose to recognize multiple positive emotions by analyzing brain activities and explore the neural representation of different positive emotions. Thirty-seven participants volunteered to participate in our study, in which their brain activities were recorded when watching five selected film clips. First, 150 well-known power features extracted from Electroencephalography (EEG) signals and 105 multimedia content analysis features were collected as the pool of candidate features. Second, based on the collected features, we propose to use a linear model and a nonlinear model to predict the percentage of five positive emotions. Then, percentage values were converted to ranking numbers and Kendall rank correlation coefficients were calculated. Our results showed that (1) ensemble of regressor chains using LSTM as unit regressor obtained both the best regression results and the best Kendall rank correlation coefficient on EEG features merely, and (2) top features from alpha frequency bands of EEG signals could represent different positive emotions. These results demonstrate the effectiveness of selective EEG features on recognizing different positive emotions.

   doi: 10.1109/TAFFC.2020.3043135

   @article{zhao20_taffc, title = {Multi-target Positive Emotion Recognition from EEG Signals}, author = {Zhao, Guozhen and Zhang, Yulin and Zhang, Guanhua and Zhang, Dan and Liu, Yong-Jin}, year = {2020}, journal = {IEEE Transactions on Affective Computing (TAFFC)}, doi = {10.1109/TAFFC.2020.3043135}, pages = {1-13} }

2. An Efficient LSTM Network for Emotion Recognition from Multichannel EEG Signals

   Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng, Yong-Jin Liu, Hongan Wang

   IEEE Transactions on Affective Computing (TAFFC), , pp. 1-12, 2020.

   Abstract Links BibTeX Project

   Most previous EEG-based emotion recognition methods studied hand-crafted EEG features extracted from different electrodes. In this paper, we study the relation among different EEG electrodes and propose a deep learning method to automatically extract the spatial features that characterize the functional relation between EEG signals at different electrodes. Our proposed deep model is called ATtention-based LSTM with Domain Discriminator (ATDD-LSTM) that can characterize nonlinear relations among EEG signals of different electrodes. To achieve state-of-the-art emotion recognition performance, the architecture of ATDD-LSTM has two distinguishing characteristics: (1) By applying the attention mechanism to the feature vectors produced by LSTM, ATDD-LSTM automatically selects suitable EEG channels for emotion recognition, which makes the learned model concentrate on the emotion related channels in response to a given emotion; (2) To minimize the significant feature distribution shift between different sessions and/or subjects, ATDD-LSTM uses a domain discriminator to modify the data representation space and generate domain-invariant features. We evaluate the proposed ATDD-LSTM model on three public EEG emotional databases (DEAP, SEED and CMEED) for emotion recognition. The experimental results demonstrate that our ATDD-LSTM model achieves superior performance on subject-dependent (for the same subject), subject-independent (for different subjects) and cross-session (for the same subject) evaluation.

   doi: 10.1109/TAFFC.2020.3013711

   @article{du20_taffc, title = {An Efficient LSTM Network for Emotion Recognition from Multichannel EEG Signals}, author = {Du, Xiaobing and Ma, Cuixia and Zhang, Guanhua and Li, Jinyao and Lai, Yu-Kun and Zhao, Guozhen and Deng, Xiaoming and Liu, Yong-Jin and Wang, Hongan}, year = {2020}, journal = {IEEE Transactions on Affective Computing (TAFFC)}, doi = {10.1109/TAFFC.2020.3013711}, pages = {1-12} }

1. 

   A Review of EEG Features for Emotion Recognition (in Chinese)

   Guanhua Zhang, Minjing Yu, Guo Chen, Yiheng Han, Dan Zhang, Guozhen Zhao, Yong-Jin Liu

   SCIENTIA SINICA Informationis, 49 (9), pp. 1097-1118, 2019.

   Abstract Links BibTeX Project

   Emotion recognition is an important research topic in the human-machine interaction field, and it can be applied to medicine, education, psychology, military, and other areas. Electroencephalogram (EEG) signals are mostly used among various indices of emotion recognition. High accuracy of emotion classifiers can be achieved by extracting the most relevant and discriminant features of emotion states. This study surveys EEG features that are extensively used in current emotion recognition studies by introducing EEG features from the following four viewpoints: time domain, frequency domain, time–frequency domain, and space domain. An SLDA algorithm is imported to three public EEG-emotion datasets (SEED, DREAMER, and CAS-THU) to evaluate feature capabilities that distinguish emotion valence. Existing problems and future investigations are also discussed in this paper.

   doi: 10.1360/N112018-00337

   Paper: zhang19_ssi.pdf

   @article{zhang19_ssi, title = {A Review of EEG Features for Emotion Recognition (in Chinese)}, author = {Zhang, Guanhua and Yu, Minjing and Chen, Guo and Han, Yiheng and Zhang, Dan and Zhao, Guozhen and Liu, Yong-Jin}, year = {2019}, journal = {SCIENTIA SINICA Informationis}, doi = {10.1360/N112018-00337}, pages = {1097-1118}, volume = {49}, number = {9} }