iNSYNC

SynCocreate
SynCocreate is a VR-based interpersonal EEG neurofeedback co-creation platform. It offers a playful and engaging experience for co-creating artwork within a VR environment through synchronizing brainwaves between pairs of users. This generative VR platform enables paired individuals to interact via brainwaves in a 3D virtual canvas, painted and animated collaboratively through their real-time brainwave data. The platform employs synchronized visual cues, aligned with the real-time brainwaves of paired users, to investigate the potential of collaborative neurofeedback in enhancing co-creativity and emotional connection. It also explores the use of VR in fostering creativity and togetherness through immersive, collective visualizations of brainwaves.

The exploration of collaborative artwork and co-creation in human interactions is extensive, ranging from the classic exquisite corpse game to traditional forms like visual art, musical composition, and choreography [1].These practices are known to foster social wellbeing such as enhancing cooperation [2] and reinforcing bonds [3]. However, collaborative art creation based on mutual brainwave interaction remains relatively underexplored. Meanwhile, measurable biosignals, such as EEG, relay internal activities and emotional states. Sharing and synchronizing biosignals such as brainwaves bear prosocial potential for interaction [4], yet expressing and sharing these signals, especially in remote social contexts, remains challenging.

In SynCocreate, by harnessing the collaborative potential of neurofeedback, we offer a playful and engaging experience for co-creating artwork within a VR environment through synchronizing brainwaves between pairs of users. The experience unfolds within a dome that symbolizes the brain. Two spectral, smoke-like sprites dance in the space, each guided by the real-time brainwave signals of a user. As the users' brainwaves synchronize, a splash of paint materializes on the dome's surface like a firework. Each user’s environment incorporates visual cues — such as light, color, and pulsations synchronized with their paired partner’s brainwaves — to encourage closer brainwave alignment. With each synchronization, more paint is added, gradually creating a vibrant, unique painting on the dome's interior, resembling a sky filled with fireworks, celebrating their connection.

The scene is set in a starry dome arena designed to resemble the hemispherical shape of the brain, transporting users into an imaginative cognitive realm. This space is complemented by a rippling, reflective water surface, on which an array of sensors floats, metaphorically serving as conduits for streaming real-time data from an external source. Luminescent trails swim beneath the water, symbolizing transient thoughts. Above the water under the dome, two ethereal sprites, each representing a user's brain activity, dance dynamically.

Departing from conventional two-dimensional art canvases, the hemispherical dome serves as a three dimensional canvas for collaborative artwork driven by the users' brainwave data. This approach serves to reveal the subtleties of the users' brainwave activities and provides a tangible representation of their cognitive processes.

Previous studies have identified distinct patterns linking brainwave bands to specific brain regions: positiveemotions typically activate the β and γ bands in temporal areas, while negative emotions often result inheightened δ band activity in occipital regions [5]. Similarly, [6] noted variations in EEG power acrossemotional states induced by VR videos, emphasizing the α-band's significant role in the prefrontal region forpositive emotions compared to negative ones. In our scene, the mapping of sprite and paint stroke positions isan artistic interpretation of these findings, harmonized with symmetry rules, and adjusted to ensure overlapsamong the areas. Drawing on the relationship between brainwave band types and their primary brain activationregions, we have associated the β and γ bands with the brain's middle latitudinal area, the θ band with themiddle longitudinal area, the α-band with the frontal area, and the δ band with the rear area

Imagine a patient with early signs of dementia, coupled with social anxiety and mistrust towards a new caregiver.They step into this immersive starry brain dome together and observe as their neural patterns collaborativelypaint the dome by synchronizing their brainwaves. This experience offers them a unique opportunity to expresstheir mind states in a controlled, creative environment, fostering emotional comfort through non-verbalinteraction. As they witness the visual harmony created by their joint brain activity, they find a new way toconnect and understand each other beyond words.

References
[1] Karen Barbour, David Ratana, Clive Waititi, and Kent Walker. 2007. Researching collaborative artistic practice. Waikato Journal ofEducation 13 (2007), 49-76. https://researchcommons.waikato.ac.nz/handle/10289/6186
[2] Julie Van de Vyver and Dominic Abrams. 2018. The Arts as a Catalyst for Human Prosociality and Cooperation. Social Psychologicaland Personality Science 9, 6 (2018), 664-674. https://doi.org/10.1177/1948550617720275
[3] Melissa G. Bublitz, Tracy Rank-Christman, Luca Cian, Xavier Cortada, Adriana Madzharov, Vanessa M. Patrick, Laura A. Peracchio,Maura L. Scott, Aparna Sundar, Ngoc (Rita) To, and Claudia Townsend. 2019. Collaborative Art: A Transformational Force withinCommunities. J. Assoc. Consum. Res. 4, 4 (2019), 313-331. https://doi.org/10.1086/705023
[4] Yi Hu, Yinying Hu, Xianchun Li, Yafeng Pan, and Xiaojun Cheng. 2017. Brain-to-brain synchronization across two persons predicts mutualprosociality. Soc. Cogn. Affect. Neurosci. 12, 12 (Dec. 2017), 1835–1844. Published online 2017 Oct 13.
[5] Wei-Long Zheng, Jia-Yi Zhu, and Bao-Liang Lu. 2017. Identifying stable patterns over time for emotion recognition from EEG. In IEEETransactions on Affective Computing, Vol. 10, No. 3 (2017), IEEE, 417–429. https://doi.org/10.1109/taffc.2017.2712143
[6] Ming Li, Junjun Pan, Yang Gao, Yang Shen, Fang Luo, Ju Dai, Aimin Hao, and Hong Qin. 2022. Neurophysiological and subjectiveanalysis of VR emotion induction paradigm. In IEEE Transactions on Visualization and Computer Graphics, Vol. 28, No. 11 (2022), IEEE,3832–3842. https://doi.org/10.1109/TVCG.2022.3203099

Back to Home