-
摘要: 近期,隨著元宇宙研究重心轉向內容交流和社會互動,如何打破當前視聽媒體交互瓶頸成為了亟待解決的問題,使用腦機接口進行感官模擬就是解決方案之一. 目前,腦機接口已經作為生理信號采集工具在元宇宙諸多領域表現出了不可替代的應用潛力. 研究三個應用場景,元宇宙藝術中的生成藝術、元宇宙醫學中醫療保健嚴肅游戲、元宇宙虛擬社會中的虛擬人表情合成中腦機接口應用現狀,調查已經存在的商業產品和專利(MindWave Mobile、GVS、Galea ),類比網絡安全和神經安全、生物倫理學和神經倫理學的發展過程,探討腦機接口成熟并被廣泛應用后可能面臨的挑戰和潛在問題,展望未來腦機接口在元宇宙中深入多樣應用的可能性.Abstract: The brain–machine interface has been an integral component of the metaverse since the inception of the latter, in his classic science fiction novel “True Names,” Vernor Vinge, the American mathematician and computer science Professor, describes a virtual world that can be accessed and experienced via a brain–machine interface. Following the introduction of this idea, the science fiction novel “Avalanche” formally proposed the concept of a metaverse, where a virtual world constructed by humans using digital technology can be mapped onto and interact with the real world. Large companies such as Meta, Apple, Sony, Microsoft, and Samsung have launched new metaverse-related hardware and software products. Domestic giants such as Tencent, Alibaba, and Baidu have also integrated themselves into the metaverse, confirming its future development and commercial value. Goldman Sachs estimates that trillions of dollars will be invested in the development of the metaverse over the next few years. As the focus of metaverse research shifts toward content exchange and social interaction, the issue of addressing the current bottlenecks in audiovisual media interaction has become an urgent matter, and the brain–computer interface is one of its solutions. Brain–computer interfaces are becoming increasingly complex. As a physiological signal acquisition tool, it has demonstrated indispensable application potential in numerous fields of the metaverse. A non-invasive brain–computer interface possesses the advantages of being easy to obtain and having good performance and accuracy. It is the preferred method for detecting brain signals in brain–computer interfaces. The Electroencephalogram is a unique physiological signal conducive to reflecting people's psychological state. By reading and categorizing the relevant papers in the paper database, including Web of Science, CNKI, IEL, and ACM Digital Library; investigating the products and functional parameters of Neuralink, Synchron, OpenBCI, and Emotiv; studying three application scenarios, namely, the generative art in the metaverse art, the serious game of medicine and healthcare in the medical metaverse, and the application status of the brain–machine interface in virtual human expression synthesis in the social metaverse; and by investigating the existing commercial products and patents (MindWave Mobile, GVS, Galea), this paper discusses the challenges and potential problems that brain–computer interfaces may face with their widespread use by drawing parallels with the development process of network and neural security and bioethics. Furthermore, the possibility of in-depth and diverse applications of brain–computer interfaces in the future is explored, for instance, the use of sensory simulation technology to simulate olfactory sensation, gustatory sense, and tactile sensation, and the use of motor imagery to assist disabled people in participating in the metaverse.
-
Key words:
- brain–computer interface /
- metaverse /
- human-computer interaction /
- generative art /
- serious game /
- affective computing /
- neurosecurity /
- neuroethics
-
表 1 5個流行社交虛擬平臺
Table 1. Five popular social virtual platforms
Platform Company Status Initial release Genre AltspaceVR Microsoft Shut down May 2015 Virtual social community Horizon Worlds Meta Operating December 2021 Game creation system, massively multiplayer online Mozilla Hubs Mozilla Operating April 2014 Virtual collaboration platform Rec Room Rec Room Operating June 2016 Game creation system, massively multiplayer online VRChat VRChat Operating January 2014 Massively multiplayer online 表 2 神經技術商業化產品
Table 2. Neurotechnology commercialization products
Product Company Price/$ Outputs (bit) Sampling rate/Hz Channel Application MindWave Mobile 2 NeuroSky 109.99 12 512 1 Games, education, wellness, research, develop Ganglion Board OpenBCI 499.00 24 200 4 Research, develop Cyton Biosensing Board OpenBCI 999.00 24 250 8 Research, develop Cyton+Daisy Biosensing Boards OpenBCI 1999.00 24 125/250 16 Research, develop Insight2 Emotiv 499.00 16 128 5 Research, develop EPOC X Emotiv 849.00 14/16 128/256 14 Research, develop EPOC Flex Emotiv 1699.00 /2099.00 14/16 128 32 Research, develop www.77susu.com -
參考文獻
[1] Stephenson N. Snow Crash. New York: Bantam Dell, 2003. [2] Wang W X, Zhou F, Wan Y L, et al. A survey of metaverse technology. Chin J Eng, 2022, 44(4): 744 doi: 10.3321/j.issn.1001-053X.2022.4.bjkjdxxb202204025王文喜, 周芳, 萬月亮, 等. 元宇宙技術綜述. 工程科學學報, 2022, 44(4):744 doi: 10.3321/j.issn.1001-053X.2022.4.bjkjdxxb202204025 [3] Wolpaw J R, Birbaumer N, McFarland D J, et al. Brain-computer interfaces for communication and control. Clin Neurophysiol, 2002, 113(6): 767 doi: 10.1016/S1388-2457(02)00057-3 [4] Wolpaw J R, Wolpow E W. Brain-Computer Interfaces. Oxford: Oxford University Press, 2012 [5] Aricò P, Sciaraffa N, Babiloni F. Brain-computer interfaces: Toward a daily life employment. Brain Sci, 2020, 10(3): 157 doi: 10.3390/brainsci10030157 [6] Wang Z H, Yu Y, Xu M, et al. Towards a hybrid BCI gaming paradigm based on motor imagery and SSVEP. Int J Human–computer Interact, 2019, 35(3): 197 doi: 10.1080/10447318.2018.1445068 [7] Cheng Q M. Structural forms of metaverse and the characteristics of metaverse art. J Zhejiang Shuren Univ, 2022, 22(4): 78 doi: 10.3969/j.issn.1671-2714.2022.04.010成喬明. 元宇宙的結構形態與藝術特征探析. 浙江樹人大學學報, 2022, 22(4):78 doi: 10.3969/j.issn.1671-2714.2022.04.010 [8] Christiane Paul. A Companion to Digital Art. New Jersey: Wiley-Blackwell, 2016: 146. [9] Lin C T, Rajapakse R P C J, Tokuyama Y. Development of EEG data-driven generative art application for real-time and dynamic interaction. J Robotics Netw Artif Life, 2021, 8(2): 117 doi: 10.2991/jrnal.k.210713.010 [10] Lee L H, Lin Z J, Hu R, et al. When creators meet the metaverse: A survey on computational arts [J/OL]. arXiv preprint (2021-11-26) [2023-2-15].https://arxiv.org/abs/2111.13486 [11] Gao F. Meta-universe—a new ecology for the future development of computing art. Art Obs, 2022(4): 24 doi: 10.3969/j.issn.1006-8899.2022.04.009高峰. 元宇宙—未來計算藝術發展的新生態. 美術觀察, 2022(4):24 doi: 10.3969/j.issn.1006-8899.2022.04.009 [12] Sun M T, Yang D W, Xie L S, et al. Research progress on the application of metaverse in chronic diseases health management. Fudan Univ J Med Sci, https://kns.cnki.net/kcms/detail/31.1885.R.20230312.2036.006.html孫夢婷, 楊達偉, 謝林杉, 等. 元宇宙醫學在慢性疾病健康管理中應用的研究進展. 復旦學報(醫學版), https://kns.cnki.net/kcms/detail/31.1885.R.20230312.2036.006.html [13] Susi T, Johannesson M, Backlund P. Serious Games: An Overview. Skovde: Institutionen for Kommunikation Och Information, 2007: 28 [14] Hepburn S. Early Intensive Behavioral Intervention (EIBI). Encyclopedia of Autism Spectrum Disorders. 2nd Ed. London: Springer International Publishing, 2021 [15] Mesa-Gresa P, Gil-Gómez H, Lozano-Quilis J A, et al. Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: An evidence-based systematic review. Sensors, 2018, 18(8): 2486 doi: 10.3390/s18082486 [16] Bozgeyikli L, Raij A, Katkoori S, et al. A survey on virtual reality for individuals with autism spectrum disorder: Design considerations. IEEE Trans Learn Technol, 2018, 11(2): 133 doi: 10.1109/TLT.2017.2739747 [17] Chen J, Du Y S. Advances on the application of immersive virtual reality technology in the treatment of autism spectrum disorders. Chin J Child Health, 2020, 28(5): 543 doi: 10.11852/zgetbjzz2019-0615陳靜, 杜亞松. 沉浸式虛擬現實技術在孤獨癥譜系障礙治療中的應用進展. 中國兒童保健雜志, 2020, 28(5):543 doi: 10.11852/zgetbjzz2019-0615 [18] Cevette M J, Stepanek J, Cocco D, et al. Oculo-vestibular recoupling using galvanic vestibular stimulation to mitigate simulator sickness. Aviat Space Environ Med, 2012, 83(6): 549 doi: 10.3357/ASEM.3239.2012 [19] Oberman L M, Hubbard E M, Mccleery J P, et al. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Cognitive Brain Research, 2005, 24(2): 190 doi: 10.1016/j.cogbrainres.2005.01.014 [20] Livingston L A, Colvert E, Bolton P, et al. Good social skills despite poor theory of mind: Exploring compensation in autism spectrum disorder. J Child Psychol Psychiatry, 2019, 60(1): 102 doi: 10.1111/jcpp.12886 [21] Robles M, Namdarian N, Otto J, et al. A virtual reality based system for the screening and classification of autism. IEEE Trans Vis Comput Graph, 2022, 28(5): 2168 doi: 10.1109/TVCG.2022.3150489 [22] Cheng R Z, Wu N, Varvello M, et al. Are we ready for metaverse? A measurement study of social virtual reality platforms // Proceedings of the 22nd ACM Internet Measurement Conference. Nice, 2022: 504 [23] Song T F, Zheng W M, Song P, et al. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput, 2020, 11(3): 532 doi: 10.1109/TAFFC.2018.2817622 [24] Allison B, Luth T, Valbuena D, et al. BCI demographics: How many (and what kinds of) people can use an SSVEP BCI? IEEE Trans Neural Syst Rehabil Eng, 2010, 18(2): 107 [25] Picard R W. Affective Computing. Cambridge: MIT press, 2000 [26] Bernal S L, Celdrán A H, Pérez G M, et al. Security in brain-computer interfaces: State-of-the-art, opportunities, and future challenges. ACM Comput Surv, 2020, 54(1): 1 [27] Denning T, Matsuoka Y, Kohno T. Neurosecurity: Security and privacy for neural devices. Neurosurg Focus, 2009, 27(1): E7 doi: 10.3171/2009.4.FOCUS0985 [28] Hu R, Zhang L Q, Meng P, et al. The neural responses of visual complexity in the oddball paradigm: An ERP study. Brain Sci, 2022, 12(4): 447 doi: 10.3390/brainsci12040447 [29] Fisher R S, Acharya J N, Baumer F M, et al. Visually sensitive seizures: An updated review by the Epilepsy Foundation. Epilepsia, 2022, 63(4): 739 doi: 10.1111/epi.17175 [30] Chiong W. Insiders and outsiders: Lessons for neuroethics from the history of bioethics. AJOB Neurosci, 2020, 11(3): 155 doi: 10.1080/21507740.2020.1778118 [31] Carvalho C, Gaspar A, Knight A, et al. Ethical and scientific pitfalls concerning laboratory research with non-human primates, and possible solutions. Animals, 2018, 9(1): 12 doi: 10.3390/ani9010012 [32] Ning H S, Tian Q H, Li S. Metaverse social governance. J Chongqing Univ Posts and Telecommun Soc Sci Ed, https://kns.cnki.net/kcms/detail/50.1180.C.20221123.1513.004.html寧煥生, 田巧惠, 李莎. 元宇宙社會治理. 重慶郵電大學學報(社會科學版), https://kns.cnki.net/kcms/detail/50.1180.C.20221123.1513.004.html [33] Naughton J. Are big tech’s efforts to show it cares about data ethics another diversion? The Guardian, 2019, 7: 1 [34] Al-Saegh A, Dawwd S A, Abdul-Jabbar J M. Deep learning for motor imagery EEG-based classification: A review. Biomed Signal Process Control, 2021, 63: 102172 doi: 10.1016/j.bspc.2020.102172 [35] Shi F F, Zhou F, Liu H, et al. Survey and tutorial on hybrid human-artificial intelligence. Tsinghua Sci Technol, 2022, 28(3): 486 [36] Panagiotakopoulos D, Marentakis G, Metzitakos R, et al. Digital scent technology: Toward the Internet of senses and the metaverse. Prof, 2022, 24(3): 52 [37] Ganzer P D, Colachis S C, Schwemmer M A, et al. Restoring the sense of touch using a sensorimotor demultiplexing neural interface. Cell, 2020, 181(4): 763 doi: 10.1016/j.cell.2020.03.054 [38] Brooks J, Teng S Y, Wen J, et al. Stereo-smell via electrical trigeminal stimulation // Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Yokohama, 2021: 1 -