Understanding how accurate people make decisions in complex tasks becomes more clear, thanks to new ways of combining brain signals and image information. The innovative study was recently published in the journal Scientific Reports. The study was conducted by Xuan-Tran, a PhD student on the collaborative committee of Professor Chin-Teng Lin, Professor Nikhil Pal, Professor Tzyy-Ping Jung and Dr. Thomas Do, who are affiliated with Sydney, an institution that includes the Technical University. , Institute of Statistics of India and University of California, San Diego
Researchers have created a framework that uses machine learning, an artificial intelligence that learns patterns from data that can work together to analyze brain activity and image details to predict whether a person will do it correctly in challenging tasks. response. This method uses any model (SAM) of the segment to identify and isolate objects in the image. It extracts features from the target object’s features and the relationship between the target object and the neighboring object to enhance prediction accuracy. Brain signals are collected using electroencephalography (EEG), a non-invasive technique. The functions extracted from the EEG data are then fused with the image features to further improve prediction accuracy. “This advancement highlights how combining information from the brain and images can improve our understanding of how people make decisions,” Lin explained.
In the study, participants were asked to find animals in pictures. The animals were camouflaged to make the task more difficult, emulating challenges similar to real-life situations. “Unlike studies where other participants could guess by chance, this setup makes people guess harder, thus better testing people’s mindsets and decisions,” Dr. Thomas explained. The researchers recorded electrical activity of the brain measured using an EEG, which captures brain signals through sensors placed on the scalp and analyzes them with image features to see how both influence decisions.
The results show that combining brain and image data is much better than using it alone. “This combination approach is significantly more accurate in predicting correct decisions than models that rely only on one type of data,” said lead author Xuan-tran. This highlights the convergence of multiple sources of information to Better understand the advantages of human behavior.
“This study not only helps predict the accuracy of decision making, but also provides a framework for designing systems that can alert potential errors before a user occurs. In key areas such as healthcare or defense, such systems may reach Crucially, avoiding mistakes can save lives.
A key element of success is the in-depth use of image functionality. The extracted features determine the relationship between objects in the picture and are transformed to integrate seamlessly with the EEG neural features. “Brain signals from regions known to be related to object detection and decision-making, such as the occipital and parietal wall areas, are responsible for processing sensory information and making decisions, playing an important role in the performance of the model.” The team found that training their model The effect of data from individual participants is better than training the combined data of the group, which shows how decisions vary from person to person.
By summarizing detailed brain activity analysis and complex image analysis, the study opens up exciting possibilities for developing systems that can predict people’s tasks in real time. The team plans to expand research by using more data and refining its models, making it more practical in everyday applications.
Journal Reference
Tran XT, Do T., Pal NR, Jung TP, Lin CT “Multi-mode fusion of predicting human decision-making performance.” Scientific Reports, 2024. doi: https://doi.org/10.1038/s41598-024-63651-2-2
About the Author
Chin-teng lin In 1986, Distinguished Professor Chin-teng Lin received a bachelor’s degree in science from the National Chiao-tung University (NCTU) from Taiwan, and received a master’s degree and a doctorate in electrical engineering from Purdue University in the United States, in 1989 and 1992 respectively. Obtained in 2018.
He is currently an outstanding professor at the School of Computer Science, director of the Centre for AI (HAI) of the Centre for Humanity and co-director of the Australian Institute of Artificial Intelligence (AAII), the School of Engineering and Information Technology, Sydney University of Technology. He is also honorary chairman of NCTU electrical and computer engineering. Due to his contribution to biology-inspired information systems, Professor Lin received an IEEE scholarship in 2005 and awarded a scholarship with the International Fuzzy Systems Association (IFSA) in 2012. He won the IEEE Fuzzy Systems Pioneer Award in 2017. From 2011 to 2016, IEEE Transactions, Editor-in-Chief of IEEE Transactions; IEEE Circuits and Systems (CAS) Association (2005-2008), IEEE Systems, Human, Cybernetics IEEE Society (SMC) Association (2003-2005), IEEE Seats of the Society of Computational Intelligence (2008-2010) (CAS) Association (2005-2008) (2008-2005) (2008-2010); Chairman of the IEEE TAIPEI Department (2009-2010); Chairman of the IEEE CIS Award Committee ( 2022, 2023); Outstanding Lecturer of IEEE CAS Association (2003-2005) and CIS Association (2015-2017); Chairman of the IEEE CIS Outstanding Lecturer Program Committee (2018-2019); Editor-in-chief of IEEE Transactions on Circuits and Systems-II ( 2006-2008); Chairman of the International Conference on International Conference on IEEE Systems, MAN and Cybernetics (2005); and President of the 2011 IEEE International Fuzzy Systems Conference.
Professor Lin is co-author of the Prentice-Hall and the author of the Neurofuzzy Control System with Structural and Parameter Learning (World Science). His 948 publications include 3 books; 28 chapters; 485 journal papers; and 432 guided conference papers, including approximately 232 IEEE journal papers, which are in neural networks, fuzzy systems, brain computer interfaces, Multimedia information processing, cognitive neuroengineering and human-computer combination teams have been cited more than 40,065 times. Currently, his H index is 96 and his i10 index is 464.
Nikil R. Parr He was a professor in the Department of Electronics and Communication Sciences and was the founding director of the Centre for Artificial Intelligence and Machine Learning at the Institute of Statistics in India. His current research interests include brain science, computing intelligence, machine learning, and data mining.
He is the editor-in-chief of the fuzzy system IEEE transaction from January 2005 to December 2010. He serves on the Editorial/Advisory Board/Steering Board/Historing Board of Editorial/Advisory Board/Steering Board of several journals, including the International Journal of Approximate Reasoning, Applied, Soft Computing, International Journal of Neurosyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy IEEE transactions on fuzzy systems and IEEE transactions on cybernetics.
He is the winner of the 2015 IEEE Computational Intelligence Association (CIS) Fuzzy Systems Pioneer Award and the 2021 IEEE CIS CIS Functional Service Award. He has delivered many plenary/key speeches at different premier international conferences in the field of computing intelligence. He has served as the President of several conferences, the President of Programmes and the Chairman of the Joint Programmes. He has been an outstanding lecturer at IEEE CIS (2010-2012, 2016-2018, 2022-2024) and is a member of the IEEE CIS Administrative Committee (2010-2012). He has served as Vice President of IEEE CIS (2013-2016) publications and President of IEEE CIS (2018-2019).
He is a member of the West Bengal College of Science and Technology, the College of Electronics and Telecommunications, the National Academy of Sciences Indian Academy of Sciences – India National Academy of Engineering, the National Academy of Sciences, the International Fuzzy Systems Association (IFSA), the World, the World Academy of Sciences and the US IEEE .
tzyy-ping Jung (S’91-M’92-SM’06-F’15) received a bachelor’s degree in electronic engineering from the national Chihao Tung University of Hsinchu, Taiwan in 1984 and MS and Ph.D in 1984. He received his degree in electrical engineering from Ohio State University in 1989 and 1993, respectively. He is currently the co-director of the Center for Advanced Neuroengineering and associate director of the Swartz Center for Computational Neuroscience at the University of California, San Diego. Additionally, he is an adjunct professor in the Department of Bioengineering at UC San Diego. Dr. Jung has expanded his academic contributions internationally, serving as an adjunct professorship at Tianjin University and the University of Science and Technology of China, as well as Taiwan’s national TSING HUA University and National tsing Hua University and National Yang Ming Chihao Tung University.
Dr. Jung pioneered the transformational technology for applying blind source separation to decompose multi-channel EEG, MEG, ERP and fMRI data. In recognition of his contribution to the blind separation of biomedical applications, he was promoted to IEEE Fellow in 2015. He is also a fellow member of the Asia-Pacific Artificial Intelligence Association (AAIA). Dr. Jung’s research highlights the integration of cognitive science, computer science and engineering, neuroscience, bioengineering and electrical engineering. His interdisciplinary work has been highly rated and striking by his peers, with about 47,000 citations h– According to Google Scholar, the index of 92.
Thomas does He is a senior lecturer and co-director at the Human Interaction (HAI) Centre at the University of Technology (UTS). He received his PhD in Computer Science from UTS, a master’s degree in Human Computing from the Korean Institute of Science and Technology.
His research focuses on the integration of artificial intelligence (AI), brain computer interface (BCI), human computer interactions and robotics technology, and emphasizes the use of BCI technology for assisted applications. Do Do’s vision is to bridge the gap between neuroengineering and practical real-life applications by developing cutting-edge AI-powered systems that translate brain signals into actionable outputs.
