|
|
|
题名
|
作者
|
年代
|
出处
|
被引量
|
| 1 | Digital assembly technology based on augmented reality and digital twins: a review显示文摘Product assembly simulation is considered as one of the key technologies in the process of complex product design and manufacturing.Virtual assembly realizes the assembly process design,verification,and optimization of complex products in the virtual environment,which plays an active and effective role in improving the assembly quality and efficiency of complex products.In recent years,augmented reality(AR)and digital twin(DT)technology have brought new opportunities and challenges to the digital assembly of complex products owing to their characteristics of virtual reality fusion and interactive control.This paper expounds the concept and connotation of AR,enumerates a typical AR assembly system structure,analyzes the key technologies and applications of AR in digital assembly,and notes that DT technology is the future development trend of intelligent assembly research. | Chan QIU Shien ZHOU Zhenyu LIU Qi GAO Jianrong TAN | 2019 | Virtual Reality & Intelligent Hardware2019，1，6： | 7 |
| 2 | Applications and research trends of digital human models in the manufacturing industry显示文摘Virtual reality(VR)has been widely used in various manufacturing industries,and VR-based virtual manufacturing has received significant attention in the current intelligent manufacturing era.Digital human models(DHMs)are essential for virtual manufacturing applications.Additionally,researching new applications of DHMs has developed into an important academic research field.This paper aims to identify the applications and research trends of DHMs in the manufacturing industry and to provide a reference for the continued development of virtual manufacturing and DHMs.We selected a total of 49 related articles from a large number of articles published between 2014 and 2019.The applications of DHMs in the manufacturing industry are analyzed from different perspectives and various relevant technical limitations are discussed.The results indicate that the applications of DHMs differ significantly between different types of fields.The automotive industry is the main application field for DHMs,and assembly/maintenance simulations and evaluations are the main application types.Additionally,there are still some limitations in the establishment of virtual environments,motion control,and DHM evaluation that should be addressed.Finally,research trends in the application of DHMs are illustrated and discussed,including the planning and assessment of human-robot collaboration systems,the combination of DHMs and augmented reality,and improved motion planning for DHMs.In summary,the application of DHMs can improve the realism and effectiveness of virtual manufacturing,and DHMs will be more widely and deeply studied and applied in various manufacturing industries in the near future. | Wenmin ZHU Xiumin FAN Yanxin ZHANG | 2019 | Virtual Reality & Intelligent Hardware2019，1，6： | 5 |
| 3 | A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection显示文摘Background Assembly guided by paper documents is the most widespread type used in the process of aircraft cable assembly.This process is very complicated and requires assembly workers with high-level skills.The technologies of wearable Augmented Reality(AR)and portable visual inspection can be exploited to improve the efficiency and the quality of cable assembly.Methods In this study,we propose a smart assistance system for cable assembly that combines wearable AR with portable visual inspection.Specifically,a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets that are installed on aircraft airframes.A Convolutional Neural Network(CNN)is then developed to read the texts on cables after images are acquired from the camera of the wearable AR device.An authoring tool that was developed to create and manage the assembly process is proposed to realize visual guidance of the cable assembly process based on a wearable AR device.The system is applied to cable assembly on an aircraft bulkhead prototype.Results The results show that this system can recognize the number,types,and locations of brackets,and can correctly read the text of aircraft cables.The authoring tool can assist users who lack professional programming experience in establishing a process plan,i.e.,assembly outline based on AR for cable assembly.Conclusions The system can provide quick assembly guidance for aircraft cable with texts,images,and a 3 D model.It is beneficial for reducing the dependency on paper documents,labor intensity,and the error rate. | Lianyu ZHENG Xinyu LIU Zewu AN Shufei LI Renjie ZHANG | 2020 | Virtual Reality & Intelligent Hardware2020，2，1： | 5 |
| 4 | Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality显示文摘Although VSLAM/VISLAM has achieved great success,it is still difficult to quantitatively evaluate the localization results of different kinds of SLAM systems from the aspect of augmented reality due to the lack of an appropriate benchmark.For AR applications in practice,a variety of challenging situations(e.g.,fast motion,strong rotation,serious motion blur,dynamic interference)may be easily encountered since a home user may not carefully move the AR device,and the real environment may be quite complex.In addition,the frequency of camera lost should be minimized and the recovery from the failure status should be fast and accurate for good AR experience.Existing SLAM datasets/benchmarks generally only provide the evaluation of pose accuracy and their camera motions are somehow simple and do not fit well the common cases in the mobile AR applications.With the above motivation,we build a new visual-inertial dataset as well as a series of evaluation criteria for AR.We also review the existing monocular VSLAM/VISLAM approaches with detailed analyses and comparisons.Especially,we select 8 representative monocular VSLAM/VISLAM approaches/systems and quantitatively evaluate them on our benchmark.Our dataset,sample code and corresponding evaluation tools are available at the benchmark website http://gffzz3e1cca23d9984cb3hp5bowoxbx56p6kux.ffgz.tsg.suse.edu.cn/eval-vislam/. | Jinyu LI Bangbang YANG Danpeng CHEN Nan WANG Guofeng ZHANG Hujun BAO | 2019 | Virtual Reality & Intelligent Hardware2019，1，4： | 3 |
| 5 | Multimodal teaching,learning and training in virtual reality:a review and case study显示文摘It is becoming increasingly prevalent in digital learning research to encompass an array of different meanings,spaces,processes,and teaching strategies for discerning a global perspective on constructing the student learning experience.Multimodality is an emergent phenomenon that may influence how digital learning is designed,especially when employed in highly interactive and immersive learning environments such as Virtual Reality(VR).VR environments may aid students'efforts to be active learners through consciously attending to,and reflecting on,critique leveraging reflexivity and novel meaning-making most likely to lead to a conceptual change.This paper employs eleven industrial case-studies to highlight the application of multimodal VR-based teaching and training as a pedagogically rich strategy that may be designed,mapped and visualized through distinct VR-design elements and features.The outcomes of the use cases contribute to discern in-VR multimodal teaching as an emerging discourse that couples system design-based paradigms with embodied,situated and reflective praxis in spatial,emotional and temporal VR learning environments. | Stéphanie PHILIPPE Alexis DSOUCHET Petros LAMERAS Panagiotis PETRIDIS Julien CAPORAL Gildas COLDEBOEUF Hadrien DUZAN | 2020 | Virtual Reality & Intelligent Hardware2020，2，5： | 3 |
| 6 | Multimodal interaction design and application in augmented reality for chemical experiment显示文摘Background Augmented reality classrooms have become an interesting research topic in the field of education,but there are some limitations.Firstly,most researchers use cards to operate experiments,and a large number of cards cause difficulty and inconvenience for users.Secondly,most users conduct experiments only in the visual modal,and such single-modal interaction greatly reduces the users'real sense of interaction.In order to solve these problems,we propose the Multimodal Interaction Algorithm based on Augmented Reality(ARGEV),which is based on visual and tactile feedback in Augmented Reality.In addition,we design a Virtual and Real Fusion Interactive Tool Suite(VRFITS)with gesture recognition and intelligent equipment.Methods The ARGVE method fuses gesture,intelligent equipment,and virtual models.We use a gesture recognition model trained by a convolutional neural network to recognize the gestures in AR,and to trigger a vibration feedback after a recognizing a five finger grasp gesture.We establish a coordinate mapping relationship between real hands and the virtual model to achieve the fusion of gestures and the virtual model.Results The average accuracy rate of gesture recognition was 99.04%.We verify and apply VRFITS in the Augmented Reality Chemistry Lab(ARCL),and the overall operation load of ARCL is thus reduced by 29.42%,in comparison to traditional simulation virtual experiments.Conclusions We achieve real-time fusion of the gesture,virtual model,and intelligent equipment in ARCL.Compared with the NOBOOK virtual simulation experiment,ARCL improves the users'real sense of operation and interaction efficiency. | Mengting XIAO Zhiquan FENG Xiaohui YANG Tao XU Qingbei GUO | 2020 | Virtual Reality & Intelligent Hardware2020，2，4： | 3 |
| 7 | Self-attention transfer networks for speech emotion recognition显示文摘Background A crucial element of human-machine interaction,the automatic detection of emotional states from human speech has long been regarded as a challenging task for machine learning models.One vital challenge in speech emotion recognition(SER)is learning robust and discriminative representations from speech.Although machine learning methods have been widely applied in SER research,the inadequate amount of available annotated data has become a bottleneck impeding the extended application of such techniques(e.g.,deep neural networks).To address this issue,we present a deep learning method that combines knowledge transfer and self-attention for SER tasks.Herein,we apply the log-Mel spectrogram with deltas and delta-deltas as inputs.Moreover,given that emotions are time dependent,we apply temporal convolutional neural networks to model the variations in emotions.We further introduce an attention transfer mechanism,which is based on a self-attention algorithm to learn long-term dependencies.The self-attention transfer network(SATN)in our proposed approach takes advantage of attention transfer to learn attention from speech recognition,followed by transferring this knowledge into SER.An evaluation built on Interactive Emotional Dyadic Motion Capture(IEMOCAP)dataset demonstrates the effectiveness of the proposed model. | Ziping ZHAO Keru Wang Zhongtian BAO Zixing ZHANG Nicholas CUMMINS Shihuang SUN Haishuai WANG Jianhua TAO Björn WSCHULLER | 2021 | Virtual Reality & Intelligent Hardware2021，3，1： | 3 |
| 8 | Three-dimensional virtual-real mapping of aircraft autom atic spray operation and online simulation monitoring显示文摘Background This study aims at addressing the lack of closed-loop feedback optimization enabling tool in aircraft automatic spraying systems;we systematically analyze a three-dimensional(3D)virtual-real mapping technique,namely the digital twin technique,used by the automatic spraying system.Methods With the sensors installed in the spraying system,the spraying working parameters are collected online and are used for driving the 3D virtual spraying system to realize the total-factor monitoring of the spraying operation.Furthermore,the operation-evaluation model is applied for analyzing and managing the key indexes of the spraying quality;once the data value of the key indexes exceeds a threshold,the operation shall be optimized automatically.Results This approach can effectively support the high-efficiency analysis,evaluation,and optimization of the spraying process. | Shiguang QIU Shuntao LIU Deshuai KONG Qichang HE | 2019 | Virtual Reality & Intelligent Hardware2019，1，6： | 3 |
| 9 | A survey on image and video stitching显示文摘Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions. | Wei LYU Zhong ZHOU Lang CHEN Yi ZHOU | 2019 | Virtual Reality & Intelligent Hardware2019，1，1： | 3 |
| 10 | Gesture interaction in virtual reality显示文摘With the development of virtual reality(VR)and human-computer interaction technology,how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research.Gesture is one of the most important communication methods of human beings,which can effectively express users'demands.In the past few decades,gesture-based interaction has made significant progress.This article focuses on the gesture interaction technology and discusses the definition and classification of gestures,input devices for gesture interaction,and gesture interaction recognition technology.The application of gesture interaction technology in virtual reality is studied,the existing problems in the current gesture interaction are summarized,and the future development is prospected. | Yang LI Jin HUANG Feng TIAN Hong-An WANG Guo-Zhong DAI | 2019 | Virtual Reality & Intelligent Hardware2019，1，1： | 3 |
| 11 | Prospects and challenges in augmented reality displays显示文摘Augmented reality(AR)displays are attracting significant attention and efforts.In this paper,we review the adopted device configurations of see-through displays,summarize the current development status and highlight future challenges in micro-displays.A brief introduction to optical gratings is presented to help understand the challenging design of grating-based waveguide for AR displays.Finally,we discuss the most recent progress in diffraction grating and its implications. | Yun-Han LEE Tao ZHAN Shin-Ison WU | 2019 | Virtual Reality & Intelligent Hardware2019，1，1： | 2 |
| 12 | Orthodontic simulation system with force feedback for training complete bracket placement procedures显示文摘Background A virtual system that simulates the complete process of orthodontic bracket placement can be used for pre-clinical skill training to help students gain confidence by performing the required tasks on a virtual patient.Methods The hardware for the virtual simulation system is built using two force feedback devices to support bi-manual force feedback operation.A 3D mouse is used to adjust the position of the virtual patient.A multi-threaded computational methodology is adopted to satisfy the requirements of the frame rate.The computation threads mainly consist of the haptic thread running at a frequency of>1000Hz and the graphic thread at>30Hz.The graphic thread allows the graphics engine to effectively display the visual effects of biofilm removal and acid erosion through texture mapping.Using the haptic thread,the physics engine adopts the hierarchy octree collision-detection algorithm to simulate the multi-point and multi-region interaction between the tools and the virtual environment.Its high efficiency guarantees that the time cost can be controlled within 1 ms.The physics engine also performs collision detection between the tools and particles,making it possible to simulate paint and removal of colloids.The surface-contact constraints are defined in the system;this ensures that the bracket will not divorce from or embed into the tooth during the adjustment of the bracket.Therefore,the simulated adjustment is more realistic and natural.Results A virtual system to simulate the complete process of orthodontic bracket bonding was developed.In addition to bracket bonding and adjustment,the system simulates the necessary auxiliary steps such as smearing,acid etching,and washing.Furthermore,the system supports personalized case training.Conclusions The system provides a new method for students to practice orthodontic skills. | Fan YE Luwei LIU Bin YAN Xiaohan ZHAO Aimin HAO | 2021 | Virtual Reality & Intelligent Hardware2021，3，4： | 2 |
| 13 | Haptic display for virtual reality: progress and challenges显示文摘Immersion, interaction, and imagination are three features of virtual reality (VR). Existing VR systems possess fairly realistic visual and auditory feedbacks, and however, are poor with haptic feedback, by means of which human can perceive the physical world via abundant haptic properties. Haptic display is an interface aiming to enable bilateral signal communications between human and computer, and thus to greatly enhance the immersion and interaction of VR systems. This paper surveys the paradigm shift of haptic display occurred in the past 30 years, which is classified into three stages, including desktop haptics, surface haptics, and wearable haptics. The driving forces, key technologies and typical applications in each stage are critically reviewed. Toward the future high-fidelity VR interaction, research challenges are highlighted concerning handheld haptic device, multimodal haptic device, and high fidelity haptic rendering. In the end, the importance of understanding human haptic perception for designing effective haptic devices is addressed. | Dangxiao WANG Yuan GUO Shiyi LIU Yuru ZHANG Weiliang XU Jing XIAO | 2019 | Virtual Reality & Intelligent Hardware2019，1，2： | 2 |
| 14 | Deep learning based point cloud registration:an overview显示文摘Point cloud registration aims to find a rigid transformation for aligning one point cloud to another.Such registration is a fundamental problem in computer vision and robotics,and has been widely used in various applications,including 3D reconstruction,simultaneous localization and mapping,and autonomous driving.Over the last decades,numerous researchers have devoted themselves to tackling this challenging problem.The success of deep learning in high-level vision tasks has recently been extended to different geometric vision tasks.Various types of deep learning based point cloud registration methods have been proposed to exploit different aspects of the problem.However,a comprehensive overview of these approaches remains missing.To this end,in this paper,we summarize the recent progress in this area and present a comprehensive overview regarding deep learning based point cloud registration.We classify the popular approaches into different categories such as correspondences-based and correspondences-free approaches,with effective modules,i.e.,feature extractor,matching,outlier rejection,and motion estimation modules.Furthermore,we discuss the merits and demerits of such approaches in detail.Finally,we provide a systematic and compact framework for currently proposed methods and discuss directions of future research. | Zhiyuan ZHANG Yuchao DAI Jiadai SUN | 2020 | Virtual Reality & Intelligent Hardware2020，2，3： | 2 |
| 15 | Serious games in science education:a systematic literature review显示文摘Teaching science through computer games,simulations,and artificial intelligence(AI)is an increasingly active research field.To this end,we conducted a systematic literature review on serious games for science education to reveal research trends and patterns.We discussed the role of virtual reality(VR),AI,and augmented reality(AR)games in teaching science subjects like physics.Specifically,we covered the research spanning between 2011 and 2021,investigated country-wise concentration and most common evaluation methods,and discussed the positive and negative aspects of serious games in science education in particular and attitudes towards the use of serious games in education in general. | Mohib ULLAH Sareer Ul AMIN Muhammad MUNSIF Muhammad Mudassar YAMIN Utkurbek SAFAEV Habib KHAN Salman KHAN Habib ULLAH | 2022 | Virtual Reality & Intelligent Hardware2022，4，3： | 2 |
| 16 | A review of cable layout design and assembly simulation in virtual environments显示文摘The layout and assembly of flexible cables play important roles in the design and development of complex electromechanical products.The rationality of cable layout design and the reliability of cable assembly greatly affect product quality.In this paper,we review the methods of cable layout design,cable assembly process planning,and cable assembly simulation.We first review research on flexible cable layout design(both interactive and automatic).Then,research on the cable assembly process planning,including cable assembly path and manipulation planning,is reviewed.Finally,cable assembly simulation is introduced,which includes general cable information,cable collision detection data,and cable assembly process modeling.Current problems and future research directions are summarized at the end of the paper. | Xiaodong YANG Jianhua LIU Naijing LV Huanxiong XIA | 2019 | Virtual Reality & Intelligent Hardware2019，1，6： | 2 |
| 17 | Survey of 3D modeling using depth cameras显示文摘Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera. | Hantong XU Jiamin XU Weiwei XU | 2019 | Virtual Reality & Intelligent Hardware2019，1，5： | 2 |
| 18 | Gesture-based target acquisition in virtual and augmented reality显示文摘Background Gesture is a basic interaction channel that is frequently used by humans to communicate in daily life. In this paper, we explore to use gesture-based approaches for target acquisition in virtual and augmented reality. A typical process of gesture-based target acquisition is: when a user intends to acquire a target, she performs a gesture with her hands, head or other parts of the body, the computer senses and recognizes the gesture and infers the most possible target. Methods We build mental model and behavior model of the user to study two key parts of the interaction process. Mental model describes how user thinks up a gesture for acquiring a target, and can be the intuitive mapping between gestures and targets. Behavior model describes how user moves the body parts to perform the gestures, and the relationship between the gesture that user intends to perform and signals that computer senses. Results In this paper, we present and discuss three pieces of research that focus on the mental model and behavior model of gesture-based target acquisition in VR and AR. Conclusions We show that leveraging these two models, interaction experience and performance can be improved in VR and AR environments. | Yukang YAN Xin YI Chun YU Yuanchun SHI | 2019 | Virtual Reality & Intelligent Hardware2019，1，3： | 2 |
| 19 | On attaining user-friendly hand gesture interfaces to control existing GUIs显示文摘Background Hand ge stur e interfaces are dedicated programs that principally perform hand tracking and hand gesture prediction to provide alternative controls and interaction methods.They take advantage of one of the most natural ways of interaction and communication,proposing novel input and showing great potential in the field of the human-computer interaction.Developing a flexible and rich hand gesture interface is known to be a time-consuming and arduous task.Previously published studies have demonstrated the significance of the finite-state-machine(FSM)approach when mapping detected gestures to GUI actions.Methods In our hand gesture interface,we broadened the FSM approach by utilizing gesture-specific attributes,such as distance between hands,distance from the camera,and time of occurrences,to enable users to perform unique GUI actions.These attributes are obtained from hand gestures detected by the RealSense SDK employed in our hand gesture interface.By means of these gesture-specific attributes,users can activate static gestures and perform them as dynamic gestures.We also provided supplementary features to enhance the efficiency,convenience,and user-friendliness of our hand gesture interface.Moreover,we developed a complementary application for recording hand gestures by capturing hand keypoints in depth and color images to facilitate the generation of hand gesture datasets.Results We conducted a small-scale user survey with fifteen subjects to test and evaluate our hand gesture interface.Anonymous feedback obtained from the users indicates that our hand gesture interface is adequately facile and self-explanatory to use.In addition,we received constructive feedback about minor flaws regarding the responsiveness of the interface.Conclusions We proposed a hand gesture interface along with key concepts to attain use r-friendlines s and effectiveness in the control of existing GUIs. | Egemen ERTUGRUL Ping LI Bin SHENG | 2020 | Virtual Reality & Intelligent Hardware2020，2，2： | 2 |
| 20 | Digital twin intelligent system for industrial internet of things-based big data management and analysis in cloud environments显示文摘This work surveys and illustrates multiple open challenges in the field of industrial Internet of Things(IoT)-based big data management and analysis in cloud environments.Challenges arising from the fields of machine learning in cloud infrastructures,artificial intelligence techniques for big data analytics in cloud environments,and federated learning cloud systems are elucidated.Additionally,reinforcement learning,which is a novel technique that allows large cloud-based data centers,to allocate more energy-efficient resources is examined.Moreover,we propose an architecture that attempts to combine the features offered by several cloud providers to achieve an energy-efficient industrial IoT-based big data management framework(EEIBDM)established outside of every user in the cloud.IoT data can be integrated with techniques such as reinforcement and federated learning to achieve a digital twin scenario for the virtual representation of industrial IoT-based big data of machines and room tem-peratures.Furthermore,we propose an algorithm for determining the energy consumption of the infrastructure by evaluating the EEIBDM framework.Finally,future directions for the expansion of this research are discussed. | Christos L.STERGIOU Kostas E.PSANNIS | 2022 | Virtual Reality & Intelligent Hardware2022，4，4： | 2 |
您所在的机构暂未开通此部分的功能，请联系管理员