eXtended Reality (XR) is an emerging umbrella term for all the immersive technologies (Virtual Reality, Augmented Reality, and Mixed Reality) able to extend the user senses by blending the virtual and real worlds and creating a fully immersive experience.
Artificial Intelligence (AI) is a broad area of computer science that makes machines seem like they have human intelligence. The goal of AI is to mimic the human brain and create systems that can function intelligently and independently.
The XR&AI Summer School is focused on the synergy of these technologies for cultural and territorial heritage, medicine and industry.
The School is open to Ph.D. and Master students, post-doctoral researchers, and academic or industrial researchers.
The XR&AI Summer School is a satellite event of the International Conference on eXtended Reality (XR Salento).
Take a look back at the previous schools:
- Pasquale Arpaia – Università di Napoli Federico II, Napoli (Italy)
- Lucio Tommaso De Paolis – Università del Salento, Lecce (Italy)
- Ugo Erra – Università della Basilicata, Potenza (Italy)
- Emanuele Frontoni – Università di Macerata (Italy)
- Nicola Masini – CNR-ISPC, Potenza (Italy)
- Roberto Pierdicca – Università Politecnica delle Marche, Ancona (Italy)
- Antonio Emmanuele Uva – Polytechnic University of Bari (Italy)
- Primo Zingaretti – Università Politecnica delle Marche, Ancona (Italy)
- Nicola Felice Capece - Università della Basilicata, Potenza (Italy)
- Carola Gatto - Università del Salento, Lecce (Italy)
- Gabriele Gilio - Università della Basilicata, Potenza (Italy)
- Nicla Notarangelo - Università della Basilicata, Potenza (Italy)
- Silke Miss – XRtechnology srl, Lecce (Italy)
- Ileana Riera Panaro – Università del Salento, Lecce (Italy)
Joaquim Jorge, University of Lisboa
Challenges and Opportunities for AI&XR in Health Applications
Rita Cucchiara, Università di Modena e Reggio Emilia
Generative AI from image to text and return
Andrea Bottino, Dipartimento di Automatica e Informatica, Politecnico di Torino
AI for procedure learning in XR
Joseph L. Gabbard, Virginia Tech, US
Implications of AI for next-generation AR user interfaces
LECTURE 1 - Challenges and Opportunities for AI&XR in Health Applications
Recent advances in VR and AR technology have enabled interactive graphics applications to support healthcare professionals in training, diagnosis, planning, and treatment. This field has progressed enough to warrant a course that can inspire new ideas within the graphics community. Medical images create virtual human anatomy models, allowing for natural interaction and visualization in healthcare scenarios. VR and AR are conceptually different and suited for different types of problems. VR is immersive and suitable for learning anatomy, surgical skills, and analyzing 3D medical data. AR, on the other hand, overlays helpful information onto the physical environment, making it useful for tasks such as communication with patients and training assistants. However, several challenges, such as nonstandard equipment and disorientation, still limit the widespread use of these technologies. This course will cover current advances and challenges in this area, including integrating AI techniques.
Joaquim Jorge holds the UNESCO Chair of Artificial Intelligence & Extended Reality at the University of Lisboa, Portugal. He joined Eurographics in 1986 and ACM/SIGGRAPH in 1989. He is Editor-in-Chief of the Computers and Graphics Journal, Eurographics Fellow, ACM Distinguished Member, and member of IEEE Computer Society Board of Governors. He organized 50+ conferences, including Eurographics 2016 (IPC CO-Chair), IEEE VR 2020/21/22 as co-(papers)chair, and ACM IUI 2012 (IPC co-chair). He served on 210+ program committees and (co)authored over 360 peer-reviewed publications and five books. His research interests include graphics, virtual reality, and advanced HCI techniques applied to health technologies.
LECTURE 3 - AI for procedure learning in XR
In many fields, procedures define work and activities. These procedures consist of well-defined sequences of instructions that are followed in a specific order and according to specific rules. For example, maintenance work in industry is defined by procedures that ensure that work is performed according to maintenance strategy, policies, and programs. However, these procedures can be complex and difficult to remember, requiring learning and training for new employees and refresher training for existing personnel. This is also true in other areas, such as learning safety and medical procedures, training employees, and improving decision-making processes. Procedures are used in almost everything we do, such as reading, writing, calculating, driving, cooking, and classifying concepts.
To teach procedural skills well, teachers need to provide activity-specific instructions that describe the logical connections between activities...according to the form of the procedure. Extended reality technologies (XR) can be used to create immersive and interactive learning environments that help learners acquire procedural knowledge and skills through self-training and self-assessment. XR technologies can simulate realistic environments that help learners practice procedures in a safe and controlled environment. For example, a trainee can practice complex medical procedures in a virtual environment before performing them on real patients. In addition, XR can enhance the learning experience by providing multimedia, interactive information, context awareness, and adaptive information delivery. Learners can interact with virtual objects, receive audio and visual feedback, and track their performance and progress in real time, allowing them to identify and correct errors.
The integration of artificial intelligence (AI) systems can significantly improve self-learning systems by making them more adaptable and tailored to learners' needs. In particular, AI enables the implementation of adaptive learning approaches that constantly monitor and analyze learners' performance and adapt content and learning methods to their needs and progress. Such adaptive learning approaches can be supported by a virtual instructor with conversational capabilities. This instructor has two main functions: providing vocal and visual instructions to deliver learning content, and answering to direct requests from learners with explanations and clarifications. This interaction is critical to maintaining motivation and enhancing the learning experience. Finally, AI can be used to control realistic NPCs that actively participate in complex learning scenarios.
This course on procedure learning in XR will discuss best practices for creating effective XR/AI-based learning programs and experiences that meet the needs of diverse learners and achieve specific learning objectives.
Andrea Bottino is Associate Professor at the Department of Control and Computer Engineering of the Politecnico di Torino, where he heads the Computer Graphics and Vision research group of the same university. His current research interests include Computer Vision, Machine Learning, Human Computer Interaction, Serious Games, and Virtual and Augmented Reality.
LECTURE 4 - Implications of AI for next-generation AR user interfaces
As we see augmented reality (AR) applications move from research labs to commercial applications, the need for usable AR-based systems has become more and more evident. Despite the fact that AR technology fundamentally changes the way we visualize, use, and interact with computer-based information, we are only just recently starting to see real investment in the design and development of user interface (UI) designs that truly leverage AR’s promise. While traditional UX methods can be applied to determine what information should be presented to users, these approaches do not tell us how and when information should be presented to the user. Given the infancy of the AR UI design field, we mostly see smartphone, tablet and even desktop UI metaphors migrating to AR, in what we can consider first-gen AR user interfaces.
With recent advancements in computer vision and machine learning, we have a unique opportunity to consider intelligent next-generation AR user interfaces. In industrial settings for example, we can consider these next-gen AR interfaces to adapt to, the worker, their work progress and quality, as well as the overall work environment. In this talk, I discuss the implications for AI for next-generation AR user interfaces. Specifically, I provide some specific examples of how AI-powered AR user interfaces could adapt to users’ contexts, and touch on both the promise and challenges associated with creating and fielding these next-gen UIs. I conclude with open questions regarding issues of inclusion, privacy and agency.
Dr. Joseph L. Gabbard is an Associate Professor of Human Factors at Virginia Tech. He holds Ph.D., M.S. and B.S. degrees in Computer Science (HCI) and a B.A. in Sociology from Virginia Tech. Dr. Gabbard’s work centers on human-computer interaction in augmented and virtual reality. For nearly 25 years, Gabbard has been researching new methods of design and evaluation for interactive virtual and augmented reality systems. Currently, Dr. Gabbard directs the COGnitive Engineering for Novel Technologies (COGENT) Lab, conducting basic and applied HCI and human factors research. His work focuses on the application of principles and theories from several disciplines to the design of augmented reality user interfaces, information presentation and user interaction. His research has been funded by the National Science Foundation, National Institute of Standards and Technology, Microsoft, Meta, the National Institutes of Health, the Office of Naval Research, and several automotive manufacturers.
Nicola Capece, Università della Basilicata, Italy
Exploring the Potential of Hand Tracking and Gesture Recognition through Neural Networks in Unity 3D
LECTURE 1 - Exploring the Potential of Hand Tracking and Gesture Recognition through Neural Networks in Unity 3D
This lesson introduces hand tracking and gesture recognition using neural networks, implemented with Keras and TensorFlow, and integrated into the Unity 3D game engine. Through practical exercises, students will learn the principles of neural network design and training and the implementation of hand tracking and gesture recognition in Unity 3D. By the end of the lesson, students will have a fundamental understanding of the potential of these technologies in interactive systems and the skills to implement them using Keras, TensorFlow, and Unity 3D.
Nicola Capece is a postdoctoral researcher at the University of Basilicata in Italy. His research interests encompass Real-Time and Offline Rendering, Deep Learning, Computer Graphics, eXtended Reality (XR), and Human-Computer Interaction (HCI). He has authored numerous papers published in international journals and conferences. He is involved in several research projects concerning XR and HCI for cultural heritage storytelling, virtual dressing rooms, virtual reality 3D modelling, and hand gesture recognition systems. Additionally, he is an adjunct professor of Software Engineering and Virtual Archeology Lab courses.
For the payment of the school participation fee is mandatory to carry out the school registration.
The XR&AI Summer School 2023 registration fee includes lectures, daily lunches and coffee breaks as well as the pizza party and the guided visit of the town.
The registration fee does not include the accommodation costs.
Early bird registration ends on 30th April, and regular registration starts on 1st May.
The following registration options are provided:
- Early bird registration: € 550,00
- Regular registration: € 600,00
Please note that the invoice will be issued with 'reverse charge' for institutions/companies based in EU that are registered in VIES (VAT Information Exchange System)
The payment can be carried out:
by bank transfer
Bank transfer details:
Beneficiary: XRtechnology srl
Bank name: Banca Sella
Reason: XR&AI Summer School 2023 + name of person attending the school
by credit card through the following PayPal link [CLICK HERE]
Please send copy of the payment receipt by email to email@example.com
To receive the invoice for the school registration fee, please please download here the application form and return it completed at firstname.lastname@example.org.
For further information, please contact chairs at email@example.com
Casa delle Tecnologie Emergenti
Via S. Rocco, 1
75100 Matera, Italy
From Bari Palese Airport
The nearest airport is Bari Palese (60 km), from where you can get to Matera in the following ways:
- by direct shuttle bus service served by Miccolis (see the “Direct Booking” section);
- by direct shuttle bus service (see the “Bus Tour of Aeroporti di Puglia” section);
- by direct shuttle bus service by Grassani E Garofalo (see the timetable);
- by train with the Nord Barese Railway Line (ferrovienordbarese.it), reaching the station of Bari Centrale. Leaving the station, you get Piazza Aldo Moro: at that point, turn left to reach FAL (Ferrovie Appulo Lucane), where several trains arrive at Matera Centrale Station in about 80 minutes;
- by private transfer booking by sending an e-mail to firstname.lastname@example.org.
From Brindisi airport
You can get to Matera from Brindisi airport (about 154 km) with Pugliairbus via Bari Palese airport (and vice versa).
Otherwise you can book a private transfer by writing to email@example.com
Casa per Ferie Sant’Anna
Via Lanera, 14
75100 Matera, Italy
+39 0835 33 34 62