XR&AI Summer School 2024

Extended reality (XR) endeavors to replicate reality through virtual elements and environments. However, contemporary XR systems predominantly emphasize visual components, often sidelining other sensory dimensions such as sound, haptics, and olfaction. This lecture aims to rectify this imbalance by delving into non-visual XR systems, exploring multisensory experiences, and elucidating multimodal interaction. We will commence by examining non-visual XR systems and delve into the intricacies of multisensory experiences. Through an exploration of fundamental research on human sensory perception, we will uncover insights into the interplay between modalities and their implications for XR design. Moreover, we will expound upon techniques for optimizing XR interaction performance by leveraging multimodal interfaces.

Wolfgang Hürst is an Associate Professor in the Department of Information and Computing Sciences at Utrecht University, located in The Netherlands. His expertise lies in the domains of human-computer interaction, augmented and virtual reality, and multimedia systems, with a particular emphasis on gaming and media interaction. Through his research, he explores innovative concepts, delves into fundamental research inquiries, and applies his findings to develop novel approaches for human-machine interaction. His contributions have been disseminated through numerous articles in esteemed scientific publications, reflecting his dedication to advancing knowledge in his field. Dr. Hürst has also served as an Associate Editor for IEEE MultiMedia and the IEEE Transactions on Multimedia. Additionally, he is recognized as a co-founder of the IEEE International Conference on Artificial Intelligence & Extended and Virtual Reality (IEEE AIxVR).

Virtual design, online marketplaces, product lifecycle workflows, AR/VR, videogames, ..., all require lifelike digital representations of real-world materials (i.e., digital twins). Acquiring these digital copies is typically a cumbersome and slow process that requires expensive machines and several manual steps, creating roadblocks for scalability, repeatability, and consistency.
In this talk, I will discuss the challenges associated with creating 3D digital copies of real-world scenes and materials. Additionally, I will explore how advancements in Artificial Intelligence (AI) have played a crucial role in advancing the field of inverse scene decomposition and rendering.

Elena Garces is a Director and Senior Research Scientist at SEDDI, where she leads the AI & Optics team. She has more than ten years of experience in computer graphics and vision, mainly focused on 3D digitization. Her career has followed a dual path combining industry and academic research and leadership, taking diverse roles in entities like Adobe, Technicolor, or Universidad Rey Juan Carlos.

Tiny Machine Learning and Embedded engineers urge new tools to support them in being faster, more productive to unleash their creativity more than ever. Therefore, ST, devoted its best resources across product divisions and system research to create the Unified AI Core Technology. It acts as the enabling unifying AI technology to serve all heterogeneous products such as micro-controllers and sensors. Furthermore, this technology interfaces the most widely used Deep Learning representation standards such as Google Keras, QKeras and Tensorflow Lite and the Open Neural Network Exchange (ONNX). It outputs optimized C code across heterogeneous instruction sets with public APIs for STM32, STM32N6, Stellar MCUs and AI MEMs sensors. Demonstration of the tool in action will be provided to the audience.

Danilo Pau is Technical Director, IEEE AAIA & ST Fellow, APSIPA Life Member in STMicroelectronics. Danilo (h-index 27, i10-index 74) graduated at Politecnico di Milano. He worked on memory reduced HDMAC HW design, MPEG2 video memory reduction. on video coding, transcoding, embedded (Khronos) 2/3D graphics, and (ISO/IEC/MPEG CDVS and CDVA with Leonardo Chiariglione) computer vision. Currently, his work focuses on the ST Unified AI Core Technology. He supervised many students.

Deep Neural Networks (DNNs) have witnessed remarkable advancements across diverse domains, including XR, continually scaling in size and training on increasingly expansive datasets. This progression has bestowed upon them the remarkable ability to swiftly adapt to novel tasks with minimal training data. However, the journey of training models comprising tens to hundreds of billions of parameters on vast datasets is anything but straightforward. It demands a harmonious integration of artificial intelligence proficiency, high-performance computing (HPC) capabilities, and profound systems knowledge. This lecture is designed to offer comprehensive insights into the intricacies of training and deploying the most expansive neural networks. Delving into the complexities of this multifaceted process, the lecture will illuminate the key considerations, methodologies, and strategies required to successfully navigate the challenges posed by these cutting-edge neural networks. Whether you are a seasoned AI practitioner, an HPC enthusiast, or simply intrigued by the evolving landscape of deep learning, this lecture promises to equip you with the knowledge and tools necessary to harness the power of the largest neural networks in today's AI ecosystem.

Giuseppe Fiameni holds a PhD in Computer Engineering from the University of Modena and Reggio Emilia. He is currently a Data Scientist and AI consultant at NVIDIA where he also oversees the NVIDIA AI Technology Center for Italy, a collaboration among NVIDIA, CINI and CINECA to accelerate academic research in the field of Artificial Intelligence through collaboration projects. He has been working as HPC specialist at CINECA, the largest HPC facility in Italy, for more than 14 years providing support for large-scale data processing workloads.

XR technology allows the designer to manipulate their designs in real time, make adjustments, and immediately see how those changes would fit within the actual physical space. This level of interaction enables people to walk through and interact with designs before any physical construction begins, making it easier to identify potential issues and make changes early in the design process. In this course, we will spotlight the impact of XR in two distinct design scenarios: sketching for storytelling and custom-fit garment designing. We will discuss in deep the benefits XR offers for enhancing visualization in interactive animation and advancing smart manufacturing in fashion, respectively. Besides, the integration of Artificial Intelligence (AI) into the design process has opened up new avenues for creativity, efficiency, and innovation. We will discuss how the two design scenerios above can be benefited from AI techinques.

Anran Qi is a postdoctoral researcher at Inria Sophia-Antipolis in the GraphDeco research group. Her research focuses on garment design and human sketch understanding from graphics, machine learning, and human-computer interaction perspectives. She received a Ph.D fromthe Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey in 2021 and later spent two years as a postdoctoral researcher at the Igarashi Lab, the University of Tokyo.

Future: XR and AI are Converging Technologies - Ready for a Revolution?" embarks on a reinventing exploration of the imminent revolution at the intersection of XR and AI. This talk unveils the transformative potential of these converging technologies, showcasing the emergence of entirely new creative languages in entertainment and media. From conceptualization to production, witness the seamless blend of creativity and technological prowess. The lecture propels us into a pioneering frontier, emphasizing the inevitability of a technological shift. Join us in unraveling how XR and AI forge a revolutionary path, creating a future where innovation knows no bounds.

Elisabetta Rotolo is a multifaceted business strategist and brand management executive, renowned for her expertise in XR, spatial computing, and the Metaverse. She's the CEO of MIAT Multiverse Institute For Arts and Technology, pioneering immersive education and production. With over 20 years of experience, she's held prominent roles at organizations like the New York State, Omnicom Group, Walt Disney, Mattel, Heinz, Plasmon, the Italian Railways, the Olympic Games. A prolific speaker, she's shared insights at prestigious events globally, including VR/AR Global Summit and La Biennale Venezia film Festival. Elisabetta is also a contributor to Artribune and serves on advisory boards for cultural organizations. She holds degrees in Interior Architecture and an MBA, recognized as "Student of the Year" in 2016-2017.

The goal of this speech is to provide an introduction on Amazon Mechatronics and Sustainable Packaging (part of Amazon Global Robotics) organizational structure and scope, and an overview of some main AI/ML potential applications to Robotics Product Development process in industrial logistics. Main focus will be on AI techniques and tools as robotics product development enablers (e.g., process simulation and emulation) and/or product features (e.g., advanced collision detection), as well as on standardization and regulation aspects and their impacts on real world applications.

Gianpiero Negri received the MSc degree summa cum laude in Computer Engineering from Salerno University, a post MSc degree in Industrial Research, Mechatronics and Automation from Italian National Research Council, and a Ph.D. in Applied Mathematics, with main focus on Robotics AI Safety. He’s currently the Global Head of Central Quality Assurance and Field compliance at Amazon Global Robotics and he’s also a member of international AI and robotics standardization technical committees. He is author of several papers in the field of robotics and AI, and he has been invited speaker and panelist for some of the major robotics and AI global conferences (NVIDIA GTC, IRSC, ICRA, ECML PKDD etc.)

Embark on a journey through Spatial Computing at AWS to master the handling of intricate 3D data environments. We'll unveil AWS's robust infrastructure and the practicalities of developing for immersive devices, with a keen eye on industry-specific AR/VR use cases. The talk surveys the evolving hardware terrain and, with hands-on demos, steers participants towards starting their Spatial Computing projects, all while encapsulating AWS's strategic foresight in sculpting the future of immersive tech.

Peter Daukintis is Principal Prototyper working at AWS (Amazon Web Services) helping customers to build for Spatial Computing. Previous software experience in computer graphics industries; including games development, post-production, audio programming and computer animation.

While virtual environments have long focused on visual and auditory experiences, the integration of olfactory stimuli opens new frontiers for sensory immersion. In this lesson, we will explore the basics behind olfaction, the challenges and opportunities incorporating smell into XR environments and multisensory interaction, and the potential applications across various sectors, including industry, entertainment, healthcare, education, and gaming. We will discuss how scents can enhance the realism, emotional engagement, and overall user experience in eXtended Reality, paving the way for truly multisensory virtual worlds.

Monica Bordegoni is full professor in the Department of Mechanical Engineering, School of Design of the Politecnico di Milano. Her research interests are in the areas of product design methods, virtual and physical prototyping, eXtended Reality technologies, human-computer interaction, user experience design, multisensory interaction and olfaction and olfactory displays. She has published around 300 peer-reviewed scientific papers in international journals and conference proceedings and has served on about fifty conference and workshop program committees. She is member of the Advisory Board of the Design Society. She has been the principal investigator of several funded research projects.

Every day we are witnessing a revolution in artificial intelligence, where brute force applied to a set of black boxes overcomes old-school understanding and insight. Huge transformers assault the safe zone of human superiority with a breathtaking mix of joy and dismay. Are these feelings real, or are they just part of the hallucinations that current AI tools can induce? The devil is in the details. If we look more closely, get more precise control, or search for uniqueness, we encounter a firm barrier that is surprisingly difficult to overcome. The metaphorical moment when Deckard falls in love with Rachel seems a long way off. In this talk I will attempt to explore the obstacles and discuss the possibilities of whether this epic moment will ever happen.

Daniel Sýkora is a Professor of Computer Graphics and Interaction at CTU in Prague. He leads a research group focused on developing tools that help artists and animators in their work to eliminate repetitive, time-consuming tasks while preserving the uniqueness of the hand-created work, thus providing complete creative freedom. The team works with renowned industry partners like Google, Adobe, Disney, and Snap to integrate the developed algorithms into professional tools. Daniel has won numerous awards for his work, including the prestigious Günter Enderle Best Paper Award and the Neuron Prize for promising young scientists.

Casa delle Tecnologie Emergenti

Casa per Ferie Sant’Anna

La casa del nonno