Illinois Extended Reality testbed or ILLIXR (pronounced like elixir) is a fully open-source Extended Reality (XR) system and testbed. The modular, extensible, and OpenXR-compatible ILLIXR runtime integrates state-of-the-art XR components into a complete XR system.
The current ILLIXR release is v4.0. Source code is available from our GitHub repository, which also contains instructions for building fully contained Docker images. Docker images can also be downloaded from a hub (see here for instructions.)
ILLIXR provides its components in standalone configurations to enable architects and system designers to research each component in isolation. The standalone components are packaged together as of the v3.1.0 release of ILLIXR.
ILLIXR's modular and extensible runtime allows adding new components and swapping different implementations of a given component. ILLIXR currently contains the following plugins and services:
- Filter-based visual-inertial estimator
- Tracking
- Head tracking: ORB_SLAM31 and lighthouse
- Hand tracking1
- openni
- depthai
- IMU integrators
- Pose Related Service
- Cameras
- zed supporting ZED Mini
- realsense supporting Intel RealSense
- webcam
- Asynchronous reprojection:
- timewarp_gl, OpenGL based
- timewarp_vk, Vulkan based
- Asynchronous 6-degree reprojection openwarp
- vkdemo - toy application, with native ILLIXR rendering interface gldemo
- native_renderer - render management
- gldemo - stand-in application when ILLIXR is run as a standalone application without an actual OpenXR application
- offload_data
- record_imu_cam
- record_rgb_depth
- zed_capture (standalone executable)
- offload_vio
- offload_rendering_client
- offload_rendering_client_jetson
- offload_rendering_server
- tcp_network_backend
Some components, such as eye tracking and reconstruction are available as standalone components and are in the process of being integrated.
We continue to refine and add new components and implementations. Many of the current components of ILLIXR were developed by domain experts and obtained from publicly available repositories. They were modified for one or more of the following reasons: fixing compilation, adding features, or removing extraneous code or dependencies. Each component not developed by us is available as a forked GitHub repository for proper attribution to its authors.
A paper with details on ILLIXR, including its components, runtime, telemetry support, and a comprehensive analysis of performance, power, and quality on desktop and embedded systems.
A talk presented at NVIDIA GTC'21 describing ILLIXR and announcing the ILLIXR consortium: Video. Slides.
A demo of an OpenXR application running with ILLIXR.
The ILLIXR consortium is an industry-supported community effort to democratize XR systems research, development, and benchmarking. Visit our website for more information. For news and papers go to illixr.org, for talks see our YouTube channel, and join our Discord for announcements.
All of our demo videos can be seen on our YouTube channel.
We request that you cite our following paper when you use ILLIXR for a publication. We would also appreciate it if you send us a citation once your work has been published.
@inproceedings{huzaifa2021illixr,
title={ILLIXR: Enabling end-to-end extended reality research},
author={Huzaifa, Muhammad and Desai, Rishi and Grayson, Samuel and Jiang, Xutao and Jing, Ying and Lee, Jae and Lu, Fang and Pang, Yihan and Ravichandran, Joseph and Sinclair, Finn and Tian, Boyuan and Yuan, Hengzhi and Zhang, Jeffrey and Adve, Sarita V.},
booktitle={2021 IEEE International Symposium on Workload Characterization (IISWC)},
pages={24--38},
year={2021},
organization={IEEE}
}
Once your work is published, please send the citation to us at illixr@cs.illinois.edu.
We welcome anyone to contribute to ILLIXR. If you wish to contribute, please see out contribution guide. Full documentation, including API specifications, can be found at our Getting Started page.
The ILLIXR project started in Sarita Adve’s research group, co-led by PhD candidate Muhammad Huzaifa, at the University of Illinois at Urbana-Champaign. Other major contributors include Rishi Desai, Douglas Friedel, Steven Gao, Samuel Grayson, Qinjun Jiang, Xutao Jiang, Ying Jing, Jae Lee, Jeffrey Liu, Fang Lu, Yihan Pang, Joseph Ravichandran, Giordano Salvador, Rahul Singh, Finn Sinclair, Boyuan Tian, Henghzhi Yuan, and Jeffrey Zhang.
ILLIXR came together after many consultations with researchers and practitioners in many domains: audio, graphics, optics, robotics, signal processing, and extended reality systems. We are deeply grateful for all of these discussions and specifically to the following: Wei Cu, Aleksandra Faust, Liang Gao, Matt Horsnell, Amit Jindal, Steve LaValle, Steve Lovegrove, Andrew Maimone, Vegard Øye, Martin Persson, Archontis Politis, Eric Shaffer, Paris Smaragdis, Sachin Talathi, and Chris Widdowson.
Our OpenXR implementation is derived from Monado. We are particularly thankful to Jakob Bornecrantz and Ryan Pavlik.
The development of ILLIXR was supported by the Applications Driving Architectures (ADA) Research Center (a JUMP Center co-sponsored by SRC and DARPA), the Center for Future Architectures Research (C-FAR, a STARnet research center), a Semiconductor Research Corporation program sponsored by MARCO and DARPA, National Science Foundation grants 2120464 and 2217144 and by a Google Faculty Research Award. The development of ILLIXR was also aided by generous hardware and software donations from ARM and NVIDIA. Facebook Reality Labs provided the OpenEDS Semantic Segmentation Dataset.
Wesley Darvin came up with the name for ILLIXR.
ILLIXR is available as open-source software under the permissive University of Illinois/NCSA Open Source License. As mentioned above, ILLIXR largely consists of components developed by domain experts and modified for the purposes of inclusion in ILLIXR. However, ILLIXR does contain software developed solely by us. The NCSA license is limited to only this software. The external libraries and software included in ILLIXR each have their own licenses and must be used according to those licenses:
Any LGPL or GPL licensed code are contained in optional components. ILLIXR's extensibility allows the source to be configured and compiled using only permissively licensed software, if desired. See out Getting Started page for instructions.
Whether you are a computer architect, a compiler writer, a systems person, work on XR related algorithms or applications, or just anyone interested in XR research, development, or products, we would love to hear from you and hope you will contribute! You can join our Discord, mailing list, email us, or just send a pull request!
Footnotes
-
Source is hosted in an external repository under the ILLIXR project. ↩ ↩2 ↩3 ↩4
-
Current as of March 5, 2025. ↩
-
ILLIXR uses a customized version of FFmpeg, compiled with x264 and x265 encoding support, thus it is licensed under GPL v3.0 ↩
-
This software is covered by multiple open source licenses, see the link for details. ↩