Our research group works with holographic visualization technology and develops visualisation platforms for clinicians, primarily for use in neurosurgical planning. Augmented reality (AR) and holographic display technology are  new innovative ways to improve medical data visualization for surgical procedures. Transforming traditional streams of 2D neuroimaging data into true 3D visualizations may facilitate the understanding of the complex spatial relationship between brain structures for clinicians during training and i relation to surgical planning.

Clinical use cases

Neurosurgery has a small margin of error. Planning and executing invasive procedures rely on surgeons having access to detailed information about the patient’s specific anatomy, functional regions, neuronal pathways, ventricles, blood vessels, diseased tissue, and their spatial interrelationships (Figure A). During brain tumor surgery, for example, physicians are challenged by cancerous tissues that often displace structures from their expected locations. Preoperative imaging (MRI/CT/PET) therefore plays a vital role in guiding surgeons and advances in neuroimaging bring increasingly complex data modalities for surgeons to identify critical brain structures. However, there has been little innovation in how these data neuroimaging streams are visualized in clinical settings - typically via simplified presentations on 2D screens.

Our research group works to develop a modular platform for visualizing individual patient brain imaging data in an interactive, multi-dimensional, holographic space (Figure B). Our aim is to build a platform that can be used for supporting neurosurgical planning, reviewing cases within a surgical team, or even during surgery for re-examining surgical planning details.

Holographic visualization technology

Recent technological developments have introduced both wearable head-mounted AR systems (notably Hololens and Magic Leap) and stand-alone stereoscopic 3D monitors (e.g., Looking Glass). AR headsets allow us to project and integrate interactive virtual elements anywhere into the surrounding physical space. Multiple individuals wearing these headsets can interact within the same AR environment, with fully transparent displays allowing users to maintain eye contact and engage with their physical surroundings. These elements render the system valuable for collaborative reviews and discussions, which are necessary in surgical planning procedures, as well as in teaching and training contexts.

Our aim is to change the way in which surgeons interact with neuroimaging data (Figure C), to develop new avenues in neurosurgical training, and to offer tools to improve the way neurosurgery is planned and executed in the future.

• Petersen MV, Mlakar J, Haber SN, Parent M, Smith Y, Strick PL, Griswold MA, McIntyre CC, Holographic Reconstruction of Axonal Pathways in the Human Brain, Neuron, Volume 104, Issue 6, 2019, Pages 1056-1064.e3, ISSN 0896-6273, https://doi.org/10.1016/j.neuron.2019.09.030.
• Husch A, Petersen MV, Gemmar P, Goncalves J, Sunde N, Hertel F, Post-operative deep brain stimulation assessment: Automatic data integration and report generation, Brain Stimulation, Volume 11, Issue 4, 2018, Pages 863-866, ISSN 1935-861X, https://doi.org/10.1016/j.brs.2018.01.031.
• Petersen MV, Husch A, Parsons CE, Lund TE, Sunde N, Østergaard K (2018), Using automated electrode localization to guide stimulation management in DBS. Ann Clin Transl Neurol, 5: 888-894. doi:10.1002/acn3.589

Postdoc Mikkel V. Petersen, e-mail: mikkel.petersen@cfin.au.dk