Add to Calendar
2023-08-16 11:00:00
2023-08-16 12:00:00
America/New_York
Thesis Defense: Physics-Inspired Deep Learning for Inverse Problems in MRI
Abstract:We demonstrate the power of combining the forward image acquisition model with deep learning solutions for inverse problems in magnetic resonance imaging (MRI), from individual network layers to the network architecture design and inference procedure.First, we propose neural network layers that combine image space representations with representations in Fourier space, where MRI data is acquired. These layers can be used as drop-in replacements for standard image space convolutions in a variety of network architectures and yield higher quality reconstructions across a wide range of MR imaging tasks.Next, we demonstrate a deep learning framework for MRI motion correction, where the forward imaging model informs both the network architecture and the inference procedure. Our method incorporates potentially unknown motion parameters as inputs to the network and then optimizes them for each test example. The optimization is performed via an objective function that forces the reconstructed image and estimated motion parameters to be consistent with the acquired data. This approach reduces the joint image-motion parameter search used by most motion correction strategies to an inference-time search over motion parameters alone, greatly simplifying the complexity of the optimization problem to be solved for a novel image. Our hybrid method achieves the high reconstruction fidelity characteristic of deep learning solutions while retaining the benefits of explicit model-based optimization -- in particular, the ability to reject examples where the network produces poor reconstructions. Experiments demonstrate the advantages of this combined approach over purely learning or model-based reconstruction techniques.Committee: Polina Golland (MIT), Elfar Adalsteinsson (MIT), Bruce Rosen (HMS), Robert Frost (HMS)
Seminar Room D463 (Star)
August 16
May 11
Learning-based Medical Image Analysis
Mert Sabuncu
Cornell University
Add to Calendar
2023-05-11 13:00:00
2023-05-11 14:00:00
America/New_York
Learning-based Medical Image Analysis
Over the last 7+ years, deep learning has been transforming medical imaging, from enhancing acquisition to maximizing downstream utility of scans. Much of this progress relies on supervised learning approaches with “black box” models. In this talk, I will show two examples of recent work from my group where we move beyond this traditional paradigm to develop tools designed for the unique considerations of medical imaging. In the first part, I will focus on a classic problem: multi-modal image registration. I will describe a novel architecture we developed, coined KeyMorph, that affords the user robustness, control, and interpretability. In the second part, I will discuss the problem of localizing and quantifying change in longitudinal imaging studies, another classic setting in medical applications. Our learning-based approach exploits the arrow of time to characterize temporal change in serial images of a given individual, in an efficient, scalable, and easily implementable manner.
32-D451
April 25
Representational Models of Brain Connectivity and Behavior
Niharika DSouza
IBM Research
Add to Calendar
2023-04-25 13:00:00
2023-04-25 14:00:00
America/New_York
Representational Models of Brain Connectivity and Behavior
We live in a world which is naturally organized into intrinsic network structures. The study of networks is thus very relevant to modern day data-science, as we gain a lot of insight into otherwise mysterious phenomena. One such complex network is the human brain. There has been a lot of interest in understanding how regions in the brain communicate with each other and how these communication patterns influence our behavior and health. This sets us up for an important, yet really challenging question in healthcare: of how to represent these interactions and relate them to meaningful patient level characterization. My talk will briefly highlight three key projects from my PhD work which blend knowledge from the machine learning, statistics and graph signal processing worlds to analyse brain functional (rs-fMRI) and structural (DTI) connectivity data. The broad goal is to relate these representations to behavioral deficits in patient populations. First, I will present a joint network optimization framework to predict clinical severity from resting state fMRI data. This model couples two representational terms: a generative matrix factorization to capture the data geometry of fMRI correlation matrices and a discriminative regression framework. One key novelty of this algorithm lies in jointly optimizing the representation learning and prediction, which is key to the generalization onto unseen examples. Building off of this framework, I will then introduce an extension that incorporates multimodal information from Diffusion Tensor Imaging (DTI) and dynamic functional connectivity (rs-fMRI). At a high level, the generative matrix factorization now estimates a time-varying functional decomposition guided by anatomical connections in a graph regularized setting. We couple this representation with a deep network to predict multidimensional clinical characterizations. This deep network consists of an LSTM to model temporal-attention based dynamics of scan evolution and an ANN for prediction. Based on the principles underlying these geometry-informed models, we also develop a multimodal graph convolutional network (M-GCN) designed to exploit the native topology of functional and structural connectivity data in its formulation. This framework provides us with the flexibility to exploit the complementary nature of structure and function and faithfully map to behavioral outcomes even in the presence of limited training dataThese models help us construct a more holistic data-driven picture of brain connectivity and behavior. Overall, these frameworks make minimal assumptions and can potentially find a broad range of applications outside of the medical realm.
32-D451
April 10
NEW DATE/TIME/ROOM: Deep Generative Physical Models for MRI Reconstruction
Jon Tamir
University of Texas at Austin
Add to Calendar
2023-04-10 10:00:00
2023-04-10 11:00:00
America/New_York
NEW DATE/TIME/ROOM: Deep Generative Physical Models for MRI Reconstruction
Recently, deep learning techniques have been used as powerfuldata-driven reconstruction methods for inverse problems, and inparticular have led to reduced scan times in magnetic resonanceimaging (MRI). Typically, these methods are implemented usingend-to-end supervised learning based on idealized imaging conditions.While promising, reconstruction quality is known to degrade whenapplied to natural measurement and anatomy perturbations. In this talkwe present an alternative approach to deep learning reconstructionbased on distribution learning, in which we train a deep generativemodel to learn image priors without reference to the measurementprocess. We show that decoupling the measurement and statisticalmodels provides a powerful framework for MRI reconstruction. Weleverage recent advances in score-based generative modeling to learnthe prior distribution and we pose the image reconstruction task asposterior sampling. We show that this approach is competitive withend-to-end methods when applied to in-distribution data, and wedemonstrate theoretical and empirical robustness to variousout-of-distribution shifts. In cases where the distribution shift islarge, we empirically show a small amount of training data issufficient to recover the performance. Finally, we demonstrate theutility of our framework for image reconstruction in the presence ofsubject motion.
32-D451
January 23
Imaging early human brain development
Ali Gholipour
Boston Children's Hospital, Harvard Medical School
Add to Calendar
2023-01-23 13:00:00
2023-01-23 14:00:00
America/New_York
Imaging early human brain development
The human brain undergoes its most rapid formative growth during thefetal period, in which a sequence of amazingly programmed processeseventually forms the most complex living organ known. Until recently,our ability to study brain development in-utero was limited to crudelinear measurements of the brain anatomy on prenatal ultrasound orfetal MRI slices. With the advent of motion-corrected, robustsuper-resolution MRI reconstruction, the field progressed rapidly withnew tools and resources such as atlases that have enabled mapping andanalyzing the development of the brain microstructure and functionbefore birth. These technological advances in fetal imaging arecrucial to study mechanisms and patterns of normal and altered braindevelopment. In this talk, we will review the technical advances thathave made the foundation of next-generation in-vivo fetal neuroimagingtechniques. We will discuss motion-robust diffusion-weighted MRI thatoffers a unique ability to study the development of the fetal brainconnectome. In addition to slice-to-volume reconstruction, atlasconstruction, and their applications, we will discuss how deeplearning techniques have contributed to advancing various fetal MRItechnologies at the acquisition and the post-acquisition processingsteps.
32-D451