Cognitive AI Community of Research

We are developing a general framework that enforces privacy transparently enabling different kinds of machine learning to be developed that are automatically privacy preserving.

Integrating multi-modal clinical data and using recurrent and convolution neural networks to predict when patients will need important interventions.

We aim to learn language by distant supervision through captioned videos, similarly to how children learn language through interacting with the world around.

This project aims to build models grounded in perception that tackle classical planning problems in AI in a new realm.

In this project, we aim to build a vision and language system which learns and understands the world like a child.

The START Natural Language System understands and generates language and answers questions that are posed to it in natural language.

When discussing racial disparities in medical treatments, critics often cite social factors as confounders which explain away any differences. Comparing the health of whites to that of non-whites we do see that environmental and social factors conspire to yield higher rates of disease and shorter life spans in non-white populations. But does that really show that medical treatment itself is free from bias? We examine end-of-life care in the ICU, stratified by ethnicity, and controlled for acuity using severity assessment scores. Our analysis agrees with previous studies that nonwhites tend to receive more aggressive (high-risk, high reward) treatments, such as mechanical ventilation than non-whites, despite receiving comparable-or-moderately-less noninvasive treatments. Going further, we show that using treatment patterns and clinical notes, we are able to infer a patient's race. Finally, we show evidence suggesting nonwhite have a much greater distrust of the medical community among than whites do. We find that race, even in the great equalizer of end-of-life care, does continue to influence the treatments administered to a patient.

Using medical knowledge to improve learned representations of patient health trajectories.

Clinical concept extraction (CCE) of named entities - such as problems, tests, and treatments - aids in forming an understanding of notes and provides a foundation for many downstream clinical decision-making tasks. Historically, this task has been posed as a standard named entity recognition (NER) sequence tagging problem, and solved with feature-based methods using hand-engineered domain knowledge. Recent advances, however, have demonstrated the efficacy of LSTM-based models for NER tasks, including CCE. This work presents CliNER 2.0, a simple-to-install, open-source tool for extracting concepts from clinical text. CliNER 2.0 uses a word- and character- level LSTM model, and achieves state-of-the-art performance. For ease of use, the tool also includes pre-trained models available for public use.

A "precision medicine" approach for finding relevant cancer treatments in clinical literature and eligible trials. For a given patient with associated demographics (age, gender) and disease (cancer type, genetic variants), we query a database of all pubmed articles and clinicaltrials.gov trials using NLP techniques to find the most useful and relevant treatments for the patient. Our ensemble-based system performed very well in the TREC 2016 Precision Medicine challenge.

Clinical notes often describe the most important aspects of a patient's physiology and are therefore critical to medical research. However, these notes are typically inaccessible to researchers without prior removal of sensitive protected health information (PHI), a natural language processing (NLP) task referred to as de-identification. In order to build tools that perform deid, one typically needs the very same data that is private, thus creating a chicken-and-the-egg problem. In this work, we generate "fake" clinical notes where the deidentified information is replaced with real-seeming values (e.g. "Tim Lywood" instead of "George Beveridge") that still respect reasonable distributional semantics. We evaluate models trained on this synthetic data and show that they perform just as well as models trained on the sensitive PHI-bearing notes.

Using adversarial signals and a cycle-consistency based regularization, we can supplement paired regression tasks with unpaired data to improve regression performance.

Adding domain knowledge to word embeddings.

We've developed an object-based neural network architecture for learning predictive models of intuitive physics that extrapolates to variable object count and variable scene configurations with only spatially and temporally local computation.

This project focuses on holistic decision making for network optimization.

We develop a computational model that explains how people make causal judgments in physical scenes by mentally simulating counterfactual outcomes and comparing those to what actually happened.

Our goal is to create an online risk-aware planner for vehicle maneuvers that can make driving safer and less stressful through a “parallel” autonomous system that assists the driver by watching for risky situations, and by helping the driver take proactive, compensating actions before they become crises.

We aim to understand 3D object structure from a single image. We propose an end-to-end framework which sequentially estimates 2D keypoint heatmaps and 3D object structure, by training it on both real 2D-annotated images and synthetic 3D data and by integrating a 3D-to-2D projection layer.

Solid aims to radically change the way Web applications work today, resulting in true data ownership as well as improved privacy.

We use deep reinforcement learning and self-play to train human-level agents for a multiplayer fighting game.

We study the problem of 3D object generation. We propose a novel framework, 3D Generative Adversarial Network (3D-GAN), leveraging recent advances in volumetric convolutional networks and generative adversarial nets.

Our goal is to develop algorithms for allowing robots to adapt intelligently to humans when they are working together as a team.

This project focuses on smart routing in networks to guarantee on time delivery of messages.

Developing techniques to allow self-driving cars and other AI-driven systems to explain behaviors and failures.

Using AI methods, we are developing an attack tree generator that automatically enumerates cyberattack vectors for industrial control systems in critical infrastructure (electric grids, water networks and transportation systems). The generator can quickly assess cyber risk for a system at scale.

IoT devices primarily use free embedded Linux which has many security flaws. We are conducting penetration tests on IoT and developing a secure version of embedded Linux.

Transitioning machine learning models across electronic health record (EHR) versions can be improved by mapping different EHR encodings to a common vocabulary.