Publications

2025

1. FORLA: Federated Object-Centric Representation Learning with Slot Attention

   Guiqiu Liao, Matjaz Jogan, Eric Eaton, and 1 more author

   Neural Information Processing Systems (NeurIPS), 2025

   Abstract Bib PDF Code  

   Learning efficient visual representations across heterogeneous unlabeled datasets remains a central challenge in federated learning. Effective federated representations require features that are jointly informative across clients while disentangling client-specific factors without supervision. We thus introduce FORLA, a novel framework for federated object-centric representation learning and feature adaptation using unsupervised slot attention. At the core of our method is a shared feature adapter, trained collaboratively across clients to adapt features from foundation models, and a shared slot attention module that learns to reconstruct the adapted features. To optimize this adapter, we design a two-branch student–teacher architecture. In each client, a student decoder learns to reconstruct full features from foundation models, while a teacher decoder reconstructs their adapted, low-dimensional counterpart. The shared slot attention module bridges cross-domain learning by aligning object-level representations across clients. Experiments in multiple real-world datasets show that our framework not only outperforms centralized baselines on object discovery but also learns a compact, universal representation that generalizes well across domains. This work highlights federated slot attention as an effective tool for scalable, unsupervised visual representation learning from cross-domain data with distributed concepts. Our code, data, and pretrained models are available at: https://github.com/PCASOlab/FORLA.

   @article{Liao2025FORLA,
     author = {Liao, Guiqiu and Jogan, Matjaz and Eaton, Eric and Hashimoto, Daniel A.},
     title = {FORLA: Federated Object-Centric Representation Learning with Slot Attention},
     journal = {Neural Information Processing Systems (NeurIPS)},
     year = {2025},
   }

2. Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces

   Eric Eaton, Marcel Hussing, Michael Kearns, and 3 more authors

   In International Conference on Machine Learning (ICML), 2025

   Abstract Bib PDF

   In traditional reinforcement learning (RL), the learner aims to solve a single objective optimization problem: find the policy that maximizes expected reward. However, in many real-world settings, it is important to optimize over multiple objectives simultaneously. For example, when we are interested in fairness, states might have feature annotations corresponding to multiple (intersecting) demographic groups to whom reward accrues, and our goal might be to maximize the reward of the group receiving the minimal reward. In this work, we consider a multi-objective optimization problem in which each objective is defined by a state-based reweighting of a single scalar reward function. This generalizes the problem of maximizing the reward of the minimum reward group. We provide oracle-efficient algorithms to solve these multi-objective RL problems even when the number of objectives is very large — for tabular MDPs, as well as for large MDPs when the group functions have additional structure. The contribution of this paper is that we are able to solve this class of multi-objective RL problems with a possibly exponentially large class of constraints over intersecting groups in both tabular and large state space MDPs in an oracle-efficient manner. Finally, we experimentally validate our theoretical results and demonstrate applications on a preferential attachment graph MDP.

   @inproceedings{Eaton2025IntersectionalFairness,
     author = {Eaton, Eric and Hussing, Marcel and Kearns, Michael and Roth, Aaron and Sengupta, Sikata Bela and Sorrell, Jessica},
     year = {2025},
     title = {Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces},
     booktitle = {International Conference on Machine Learning (ICML)},
   }

3. MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL [Spotlight]

   Claas A Voelcker, Marcel Hussing, Eric Eaton, and 2 more authors

   In International Conference on Learning Representations (ICLR), 2025

   Spotlight   Abstract Bib PDF

   Spotlight paper; spotlight acceptance rate: 3.26%

   Building deep reinforcement learning (RL) agents that find a good policy with few samples has proven notoriously challenging. To achieve sample efficiency, recent work has explored updating neural networks with large numbers of gradient steps for every new sample. While such high update-to-data (UTD) ratios have shown strong empirical performance, they also introduce instability to the training process. Previous approaches need to rely on periodic neural network parameter resets to address this instability, but restarting the training process is infeasible in many real-world applications and requires tuning the resetting interval. In this paper, we focus on one of the core difficulties of stable training with limited samples: the inability of learned value functions to generalize to unobserved on-policy actions. We mitigate this issue directly by augmenting the off-policy RL training process with a small amount of data generated from a learned world model. Our method, Model-Augmented Data for Temporal Difference learning (MAD-TD) uses small amounts of generated data to stabilize high UTD training and achieve competitive performance on the most challenging tasks in the DeepMind control suite. Our experiments further highlight the importance of employing a good model to generate data, MAD-TD’s ability to combat value overestimation, and its practical stability gains for continued learning.

   @inproceedings{Voelcker2025MADTD,
     author = {Voelcker, Claas A and Hussing, Marcel and Eaton, Eric and Farahmand, Amir-massoud and Gilitschenski, Igor},
     year = {2025},
     title = {MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL [Spotlight]},
     booktitle = {International Conference on Learning Representations (ICLR)},
   }

4. Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Model

   Long Le, Jason Xie, William Liang, and 7 more authors

   In International Conference on Learning Representations (ICLR), 2025

   Abstract Bib PDF

   Interactive 3D simulated objects are crucial in AR/VR, animations, and robotics, driving immersive experiences and advanced automation. However, creating these articulated objects requires extensive human effort and expertise, limiting their broader applications. To overcome this challenge, we present ARTICULATE-ANYTHING, a system that automates the articulation of diverse, complex objects from many input modalities, including text, images, and videos. ARTICULATE-ANYTHING leverages vision-language models (VLMs) to generate code that can be compiled into an interactable digital twin for use in standard 3D simulators. Our system exploits existing 3D asset datasets via a mesh retrieval mechanism, along with an actor-critic system that iteratively proposes, evaluates, and refines solutions for articulating the objects, self-correcting errors to achieve a robust outcome. Qualitative evaluations demonstrate ARTICULATE-ANYTHING’s capability to articulate complex and even ambiguous object affordances by leveraging rich grounded inputs. In extensive quantitative experiments on the standard PartNetMobility dataset, ARTICULATE-ANYTHING substantially outperforms prior work, increasing the success rate from 8.7–12.2% to 75% and setting a new bar for state-of-the-art performance. We further showcase the utility of our system by generating 3D assets from in-the-wild video inputs, which are then used to train robotic policies for fine-grained manipulation tasks in simulation that go beyond basic pick and place. These policies are then transferred to a real robotic system.

   @inproceedings{Le2025ArticulateAnything,
     author = {Le, Long and Xie, Jason and Liang, William and Wang, Hung-Ju and Yang, Yue and Ma, Yecheng Jason and Vedder, Kyle and Krishna, Arjun and Jayaraman, Dinesh and Eaton, Eric},
     year = {2025},
     title = {Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Model},
     booktitle = {International Conference on Learning Representations (ICLR)},
   }

5. Neural Eulerian Scene Flow Fields

   Kyle Vedder, Neehar Peri, Ishan Khatri, and 7 more authors

   In International Conference on Learning Representations (ICLR), 2025

   Abstract Bib PDF Website  

   We reframe scene flow as the task of estimating a continuous space-time ordinary differential equation (ODE) that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5×, and even exceeding the next-best supervised method by over 10%. See vedder.io/eulerflow for interactive visuals.

   @inproceedings{Vedder2025NeuralEulerianSceneFlow,
     author = {Vedder, Kyle and Peri, Neehar and Khatri, Ishan and Li, Siyi and Eaton, Eric and Kocamaz, Mehmet Kemal and Wang, Yue and ZhidingYu and Ramanan, Deva and Pehserl, Joachim},
     year = {2025},
     title = {Neural Eulerian Scene Flow Fields},
     booktitle = {International Conference on Learning Representations (ICLR)},
   }

6. Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Models

   Jean Park, Kuk Jin Jang, Basam Alasaly, and 5 more authors

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2025

   Abstract DOI Bib PDF

   Multimodal large language models (MLLMs) can simultaneously process visual, textual, and auditory data, capturing insights that complement human analysis. However, existing video question-answering (VidQA) benchmarks and datasets often exhibit a bias toward a single modality, despite the goal of requiring advanced reasoning skills that integrate diverse modalities to answer the queries. In this work, we introduce the modality importance score (MIS) to identify such bias. It is designed to assess which modality embeds the necessary information to answer the question. Additionally, we propose an innovative method using state-of-the-art MLLMs to estimate the modality importance, which can serve as a proxy for human judgments of modality perception.With this MIS, we demonstrate the presence of unimodal bias and the scarcity of genuinely multimodal questions in existing datasets. We further validate the modality importance score with multiple ablation studies to evaluate the performance of MLLMs on permuted feature sets. Our results indicate that current models do not effectively integrate information due to modality imbalance in existing datasets. Our proposed MLLM-derived MIS can guide the curation of modality-balanced datasets that advance multimodal learning and enhance MLLMs’ capabilities to understand and utilize synergistic relations across modalities.

   @inproceedings{Park2025AssessingModalityBias,
     author = {Park, Jean and Jang, Kuk Jin and Alasaly, Basam and Mopidevi, Sriharsha and Zolensky, Andrew and Eaton, Eric and Lee, Insup and Johnson, Kevin},
     year = {2025},
     title = {Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Models},
     booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
     volume = {39},
     number = {19},
     pages = {19821--19829},
     doi = {10.1609/aaai.v39i19.34183},
   }

7. Disentangling spatio-temporal knowledge for weakly supervised object detection and segmentation in surgical video

   Guiqiu Liao, Matjaz Jogan, Sai Koushik Samudrala Sambasastry, and 2 more authors

   In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025

   Abstract Bib PDF Code  

   Weakly supervised video object segmentation (WSVOS) enables the identification of segmentation maps without requiring extensive annotations of object masks, relying instead on coarse video labels indicating object presence. WSVOS in surgical videos is, however, more challenging due to the complex interaction of multiple transient objects, such as surgical tools moving in and out of the surgical field. In this scenario, state-of-the-art WSVOS methods struggle to learn accurate segmentation maps. We address this problem by introducing ViDeo Spatio-Temporal disentanglement Networks (VDST-Net), a framework to disentangle complex spatio-temporal object interactions using semi-decoupled knowledge distillation to predict high-quality class activation maps (CAMs). A teacher network is designed to help a temporal-reasoning student network resolve activation conflicts, as the student leverages temporal dependencies when specifics about object location and timing in the video are not provided. We demonstrate the efficacy of our framework on a challenging surgical video dataset where objects are, on average, present in less than 60% of annotated frames, and compare our method to state-of-the-art methods on surgical data and on a public dataset commonly used to benchmark WSVOS. Our method outperforms state-of-theart techniques and generates accurate segmentation masks under video-level weak supervision. Our code is available at: https://github.com/PCASOlab/VDST-net.

   @inproceedings{Liao2025DisentanglingSpatioTemporal,
     author = {Liao, Guiqiu and Jogan, Matjaz and Sambasastry, Sai Koushik Samudrala and Eaton, Eric and Hashimoto, Daniel},
     year = {2025},
     title = {Disentangling spatio-temporal knowledge for weakly supervised object detection and segmentation in surgical video},
     booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
   }

8. Towards a Real-time Clinical Agenda Setting System for Enhancing Clinical Interactions in Primary Care Visits

   Kuk Jin Jang, Sameer Bhatti, Sydney Pugh, and 5 more authors

   In Workshop on Large Language Models and Generative AI for Health at AAAI 2025, 2025

   Abstract Bib PDF

   Technology has increasingly hindered meaningful engagement between patient and providers during primary care visits, often detracting from effective communication. However, artificial intelligence (AI) advancements present new opportunities to enhance and improve patient-provider communication. A promising application is the use of AI to identify and highlight agenda items for discussion during visits and to summarize relevant clinical details in real-time. This study explores the feasibility, potential, and challenges of developing a real-time automated agenda-setting system leveraging generative AI, specifically large language models (LLMs). From a dataset of recorded and annotated simulation visits, we evaluate the performance of LLMs in identifying agenda items and capturing associated clinical details within the conversation flow. In particular, we focus on the impact of realtime constraints and contextual factors on the ability to detect and summarize relevant items. Our findings suggest that optimizing performance requires a balance between providing contextual information through both summaries and the actual conversation. Based on these results, we discuss the challenges involved in developing a real-time agenda-setting system and offer recommendations for future advancements.

   @inproceedings{Jang2025Towards,
     author = {Jang, Kuk Jin and Bhatti, Sameer and Pugh, Sydney and Maduno, Chimezie and Sridhar, Sarang and Mopidevi, Sriharsha and Eaton, Eric and Johnson, Kevin},
     year = {2025},
     title = {Towards a Real-time Clinical Agenda Setting System for Enhancing Clinical Interactions in Primary Care Visits},
     booktitle = {Workshop on Large Language Models and Generative AI for Health at AAAI 2025},
   }

9. System and Methods for Generating Improved Decision Trees

   Gilmer Valdes, Timothy D. Solberg, Charles B. Simone II, and 3 more authors

   2025

   US Patent US-12229226-B2

   Bib

   @patent{Valdes2025SystemAndMethods,
     author = {Valdes, Gilmer and Solberg, Timothy D. and {Simone II}, Charles B. and Ungar, Lyle H. and Eaton, Eric and Luna, Jose Marcio},
     year = {2025},
     title = {System and Methods for Generating Improved Decision Trees},
     number = {US-12229226-B2},
   }

10. A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models

    Jason Hughes, Marcel Hussing, Edward Zhang, and 24 more authors

    2025

    Abstract Bib Website  

    This report presents a heterogeneous robotic system designed for remote primary triage in mass-casualty incidents (MCIs). The system employs a coordinated air-ground team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) to locate victims, assess their injuries, and prioritize medical assistance without risking the lives of first responders. The UAV identify and provide overhead views of casualties, while UGVs equipped with specialized sensors measure vital signs and detect and localize physical injuries. Unlike previous work that focused on exploration or limited medical evaluation, this system addresses the complete triage process: victim localization, vital sign measurement, injury severity classification, mental status assessment, and data consolidation for first responders. Developed as part of the DARPA Triage Challenge, this approach demonstrates how multi-robot systems can augment human capabilities in disaster response scenarios to maximize lives saved.

    @misc{Hughes2025MultiRobotTriage,
      title = {A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models},
      author = {Hughes, Jason and Hussing, Marcel and Zhang, Edward and Kannapiran, Shenbagaraj and Caswell, Joshua and Chaney, Kenneth and Deng, Ruichen and Feehery, Michaela and Kratimenos, Agelos and Li, Yi Fan and Major, Britny and Sanchez, Ethan and Shrote, Sumukh and Wang, Youkang and Wang, Jeremy and Zein, Daudi and Zhang, Luying and Zhang, Ruijun and Zhou, Alex and Zhouga, Tenzi and Cannon, Jeremy and Qasim, Zaffir and Yelon, Jay and Cladera, Fernando and Daniilidis, Kostas and Taylor, Camillo J. and Eaton, Eric},
      year = {2025},
      eprint = {2512.08754},
      archiveprefix = {arXiv},
      primaryclass = {cs.RO},
      url = {https://arxiv.org/abs/2512.08754},
    }

2024

1. Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence

   Marcel Hussing, Claas A Voelcker, Igor Gilitschenski, and 2 more authors

   In 1st Reinforcement Learning Conference (RLC), 2024

   Abstract Bib PDF

   We show that deep reinforcement learning algorithms can retain their ability to learn without resetting network parameters in settings where the number of gradient updates greatly exceeds the number of environment samples by combatting value function divergence. Under large update-to-data ratios, a recent study by Nikishin et al. (2022) suggested the emergence of a primacy bias, in which agents overfit early interactions and downplay later experience, impairing their ability to learn. In this work, we investigate the phenomena leading to the primacy bias. We inspect the early stages of training that were conjectured to cause the failure to learn and find that one fundamental challenge is a long-standing acquaintance: value function divergence. Overinflated Q-values are found not only on out-of-distribution but also in-distribution data and can be linked to overestimation on unseen action prediction propelled by optimizer momentum. We employ a simple unit-ball normalization that enables learning under large update ratios, show its efficacy on the widely used dm_control suite, and obtain strong performance on the challenging dog tasks, competitive with model-based approaches. Our results question, in parts, the prior explanation for sub-optimal learning due to overfitting early data.

   @inproceedings{Hussing2024DissectingDeepRL,
     author = {Hussing, Marcel and Voelcker, Claas A and Gilitschenski, Igor and Farahmand, Amir-massoud and Eaton, Eric},
     year = {2024},
     title = {Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence},
     booktitle = {1st Reinforcement Learning Conference (RLC)},
   }

2. Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning

   Marcel Hussing, Jorge Mendez-Mendez, Anisha Singrodia, and 2 more authors

   In 1st Reinforcement Learning Conference (RLC), 2024

   Abstract Bib PDF

   Offline reinforcement learning (RL) is a promising direction that allows RL agents to pre-train on large datasets, avoiding the recurrence of expensive data collection. To advance the field, it is crucial to generate large-scale datasets. Compositional RL is particularly appealing for generating such large datasets, since 1) it permits creating many tasks from few components, 2) the task structure may enable trained agents to solve new tasks by combining relevant learned components, and 3) the compositional dimensions provide a notion of task relatedness. This paper provides four offline RL datasets for simulated robotic manipulation created using the 256 tasks from CompoSuite (Mendez et al., 2022a). Each dataset is collected from an agent with a different degree of performance, and consists of 256 million transitions. We provide training and evaluation settings for assessing an agent’s ability to learn compositional task policies. Our benchmarking experiments show that current offline RL methods can learn the training tasks to some extent and that compositional methods outperform non-compositional methods. Yet current methods are unable to extract the compositional structure to generalize to unseen tasks, highlighting a need for future research in offline compositional RL.

   @inproceedings{Hussing2024RoboticManipulation,
     author = {Hussing, Marcel and Mendez-Mendez, Jorge and Singrodia, Anisha and Kent, Cassandra and Eaton, Eric},
     year = {2024},
     title = {Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning},
     booktitle = {1st Reinforcement Learning Conference (RLC)},
   }

3. ZeroFlow: Scalable Scene Flow via Distillation

   Kyle Vedder, Neehar Peri, Nathaniel Eliot Chodosh, and 6 more authors

   In International Conference on Learning Representations (ICLR), 2024

   Abstract Bib PDF Website  

   Scene flow estimation is the task of describing the 3D motion field between temporally successive point clouds. State-of-the-art methods use strong priors and test-time optimization techniques, but require on the order of tens of seconds to process full-size point clouds, making them unusable as computer vision primitives for real-time applications such as open world object detection. Feedforward methods are considerably faster, running on the order of tens to hundreds of milliseconds for full-size point clouds, but require expensive human supervision. To address both limitations, we propose Scene Flow via Distillation, a simple, scalable distillation framework that uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model. Our instantiation of this framework, ZeroFlow, achieves state-of-the-art performance on the Argoverse 2 Self-Supervised Scene Flow Challenge while using zero human labels by simply training on large-scale, diverse unlabeled data. At test-time, ZeroFlow is over 1000× faster than label-free state-of-the-art optimization-based methods on full-size point clouds (34 FPS vs 0.028 FPS) and over 1000× cheaper to train on unlabeled data compared to the cost of human annotation (394 vs ∼750,000). To facilitate further research, we release our code, trained model weights, and high quality pseudo-labels for the Argoverse 2 and Waymo Open datasets at https://vedder.io/zeroflow.

   @inproceedings{Vedder2024ZeroFlow,
     author = {Vedder, Kyle and Peri, Neehar and Chodosh, Nathaniel Eliot and Khatri, Ishan and Eaton, Eric and Jayaraman, Dinesh and Liu, Yang and Ramanan, Deva and Hays, James},
     year = {2024},
     title = {ZeroFlow: Scalable Scene Flow via Distillation},
     booktitle = {International Conference on Learning Representations (ICLR)},
   }

4. A Metacognitive Approach to Out-of-Distribution Detection for Segmentation

   Meghna Gummadi, Cassandra Kent, Karl Schmeckpeper, and 1 more author

   In International Conference on Robotics and Automation (ICRA), 2024

   Abstract arXiv Bib PDF Website  

   Despite outstanding semantic scene segmentation in closed-worlds, deep neural networks segment novel instances poorly, which is required for autonomous agents acting in an open world. To improve out-of-distribution (OOD) detection for segmentation, we introduce a metacognitive approach in the form of a lightweight module that leverages entropy measures, segmentation predictions, and spatial context to characterize the segmentation model’s uncertainty and detect pixel-wise OOD data in real-time. Additionally, our approach incorporates a novel method of generating synthetic OOD data in context with in-distribution data, which we use to fine-tune existing segmentation models with maximum entropy training. This further improves the metacognitive module’s performance without requiring access to OOD data while enabling compatibility with established pre-trained models. Our resulting approach can reliably detect OOD instances in a scene, as shown by state-of-the-art performance on OOD detection for semantic segmentation benchmarks.

   @inproceedings{Gummadi2024Metacognitive,
     author = {Gummadi, Meghna and Kent, Cassandra and Schmeckpeper, Karl and Eaton, Eric},
     year = {2024},
     title = {A Metacognitive Approach to Out-of-Distribution Detection for Segmentation},
     booktitle = {International Conference on Robotics and Automation (ICRA)},
   }

5. Artificial Intelligence in the CS2023 Undergraduate Computer Science Curriculum: Rationale and Challenges

   Eric Eaton and Susan L. Epstein

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2024

   Abstract DOI Bib PDF

   Roughly every decade, the ACM and IEEE professional organizations have produced recommendations for the education of undergraduate computer science students. These guidelines are used worldwide by research universities, liberal arts colleges, and community colleges. For the latest 2023 revision of the curriculum, AAAI has collaborated with ACM and IEEE to integrate artificial intelligence more broadly into this new curriculum and to address the issues it raises for students, instructors, practitioners, policy makers, and the general public. This paper describes the development process and rationale that underlie the artificial intelligence components of the CS2023 curriculum, discusses the challenges in curriculum design for such a rapidly advancing field, and examines lessons learned during this three-year process.

   @inproceedings{Eaton2024AICurriculum,
     author = {Eaton, Eric and Epstein, Susan L.},
     year = {2024},
     title = {Artificial Intelligence in the CS2023 Undergraduate Computer Science Curriculum: Rationale and Challenges},
     booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
     volume = {38},
     number = {21},
     pages = {23078--23083},
     doi = {10.1609/aaai.v38i21.30352}
   }

6. Predicting the Effect of Proton Beam Therapy Technology on Pulmonary Toxicities for Patients With Locally Advanced Lung Cancer Enrolled in the Proton Collaborative Group

   Gilmer Valdes, Jessica Scholey, Tomi F Nano, and 14 more authors

   International Journal of Radiation Oncology, Biology, Physics, 2024

   Abstract DOI Bib Website  

   Purpose: This study aimed to predict the probability of grade ≥2 pneumonitis or dyspnea within 12 months of receiving conventionally fractionated or mildly hypofractionated proton beam therapy for locally advanced lung cancer using machine learning. Methods and Materials: Demographic and treatment characteristics were analyzed for 965 consecutive patients treated for lung cancer with conventionally fractionated or mildly hypofractionated (2.2-3 Gy/fraction) proton beam therapy across 12 institutions. Three machine learning models (gradient boosting, additive tree, and logistic regression with lasso regularization) were implemented to predict Common Terminology Criteria for Adverse Events version 4 grade ≥2 pulmonary toxicities using double 10-fold cross-validation for parameter hyper-tuning without leak of information. Balanced accuracy and area under the curve were calculated, and 95% confidence intervals were obtained using bootstrap sampling. Results: The median age of the patients was 70 years (range, 20-97), and they had predominantly stage IIIA or IIIB disease. They received a median dose of 60 Gy in 2 Gy/fraction, and 46.4% received concurrent chemotherapy. In total, 250 (25.9%) had grade ≥2 pulmonary toxicity. The probability of pulmonary toxicity was 0.08 for patients treated with pencil beam scanning and 0.34 for those treated with other techniques (P = 8.97e-13). Use of abdominal compression and breath hold were highly significant predictors of less toxicity (P = 2.88e-08). Higher total radiation delivered dose (P = .0182) and higher average dose to the ipsilateral lung (P = .0035) increased the likelihood of pulmonary toxicities. The gradient boosting model performed the best of the models tested, and when demographic and dosimetric features were combined, the area under the curve and balanced accuracy were 0.75 ± 0.02 and 0.67 ± 0.02, respectively. After analyzing performance versus the number of data points used for training, we observed that accuracy was limited by the number of observations. Conclusions: In the largest analysis of prospectively enrolled patients with lung cancer assessing pulmonary toxicities from proton therapy to date, advanced machine learning methods revealed that pencil beam scanning, abdominal compression, and lower normal lung doses can lead to significantly lower probability of developing grade ≥2 pneumonitis or dyspnea.

   @article{Valdes2024PredictingTheEffect,
     author = {Valdes, Gilmer and Scholey, Jessica and Nano, Tomi F and Gennatas, Efstathios D and Mohindra, Pranshu and Mohammed, Nasir and Zeng, Jing and Kotecha, Rupesh and Rosen, Lane R and Chang, John and Tsai, Henry K and Urbanic, James J and Vargas, Carlos E and Nathan, Y Yu and Ungar, Lyle H and Eaton, Eric and {Simone II}, Charles B},
     year = {2024},
     title = {Predicting the Effect of Proton Beam Therapy Technology on Pulmonary Toxicities for Patients With Locally Advanced Lung Cancer Enrolled in the Proton Collaborative Group},
     journal = {International Journal of Radiation Oncology, Biology, Physics},
     volume = {119},
     number = {1},
     pages = {66--77},
     doi = {10.1016/j.ijrobp.2023.11.026},
     url = {https://www.redjournal.org/article/S0360-3016(23)08154-3/abstract},
   }

7. Text message system for the prediction of colonoscopy bowel preparation adequacy prior to colonoscopy: an artificial intelligence-image classification algorithm based on images of stool output

   Chethan Ramprasad, Divya Saini, Henry Del Carmen, and 8 more authors

   Gastro Hep Advances, 2024

   Abstract Bib Website  

   Background and Aims: Inadequate bowel preparation which occurs in 25% of colonoscopies is a major barrier to the effectiveness of screening for colorectal cancer. We aim to develop an artificial intelligence (machine learning) algorithm to assess photos of stool output after bowel preparation to predict inadequate bowel preparation before colonoscopy. Methods: Patients were asked to text a photo of their stool in the commode when they believed that they neared completion of their colonoscopy bowel preparation. Boston Bowel Preparation Scores of 7 and below were labeled as inadequate or fair. Boston Bowel Preparation Scores of 8 and 9 were considered good. A binary classification image-based machine learning algorithm was designed. Results: In a test set of 61 images, the binary classification machine learning algorithm was able to distinguish inadequate/fair preparation from good preparation with a positive predictive value of 78.6% and a negative predictive value of 60.8%. In a test set of 56 images, the algorithm was able to distinguish normal colonoscopy duration (<25 minutes) from long colonoscopy duration (>25 minutes) with a positive predictive value of 78.6% and a negative predictive value of 65.5%. Conclusion: Patients are willing to submit photos of their stool output during bowel preparation through text messages before colonoscopy. This machine learning algorithm demonstrates the ability to predict inadequate/fair preparation from good preparation based on image classification of stool output. It was less accurate to predict long duration of colonoscopy.

   @article{Ramprasad2024TextMessageSystem,
     author = {Ramprasad, Chethan and Saini, Divya and Carmen, Henry Del and Krasnovsky, Lev and Chandra, Rajat and Mcgregor, Ryan and Shinohara, Russell T. and Eaton, Eric and Gummadi, Meghna and Mehta, Shivan and Lewis, James D.},
     year = {2024},
     title = {Text message system for the prediction of colonoscopy bowel preparation adequacy prior to colonoscopy: an artificial intelligence-image classification algorithm based on images of stool output},
     journal = {Gastro Hep Advances},
     url = {https://www.ghadvances.org/article/S2772-5723(24)00150-X/fulltext}
   }

8. A collective AI via lifelong learning and sharing at the edge

   Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Vladimir Braverman, and 27 more authors

   Nature Machine Intelligence, Mar 2024

   Abstract DOI Bib Website  

   One vision of a future artificial intelligence (AI) is where many separate units can learn independently over a lifetime and share their knowledge with each other. The synergy between lifelong learning and sharing has the potential to create a society of AI systems, as each individual unit can contribute to and benefit from the collective knowledge. Essential to this vision are the abilities to learn multiple skills incrementally during a lifetime, to exchange knowledge among units via a common language, to use both local data and communication to learn, and to rely on edge devices to host the necessary decentralized computation and data. The result is a network of agents that can quickly respond to and learn new tasks, that collectively hold more knowledge than a single agent and that can extend current knowledge in more diverse ways than a single agent. Open research questions include when and what knowledge should be shared to maximize both the rate of learning and the long-term learning performance. Here we review recent machine learning advances converging towards creating a collective machine-learned intelligence. We propose that the convergence of such scientific and technological advances will lead to the emergence of new types of scalable, resilient and sustainable AI systems.

   @article{Soltoggio2024CollectiveAI,
     author = {Soltoggio, Andrea and Ben-Iwhiwhu, Eseoghene and Braverman, Vladimir and Eaton, Eric and Epstein, Benjamin and Ge, Yunhao and Halperin, Lucy and How, Jonathan and Itti, Laurent and Jacobs, Michael A. and Kantharaju, Pavan and Le, Long and Lee, Steven and Liu, Xinran and Monteiro, Sildomar T. and Musliner, David and Nath, Saptarshi and Panda, Priyadarshini and Peridis, Christos and Pirsiavash, Hamed and Parekh, Vishwa and Roy, Kaushik and Shperberg, Shahaf and Siegelmann, Hava T. and Stone, Peter and Vedder, Kyle and Wu, Jingfeng and Yang, Lin and Zheng, Guangyao and Kolouri, Soheil},
     year = {2024},
     title = {A collective AI via lifelong learning and sharing at the edge},
     journal = {Nature Machine Intelligence},
     volume = {6},
     pages = {251--264},
     month = mar,
     doi = {10.1038/s42256-024-00800-2},
     url = {https://www.nature.com/articles/s42256-024-00800-2},
   }

9. Learning source domain representations for electro-optical to SAR transfer

   Boya Zeng, Marcel Hussing, and Eric Eaton

   In ICLR 2024 Machine Learning for Remote Sensing (ML4RS) Workshop, Mar 2024

   Abstract Bib PDF

   Embedding distribution alignment is an approach to transfer knowledge from label-abundant electro-optical (EO) images to the label-scarce synthetic aperture radar (SAR) modality. However, this approach assumes that it is possible to learn a useful and discriminative EO representation via a neural network. In this work, we study the properties of such a representation. We analyze a recent result showing that supervised contrastive learning can improve transfer performance and find that its reduction of the effective dimension of the embedding is crucial to successful transfer. We then show that directly optimizing for this property can yield even better down-stream accuracy. Finally, we show that the powerful representation of an EO foundation model is insufficient for alignment due to its generality, but that additional representation learning can recover alignment performance.

   @inproceedings{Zeng2024ElectroOptical,
     author = {Zeng, Boya and Hussing, Marcel and Eaton, Eric},
     year = {2024},
     title = {Learning source domain representations for electro-optical to SAR transfer},
     booktitle = {ICLR 2024 Machine Learning for Remote Sensing (ML4RS) Workshop},
   }

2023

1. Prospective Learning: Principled Extrapolation to the Future

   Ashwin De Silva, Rahul Ramesh, Lyle Ungar, and 39 more authors

   In Conference on Lifelong Learning Agents (CoLLAs), Mar 2023

   Abstract Bib PDF

   Learning is a process which can update decision rules, based on past experience, such that future performance improves. Traditionally, machine learning is often evaluated under the assumption that the future will be identical to the past in distribution or change adversarially. But these assumptions can be either too optimistic or pessimistic for many problems in the real world. Real world scenarios evolve over multiple spatiotemporal scales with partially predictable dynamics. Here we reformulate the learning problem to one that centers around this idea of dynamic futures that are partially learnable. We conjecture that certain sequences of tasks are not retrospectively learnable (in which the data distribution is fixed), but are prospectively learnable (in which distributions may be dynamic), suggesting that prospective learning is more difficult in kind than retrospective learning. We argue that prospective learning more accurately characterizes many real world problems that (1) currently stymie existing artificial intelligence solutions and/or (2) lack adequate explanations for how natural intelligences solve them. Thus, studying prospective learning will lead to deeper insights and solutions to currently vexing challenges in both natural and artificial intelligences.

   @inproceedings{DeSilva2023ProspectiveLearning,
     author = {Silva, Ashwin De and Ramesh, Rahul and Ungar, Lyle and Shuler, Marshall Hussain and Cowan, Noah J and Platt, Michael and Li, Chen and Isik, Leyla and Roh, Seung-Eon and Charles, Adam and Venkataraman, Archana and Caffo, Brian and How, Javier J and Kebschull, Justus M and Krakauer, John W and Bichuch, Maxim and Kinfu, Kaleab Alemayehu and Yezerets, Eva and Jayaraman, Dinesh and Shin, Jong M and Villar, Soledad and Phillips, Ian and Priebe, Carey E and Hartung, Thomas and Miller, Michael I and Dey, Jayanta and Huang, Ningyuan and Eaton, Eric and Etienne-Cummings, Ralph and Ogburn, Elizabeth L and Burns, Randal and Osuagwu, Onyema and Mensh, Brett and Muotri, Alysson R and Brown, Julia and White, Chris and Yang, Weiwei and Rusu, Andrei A and Verstynen, Timothy and Kording, Konrad P and Chaudhari, Pratik and Vogelstein, Joshua T},
     year = {2023},
     title = {Prospective Learning: Principled Extrapolation to the Future},
     booktitle = {Conference on Lifelong Learning Agents (CoLLAs)},
     pages = {347--357},
   }

2. Replicable Reinforcement Learning

   Eric Eaton, Marcel Hussing, Michael Kearns, and 1 more author

   In Neural Information Processing Systems, Mar 2023

   Abstract Bib PDF

   The replicability crisis in the social, behavioral, and data sciences has led to the formulation of algorithm frameworks for replicability — i.e., a requirement that an algorithm produce identical outputs (with high probability) when run on two different samples from the same underlying distribution. While still in its infancy, provably replicable algorithms have been developed for many fundamental tasks in machine learning and statistics, including statistical query learning, the heavy hitters problem, and distribution testing. In this work we initiate the study of replicable reinforcement learning, providing a provably replicable algorithm for parallel value iteration, and a provably replicable version of R-max in the episodic setting. These are the first formal replicability results for control problems, which present different challenges for replication than batch learning settings.

   @inproceedings{Eaton2023ReplicableRL,
     author = {Eaton, Eric and Hussing, Marcel and Kearns, Michael and Sorrell, Jessica},
     year = {2023},
     title = {Replicable Reinforcement Learning},
     booktitle = {Neural Information Processing Systems},
   }

3. How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition

   Jorge Mendez and Eric Eaton

   Transactions on Machine Learning Research, Jun 2023

   Abstract Bib PDF

   A major goal of artificial intelligence (AI) is to create an agent capable of acquiring a general understanding of the world. Such an agent would require the ability to continually accumulate and build upon its knowledge as it encounters new experiences. Lifelong or continual learning addresses this setting, whereby an agent faces a continual stream of problems and must strive to capture the knowledge necessary for solving each new task it encounters. If the agent is capable of accumulating knowledge in some form of compositional representation, it could then selectively reuse and combine relevant pieces of knowledge to construct novel solutions. Despite the intuitive appeal of this simple idea, the literatures on lifelong learning and compositional learning have proceeded largely separately. In an effort to promote developments that bridge between the two fields, this article surveys their respective research landscapes and discusses existing and future connections between them.

   @article{Mendez2023CompositionalSurvey,
     author = {Mendez, Jorge and Eaton, Eric},
     year = {2023},
     title = {How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition},
     journal = {Transactions on Machine Learning Research},
     month = jun,
   }

4. A domain-agnostic approach for characterization of lifelong learning systems

   Megan M Baker, Alexander New, Mario Aguilar-Simon, and 44 more authors

   Neural Networks, Jun 2023

   Abstract Bib Website  

   Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to “real world” events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of “Lifelong Learning” systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

   @article{Baker2023CollectiveAI,
     author = {Baker, Megan M and New, Alexander and Aguilar-Simon, Mario and Al-Halah, Ziad and Arnold, Sbastien MR and Ben-Iwhiwhu, Ese and Brna, Andrew P and Brooks, Ethan and Brown, Ryan C and Daniels, Zachary and Daram, Anurag and Delattre, Fabien and Dellana, Ryan and Eaton, Eric and Fu, Haotian and Grauman, Kristen and Hostetler, Jesse and Iqbal, Shariq and Kent, Cassandra and Ketz, Nicholas and Kolouri, Soheil and Konidaris, George and Kudithipudi, Dhireesha and Learned-Miller, Erik and Lee, Seungwon and Littman, Michael L and Madireddy, Sandeep and Mendez, Jorge A and Nguyen, Eric Q and Piatko, Christine and Pilly, Praveen K and Raghavan, Aswin and Rahman, Abrar and Ramakrishnan, Santhosh Kumar and Ratzlaff, Neale and Soltoggio, Andrea and Stone, Peter and Sur, Indranil and Tang, Zhipeng and Tiwari, Saket and Vedder, Kyle and Wang, Felix and Xu, Zifan and Yanguas-Gil, Angel and Yedidsion, Harel and Yu, Shangqun and Vallabha, Gautam K},
     year = {2023},
     title = {A domain-agnostic approach for characterization of lifelong learning systems},
     journal = {Neural Networks},
     volume = {160},
     pages = {274--296},
     url = {https://arxiv.org/pdf/2301.07799},
   }

5. Gap Minimization for Knowledge Sharing and Transfer

   Boyu Wang, Jorge Mendez, Changjian Shui, and 4 more authors

   Journal of Machine Learning Research, Jun 2023

   Abstract Bib PDF

   Learning from multiple related tasks by knowledge sharing and transfer has become increasingly relevant over the last two decades. In order to successfully transfer information from one task to another, it is critical to understand the similarities and differences between the domains. In this paper, we introduce the notion of performance gap, an intuitive and novel measure of the distance between learning tasks. Unlike existing measures which are used as tools to bound the difference of expected risks between tasks (e.g., H-divergence or discrepancy distance), we theoretically show that the performance gap can be viewed as a data- and algorithm-dependent regularizer, which controls the model complexity and leads to finer guarantees. More importantly, it also provides new insights and motivates a novel principle for designing strategies for knowledge sharing and transfer: gap minimization. We instantiate this principle with two algorithms: 1. gapBoost, a novel and principled boosting algorithm that explicitly minimizes the performance gap between source and target domains for transfer learning; and 2. gapMTNN, a representation learning algorithm that reformulates gap minimization as semantic conditional matching for multitask learning. Our extensive evaluation on both transfer learning and multitask learning benchmark data sets shows that our methods outperform existing baselines.

   @article{Wang2023GapMinimization,
     author = {Wang, Boyu and Mendez, Jorge and Shui, Changjian and Zhou, Fan and Wu, Di and Gagn\'{e}, Christian and Eaton, Eric},
     year = {2023},
     title = {Gap Minimization for Knowledge Sharing and Transfer},
     journal = {Journal of Machine Learning Research},
     volume = {24},
     pages = {1--57},
   }

6. Expanding Capacity and Diversity in Lifelong AI Education

   Mary Lou Maher, Razvan Bunescu, Stephanie August, and 9 more authors

   In NSF Workshop: Expanding Capacity and Diversity in AI Education, Jun 2023

   Abstract Bib Website  

   This report presents the discussion summaries of the three sessions on AI curricula, quality and inclusion, and expanding capacity in AI education. The first session identifies the topics, knowledge, and skills areas in AI education at different levels, from K-12 to adult workforce, and emphasizes the need for ethical, responsible, and trustworthy AI education. The second session proposes guidelines for ensuring quality and inclusion in AI education, such as rigor, theory, accreditation, diversity, and stakeholder recognition. The third session explores strategies for expanding the capacity of AI education, such as infrastructure, people, courses, and targeting. Highlights of the findings include: 1. Quality AI Education: Guidelines for quality AI education across all levels of education are being addressed by organizations such as TeachAI.org and the ACM/AAAI Committees for curriculum guidelines. A gap in the development of curriculum guidelines exists for public AI literacy and the need to enhance the offerings and effectiveness of adult AI education. 2. Increasing Capacity in AI Education: Enhancing AI capacity requires a combination of revised educational approaches, informed partnerships, and a commitment to fostering an inclusive and adaptable AI understanding across disciplines. Short term returns are in increasing AI in adult learning and higher education. 3. Increasing Diversity in AI education: A holistic and data-driven approach is essential for fostering diversity and equity in AI education and ensuring that it is accessible and beneficial to all. Collaboration between academia and industry, hands-on experiences, and the incorporation of real-world contexts and ethical considerations are key themes. 4. AI in education: The integration of AI in education and the profession is poised to provide personalized support for anytime learning. AI in education will also redefine the equilibrium between theoretical understanding and skill acquisition. As AI automates certain tasks and challenges traditional learning pathways, the educational community must reassess the primary objectives of learning to ensure a holistic development of learners in this new paradigm.

   @inproceedings{Maher2023ExpandingCapacity,
     author = {Maher, Mary Lou and Bunescu, Razvan and August, Stephanie and Eaton, Eric and Fisher, Douglas and Gardner-McCune, Christina and Goel, Ashok and Gil, Yolanda and Sahami, Mehran and Simmons, Reid and Touretzky, David and Yongpradit, Pat},
     year = {2023},
     title = {Expanding Capacity and Diversity in Lifelong AI Education},
     booktitle = {NSF Workshop: Expanding Capacity and Diversity in AI Education},
     url = {https://sites.google.com/uncc.edu/ai-education-workshop/},
   }

7. Fuzzy Logic++: Towards Developing Fuzzy Education Curricula Using ACM/IEEE/AAAI CS2023

   Christian Servin, Brett A. Becker, Eric Eaton, and 1 more author

   In North American Fuzzy Information Processing Society Annual Conference, Jun 2023

   Award   Abstract Bib Website  

   Best Paper

   Fuzzy Logic education has been considered part of artificial intelligence, which in many cases, can be found in the electives curricula or, inclusively, at the graduate studies for the past decades. Because of the persistent emergence of computing areas, such as Artificial Intelligence and Machine Learning, Robotics, and others, there is an appetite for recognizing concepts on intelligent systems in undergraduate education. The ACM/IEEE/AAAI Computer Science Curricula, also known as CS2023, has proposed significant changes from the last version CS2013, particularly in artificial intelligence. These proposed recommendations will have an impact in the next ten years in computer science undergraduate education, including fundamentals of computer programming. This work presents the changes that may impact computer science education curricula, particularly the knowledge areas and competencies model that will influence fuzzy logic education and computing. We also present the intersection of knowledge areas in fuzzy logic, especially for explainable AI.

   @inproceedings{Servin2023FuzzyLogicPlusPlus,
     author = {Servin, Christian and Becker, Brett A. and Eaton, Eric and Kumar, Amruth},
     year = {2023},
     title = {Fuzzy Logic++: Towards Developing Fuzzy Education Curricula Using ACM/IEEE/AAAI CS2023},
     booktitle = {North American Fuzzy Information Processing Society Annual Conference},
     pages = {184--193},
     url = {https://link.springer.com/chapter/10.1007/978-3-031-46778-3_17}
   }

8. PD-0173 Proton center variations in predicting pulmonary toxicities from proton radiotherapy of lung cancer

   T. Nano, G. Valdes, J. Scholey, and 14 more authors

   Radiotherapy and Oncology, Jun 2023

   DOI Bib Website  

   @article{Nano2023ProtonCenter,
     author = {Nano, T. and Valdes, G. and Scholey, J. and Comas-Leon, A. and Gennatas, E. and Hartsell, W. and Zeng, J. and Chuong, M. and Mishra, M. and Rosen, L. and Chang, J. and Tsai, H. and Urbanic, J. and Vargas, C. and Ungar, L. and Eaton, Eric and Simone, C.},
     year = {2023},
     title = {PD-0173 Proton center variations in predicting pulmonary toxicities from proton radiotherapy of lung cancer},
     journal = {Radiotherapy and Oncology},
     volume = {182},
     pages = {S134},
     doi = {10.1016/S0167-8140(23)08817-5},
     url = {https://www.thegreenjournal.com/article/S0167-8140(23)08817-5/abstract}
   }

2022

1. Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems

   Kyle Vedder and Eric Eaton

   In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jun 2022

   Abstract Bib PDF Website  

   Bird’s Eye View (BEV) is a popular representation for processing 3D point clouds, and by its nature is fundamentally sparse. Motivated by the computational limitations of mobile robot platforms, we create a fast, high-performance BEV 3D object detector that maintains and exploits this input sparsity to decrease runtimes over non-sparse baselines and avoids the tradeoff between pseudoimage area and runtime. We present results on KITTI, a canonical 3D detection dataset, and Matterport-Chair, a novel Matterport3D-derived chair detection dataset from scenes in real furnished homes. We evaluate runtime characteristics using a desktop GPU, an embedded ML accelerator, and a robot CPU, demonstrating that our method results in significant detection speedups (2 × or more) for embedded systems with only a modest decrease in detection quality. Our work represents a new approach for practitioners to optimize models for embedded systems by maintaining and exploiting input sparsity throughout their entire pipeline to reduce runtime and resource usage while preserving detection performance. All models, weights, experimental configurations, and datasets used are publicly available at https://vedder.io/sparse_point_pillars.

   @inproceedings{Vedder2022SparsePointPillars,
     author = {Vedder, Kyle and Eaton, Eric},
     year = {2022},
     title = {Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems},
     booktitle = {Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
     pages = {2025--2031},
     url = {https://ieeexplore.ieee.org/document/9981550},
   }

2. SHELS: Exclusive feature sets for novelty detection and continual learning without class boundaries

   Meghna Gummadi, Cassandra Kent, Jorge Mendez, and 1 more author

   In Conference on Lifelong Learning Agents (CoLLAs), Jun 2022

   Abstract Bib PDF

   While deep neural networks (DNNs) have achieved impressive classification performance in closed-world learning scenarios, they typically fail to generalize to unseen categories in dynamic open-world environments, in which the number of concepts is unbounded. In contrast, human and animal learners have the ability to incrementally update their knowledge by recognizing and adapting to novel observations. In particular, humans characterize concepts via exclusive (unique) sets of essential features, which are used for both recognizing known classes and identifying novelty. Inspired by natural learners, we introduce a Sparse High-level-Exclusive, Low-level-Shared feature representation (SHELS) that simultaneously encourages learning exclusive sets of high-level features and essential, shared low-level features. The exclusivity of the high-level features enables the DNN to automatically detect out-of-distribution (OOD) data, while the efficient use of capacity via sparse low-level features permits accommodating new knowledge. The resulting approach uses OOD detection to perform class-incremental continual learning without known class boundaries. We show that using SHELS for novelty detection results in statistically significant improvements over state-of-the-art OOD detection approaches over a variety of benchmark datasets. Further, we demonstrate that the SHELS model mitigates catastrophic forgetting in a class-incremental learning setting, enabling a combined novelty detection and accommodation framework that supports learning in open-world settings.

   @inproceedings{Gummadi2022SHELS,
     author = {Gummadi, Meghna and Kent, Cassandra and Mendez, Jorge and Eaton, Eric},
     year = {2022},
     title = {SHELS: Exclusive feature sets for novelty detection and continual learning without class boundaries},
     booktitle = {Conference on Lifelong Learning Agents (CoLLAs)},
   }

3. CompoSuite: A compositional reinforcement learning benchmark

   Jorge Mendez, Marcel Hussing, Meghna Gummadi, and 1 more author

   In Conference on Lifelong Learning Agents (CoLLAs), Jun 2022

   Abstract Bib PDF

   We present CompoSuite, an open-source simulated robotic manipulation benchmark for compositional multi-task reinforcement learning (RL). Each CompoSuite task requires a particular robot arm to manipulate one individual object to achieve a task objective while avoiding an obstacle. This compositional definition of the tasks endows CompoSuite with two remarkable properties. First, varying the robot/object/objective/obstacle elements leads to hundreds of RL tasks, each of which requires a meaningfully different behavior. Second, RL approaches can be evaluated specifically for their ability to learn the compositional structure of the tasks. This latter capability to functionally decompose problems would enable intelligent agents to identify and exploit commonalities between learning tasks to handle large varieties of highly diverse problems. We benchmark existing single-task, multi-task, and compositional learning algorithms on various training settings, and assess their capability to compositionally generalize to unseen tasks. Our evaluation exposes the shortcomings of existing RL approaches with respect to compositionality and opens new avenues for investigation.

   @inproceedings{Mendez2022CompoSuite,
     author = {Mendez, Jorge and Hussing, Marcel and Gummadi, Meghna and Eaton, Eric},
     year = {2022},
     title = {CompoSuite: A compositional reinforcement learning benchmark},
     booktitle = {Conference on Lifelong Learning Agents (CoLLAs)},
   }

4. Modular lifelong reinforcement learning via neural composition

   Jorge Mendez, Harm Seijen, and Eric Eaton

   In Proceedings of the International Conference on Learning Representations (ICLR), Jun 2022

   Abstract Bib PDF

   Humans commonly solve complex problems by decomposing them into easier subproblems and then combining the subproblem solutions. This type of compositional reasoning permits reuse of the subproblem solutions when tackling future tasks that share part of the underlying compositional structure. In a continual or lifelong reinforcement learning (RL) setting, this ability to decompose knowledge into reusable components would enable agents to quickly learn new RL tasks by leveraging accumulated compositional structures. We explore a particular form of composition based on neural modules and present a set of RL problems that intuitively admit compositional solutions. Empirically, we demonstrate that neural composition indeed captures the underlying structure of this space of problems. We further propose a compositional lifelong RL method that leverages accumulated neural components to accelerate the learning of future tasks while retaining performance on previous tasks via off-line RL over replayed experiences.

   @inproceedings{Mendez2022ModularLifelongRL,
     author = {Mendez, Jorge and van Seijen, Harm and Eaton, Eric},
     year = {2022},
     title = {Modular lifelong reinforcement learning via neural composition},
     booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)},
   }

5. Zero-Shot Image Classification Using Coupled Dictionary Embedding

   Mohammad Rostami, Soheil Kolouri, Zak Murez, and 3 more authors

   Machine Learning with Applications, Jun 2022

   Abstract Bib PDF

   Zero-shot learning (ZSL) is a framework to classify images that belong to unseen visual classes using their semantic descriptions about the unseen classes. We develop a new ZSL algorithm based on coupled dictionary learning. The core idea is to enforce the visual features and the semantic attributes of an image to share the same sparse representation in an intermediate embedding space, modeled as the shared input space of two sparsifying dictionaries. In the ZSL training stage, we use images from a number of seen classes for which we have access to both the visual and the semantic attributes to train two coupled dictionaries that can represent both the visual and the semantic feature vectors of an image using a single sparse vector. In the ZSL testing stage and in the absence of labeled data, images from unseen classes are mapped into the attribute space by finding the joint-sparse representations using solely the visual dictionary via solving a LASSO problem. The image is then classified in the attribute space given semantic descriptions of unseen classes. We also provide attributeaware and transductive formulations to tackle the “domain-shift” and the “hubness” challenges for ZSL, respectively. Experiments on four primary datasets using VGG19 and GoogleNet visual features, are provided. Our performances using VGG19 features are 91.0%, 48.4%, and 89.3% on the SUN , the CUB, and the AwA1 datasets, respectively. Our performances on the SUN, the CUB, and the AwA2 datasets are 57.0%,49.7%, and 71.7%, respectively, when GoogleNet features are used. Comparison with existing methods demonstrates that our method is effective and compares favorably against the sate-of-the-art. In particular, our algorithm leads to decent performance on the all four datasets.

   @article{Rostami2022ZeroShot,
     author = {Rostami, Mohammad and Kolouri, Soheil and Murez, Zak and Owechko, Yuri and Eaton, Eric and Kim, Kuyngnam},
     year = {2022},
     title = {Zero-Shot Image Classification Using Coupled Dictionary Embedding},
     journal = {Machine Learning with Applications},
     volume = {8},
     pages = {100278},
   }

6. Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning

   Marcel Hussing, Jorge Mendez, Cassandra Kent, and 1 more author

   In CoRL 2022 Workshop on Pre-training Robot Learning, Jun 2022

   Superseded by the RLC 2024 version

   Abstract Bib PDF

   Offline reinforcement learning (RL) is a promising direction that allows RL agents to be pre-trained from large datasets avoiding recurrence of expensive data collection. To advance the field, it is crucial to generate large-scale datasets. Compositional RL is particularly appealing for generating such large datasets, since 1) it permits creating many tasks from few components, and 2) the task structure may enable trained agents to solve new tasks by combining relevant learned components. This paper provides four offline RL datasets for simulated robotic manipulation created using the 256 tasks from CompoSuite [1]. Each dataset is collected from an agent with a different degree of performance, and consists of 256 million transitions. We provide training and evaluation settings for assessing an agent’s ability to learn compositional task policies. Our benchmarking experiments on each setting show that current offline RL methods can learn the training tasks to some extent, but are unable to extract their compositional structure to generalize to unseen tasks, showing a need for further research in offline compositional RL.

   @inproceedings{Hussing2022RoboticManipulation,
     author = {Hussing, Marcel and Mendez, Jorge and Kent, Cassandra and Eaton, Eric},
     year = {2022},
     title = {Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning},
     booktitle = {CoRL 2022 Workshop on Pre-training Robot Learning},
   }

7. Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning

   Marcel Hussing, Karen Li, and Eric Eaton

   In NeurIPS 2022 Workshop on Tackling Climate Change with Machine Learning, Jun 2022

   Abstract Bib PDF

   Satellite image analysis has important implications for land use, urbanization, and ecosystem monitoring. Deep learning methods can facilitate the analysis of different satellite modalities, such as electro-optical (EO) and synthetic aperture radar (SAR) imagery, by supporting knowledge transfer between the modalities to compensate for individual shortcomings. Recent progress has shown how distributional alignment of neural network embeddings can produce powerful transfer learning models by employing a sliced Wasserstein distance (SWD) loss. We analyze how this method can be applied to Sentinel-1 and -2 satellite imagery and develop several extensions toward making it effective in practice. In an application to few-shot Local Climate Zone (LCZ) prediction, we show that these networks outperform multiple common baselines on datasets with a large number of classes. Further, we provide evidence that instance normalization can significantly stabilize the training process and that explicitly shaping the embedding space using supervised contrastive learning can lead to improved performance.

   @inproceedings{Hussing2022LandUsePrediction,
     author = {Hussing, Marcel and Li, Karen and Eaton, Eric},
     year = {2022},
     title = {Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning},
     booktitle = {NeurIPS 2022 Workshop on Tackling Climate Change with Machine Learning},
   }

2021

1. Lifelong learning of compositional structures

   Jorge Mendez and Eric Eaton

   In International Conference on Learning Representations, Jun 2021

   Abstract Bib PDF

   A hallmark of human intelligence is the ability to construct self-contained chunks of knowledge and adequately reuse them in novel combinations for solving different yet structurally related problems. Learning such compositional structures has been a significant challenge for artificial systems, due to the combinatorial nature of the underlying search problem. To date, research into compositional learning has largely proceeded separately from work on lifelong or continual learning. We integrate these two lines of work to present a general-purpose framework for lifelong learning of compositional structures that can be used for solving a stream of related tasks. Our framework separates the learning process into two broad stages: learning how to best combine existing components in order to assimilate a novel problem, and learning how to adapt the set of existing components to accommodate the new problem. This separation explicitly handles the trade-off between the stability required to remember how to solve earlier tasks and the flexibility required to solve new tasks, as we show empirically in an extensive evaluation.

   @inproceedings{Mendez2021Lifelong,
     title = {Lifelong learning of compositional structures},
     author = {Mendez, Jorge and Eaton, Eric},
     booktitle = {International Conference on Learning Representations},
     year = {2021},
   }

2. Sharing less is more: Lifelong learning in deep networks with selective layer transfer

   Seungwon Lee, Sima Behpour, and Eric Eaton

   In Proceedings of the International Conference on Machine Learning (ICML), Jun 2021

   Abstract Bib PDF

   Effective lifelong learning across diverse tasks requires the transfer of diverse knowledge, yet transferring irrelevant knowledge may lead to interference and catastrophic forgetting. In deep networks, transferring the appropriate granularity of knowledge is as important as the transfer mechanism, and must be driven by the relationships among tasks. We first show that the lifelong learning performance of several current deep learning architectures can be significantly improved by transfer at the appropriate layers. We then develop an expectation-maximization (EM) method to automatically select the appropriate transfer configuration and optimize the task network weights. This EM-based selective transfer is highly effective, balancing transfer performance on all tasks with avoiding catastrophic forgetting, as demonstrated on three algorithms in several lifelong object classification scenarios.

   @inproceedings{Lee2021SharingLessIsMore,
     author = {Lee, Seungwon and Behpour, Sima and Eaton, Eric},
     year = {2021},
     title = {Sharing less is more: Lifelong learning in deep networks with selective layer transfer},
     booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
   }

2020

1. Lifelong policy gradient learning of factored policies for faster training without forgetting

   Jorge Mendez, Boyu Wang, and Eric Eaton

   In Advances in Neural Information Processing Systems, Jun 2020

   Award   Abstract Bib PDF

   Earlier version was awarded Best Paper at the ICML’20 Workshop on Lifelong Learning

   Policy gradient methods have shown success in learning control policies for highdimensional dynamical systems. Their biggest downside is the amount of exploration they require before yielding high-performing policies. In a lifelong learning setting, in which an agent is faced with multiple consecutive tasks over its lifetime, reusing information from previously seen tasks can substantially accelerate the learning of new tasks. We provide a novel method for lifelong policy gradient learning that trains lifelong function approximators directly via policy gradients, allowing the agent to benefit from accumulated knowledge throughout the entire training process. We show empirically that our algorithm learns faster and converges to better policies than single-task and lifelong learning baselines, and completely avoids catastrophic forgetting on a variety of challenging domains.

   @inproceedings{Mendez2020LifelongPG,
     title = {Lifelong policy gradient learning of factored policies for faster training without forgetting},
     author = {Mendez, Jorge and Wang, Boyu and Eaton, Eric},
     booktitle = {Advances in Neural Information Processing Systems},
     year = {2020},
   }

2. Lifelong learning of factored policies via policy gradients

   Jorge Mendez and Eric Eaton

   In 4th Lifelong Learning Workshop at ICML, Jun 2020

   Superseded by the NeurIPS-20 version.

   Award   Abstract Bib PDF

   Best Paper at the ICML’20 Workshop on Lifelong Learning

   Policy gradient methods have shown success in learning continuous control policies for highdimensional dynamical systems. A major downside of such methods is the amount of exploration they require before yielding high-performing policies. In a lifelong learning setting, in which an agent is faced with multiple consecutive tasks over its lifetime, reusing information from previously seen tasks can substantially accelerate the learning of new tasks. We provide a novel method for lifelong policy gradient learning that trains lifelong function approximators directly via policy gradients, allowing the agent to benefit from accumulated knowledge throughout the entire training process. We show empirically that our algorithm learns faster and converges to better policies than single-task and lifelong learning baselines, and completely avoids catastrophic forgetting on a variety of challenging domains.

   @inproceedings{Mendez2020Lifelong,
     title = {Lifelong learning of factored policies via policy gradients},
     author = {Mendez, Jorge and Eaton, Eric},
     booktitle = {4th Lifelong Learning Workshop at ICML},
     year = {2020},
   }

3. Sharing less is more: Lifelong learning in deep networks with selective layer transfer

   Seungwon Lee, Sima Behpour, and Eric Eaton

   In 4th Lifelong Learning Workshop at ICML, Jun 2020

   Superseded by the ICML-2021 version.

   Abstract Bib PDF

   Effective lifelong learning across diverse tasks requires diverse knowledge, yet transferring irrelevant knowledge may lead to interference and catastrophic forgetting. In deep networks, transferring the appropriate granularity of knowledge is as important as the transfer mechanism, and must be driven by the relationships among tasks. We first show that the lifelong learning performance of several current deep learning architectures can be significantly improved by transfer at the appropriate layers. We then develop an expectation-maximization (EM) method to automatically select the appropriate transfer configuration and optimize the task network weights. This EM-based selective transfer is highly effective, as demonstrated on three algorithms in several lifelong object classification scenarios.

   @inproceedings{Lee2020Sharing,
     title = {Sharing less is more: Lifelong learning in deep networks with selective layer transfer},
     author = {Lee, Seungwon and Behpour, Sima and Eaton, Eric},
     booktitle = {4th Lifelong Learning Workshop at ICML},
     year = {2020},
   }

4. A general framework for continual learning of compositional structures

   Jorge Mendez and Eric Eaton

   In Continual Learning Workshop at ICML, Jun 2020

   Superseded by the ICLR-21 paper Lifelong learning of compositional structures.

   Abstract Bib PDF Supplement

   A hallmark of human intelligence is the ability to construct self-contained chunks of knowledge and adequately reuse them in novel combinations for solving different yet structurally related problems. Learning such compositional structures has been a significant challenge for artificial systems, due to the combinatorial nature of the underlying search problem. To date, research into compositional learning has largely proceeded separately from work on lifelong or continual learning. We integrate these two lines of work to present a general-purpose framework for lifelong learning of compositional structures that can be used for solving a stream of related tasks. Our framework separates the learning process into two broad stages: learning how to best combine existing components in order to assimilate a novel problem, and learning how to adapt the set of existing components to accommodate the new problem. This separation explicitly handles the trade-off between the stability required to remember how to solve earlier tasks and the flexibility required to solve new tasks, as we show empirically in an extensive evaluation.

   @inproceedings{Mendez2020General,
     title = {A general framework for continual learning of compositional structures},
     author = {Mendez, Jorge and Eaton, Eric},
     booktitle = {Continual Learning Workshop at ICML},
     year = {2020},
     supplement = {papers/Mendez2020General-supplement.pdf}
   }

5. Using task descriptions in lifelong machine learning for improved performance and zero-shot transfer

   Mohammad Rostami, David Isele, and Eric Eaton

   Journal of Artificial Intelligence Research, Jun 2020

   Abstract Bib PDF Website  

   Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.

   @article{Rostami2020Using,
     title = {Using task descriptions in lifelong machine learning for improved performance and zero-shot transfer},
     author = {Rostami, Mohammad and Isele, David and Eaton, Eric},
     journal = {Journal of Artificial Intelligence Research},
     volume = {67},
     pages = {673--704},
     year = {2020},
   }

6. Expert-augmented machine learning

   Efstathios D. Gennatas, Jerome H. Friedman, Lyle H. Ungar, and 11 more authors

   Proceedings of the National Academy of Sciences, Jun 2020

   Abstract Bib PDF Website  

   Machine learning is increasingly used across fields to derive insights from data, which further our understanding of the world and help us anticipate the future. The performance of predictive modeling is dependent on the amount and quality of available data. In practice, we rely on human experts to perform certain tasks and on machine learning for others. However, the optimal learning strategy may involve combining the complementary strengths of humans and machines. We present expert-augmented machine learning, an automated way to automatically extract problem-specific human expert knowledge and integrate it with machine learning to build robust, dependable, and data-efficient predictive models.Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications.

   @article{Gennatas2020ExpertAugmented,
     author = {Gennatas, Efstathios D. and Friedman, Jerome H. and Ungar, Lyle H. and Pirracchio, Romain and Eaton, Eric and Reichmann, Lara G. and Interian, Yannet and Luna, Jose Marcio and Simone, Charles B. and Auerbach, Andrew and Delgado, Elier and van der Laan, Mark J. and Solberg, Timothy D. and Valdes, Gilmer},
     title = {Expert-augmented machine learning},
     volume = {117},
     number = {9},
     pages = {4571--4577},
     year = {2020},
     publisher = {National Academy of Sciences},
     journal = {Proceedings of the National Academy of Sciences},
     supplement = {https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1906831117/-/DCSupplemental}
   }

2019

1. Building more accurate decision trees with the additive tree

   Jose Marcio Luna, Efstathios D. Gennatas, Lyle H. Ungar, and 7 more authors

   Proceedings of the National Academy of Sciences, Jun 2019

   Abstract Bib PDF Website  

   As machine learning applications expand to high-stakes areas such as criminal justice, finance, and medicine, legitimate concerns emerge about high-impact effects of individual mispredictions on people’s lives. As a result, there has been increasing interest in understanding general machine learning models to overcome possible serious risks. Current decision trees, such as Classification and Regression Trees (CART), have played a predominant role in fields such as medicine, due to their simplicity and intuitive interpretation. However, such trees suffer from intrinsic limitations in predictive power. We developed the additive tree, a theoretical approach to generate a more accurate and interpretable decision tree, which reveals connections between CART and gradient boosting. The additive tree exhibits superior predictive performance to CART, as validated on 83 classification tasks.The expansion of machine learning to high-stakes application domains such as medicine, finance, and criminal justice, where making informed decisions requires clear understanding of the model, has increased the interest in interpretable machine learning. The widely used Classification and Regression Trees (CART) have played a major role in health sciences, due to their simple and intuitive explanation of predictions. Ensemble methods like gradient boosting can improve the accuracy of decision trees, but at the expense of the interpretability of the generated model. Additive models, such as those produced by gradient boosting, and full interaction models, such as CART, have been investigated largely in isolation. We show that these models exist along a spectrum, revealing previously unseen connections between these approaches. This paper introduces a rigorous formalization for the additive tree, an empirically validated learning technique for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although the additive tree is designed primarily to provide both the model interpretability and predictive performance needed for high-stakes applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches.

   @article{Luna2019Building,
     author = {Luna, Jose Marcio and Gennatas, Efstathios D. and Ungar, Lyle H. and Eaton, Eric and Diffenderfer, Eric S. and Jensen, Shane T. and Simone, Charles B. and Friedman, Jerome H. and Solberg, Timothy D. and Valdes, Gilmer},
     title = {Building more accurate decision trees with the additive tree},
     volume = {116},
     number = {40},
     pages = {19887--19893},
     year = {2019},
     publisher = {National Academy of Sciences},
     journal = {Proceedings of the National Academy of Sciences},
     supplement = {https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1816748116/-/DCSupplemental}
   }

2. Artificial intelligence for pediatric ophthalmology

   Julia E. Reid and Eric Eaton

   Current Opinion in Ophthalmology, Jun 2019

   Abstract Bib PDF Website  

   PURPOSE OF REVIEW Despite the impressive results of recent artificial intelligence applications to general ophthalmology, comparatively less progress has been made toward solving problems in pediatric ophthalmology using similar techniques. This article discusses the unique needs of pediatric patients and how artificial intelligence techniques can address these challenges, surveys recent applications to pediatric ophthalmology, and discusses future directions. RECENT FINDINGS The most significant advances involve the automated detection of retinopathy of prematurity, yielding results that rival experts. Machine learning has also been applied to the classification of pediatric cataracts, prediction of postoperative complications following cataract surgery, detection of strabismus and refractive error, prediction of future high myopia, and diagnosis of reading disability. In addition, machine learning techniques have been used for the study of visual development, vessel segmentation in pediatric fundus images, and ophthalmic image synthesis. SUMMARY Artificial intelligence applications could significantly benefit clinical care by optimizing disease detection and grading, broadening access to care, furthering scientific discovery, and improving clinical efficiency. These methods need to match or surpass physician performance in clinical trials before deployment with patients. Owing to the widespread use of closed-access data sets and software implementations, it is difficult to directly compare the performance of these approaches, and reproducibility is poor. Open-access data sets and software could alleviate these issues and encourage further applications to pediatric ophthalmology.

   @article{Reid2019Artificial,
     title = {Artificial intelligence for pediatric ophthalmology},
     author = {Reid, Julia E. and Eaton, Eric},
     journal = {Current Opinion in Ophthalmology},
     volume = {30},
     number = {5},
     pages = {337-346},
     year = {2019},
   }

3. Learning shared knowledge for deep lifelong learning using deconvolutional networks

   Seungwon Lee, James Stokes, and Eric Eaton

   In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Jul 2019

   Abstract Bib PDF Website  

   Current mechanisms for knowledge transfer in deep networks tend to either share the lower layers between tasks, or build upon representations trained on other tasks. However, existing work in non-deep multi-task and lifelong learning has shown success with using factorized representations of the model parameter space for transfer, permitting more flexible construction of task models. Inspired by this idea, we introduce a novel architecture for sharing latent factorized representations in convolutional neural networks (CNNs). The proposed approach, called a deconvolutional factorized CNN, uses a combination of deconvolutional factorization and tensor contraction to perform flexible transfer between tasks. Experiments on two computer vision data sets show that the DF-CNN achieves superior performance in challenging lifelong learning settings, resists catastrophic forgetting, and exhibits reverse transfer to improve previously learned tasks from subsequent experience without retraining.

   @inproceedings{Lee2019Learning,
     title = {Learning shared knowledge for deep lifelong learning using deconvolutional networks},
     author = {Lee, Seungwon and Stokes, James and Eaton, Eric},
     booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)},
     pages = {2837--2844},
     year = {2019},
     month = jul,
   }

4. A lightweight approach to academic research group management using online tools: Spend more time on research and less on management

   Eric Eaton

   In Proceedings of the Educational Advances in Artificial Intelligene (EAAI) Symposium, Jul 2019

   Abstract Bib PDF Website  

   After years of taking a trial-and-error approach to managing a moderate-size academic research group, I settled on using a set of online tools and protocols that seem effective, require relatively little effort to use and maintain, and are inexpensive. This paper discusses this approach to communication, project management, document and code management, and logistics. It is my hope that other researchers, especially new faculty and research scientists, might find this set of tools and protocols useful when determining how to manage their own research group. This paper is targeted toward research groups based in mathematics and engineering, although faculty in other disciplines may find inspiration in some of these ideas.

   @inproceedings{Eaton2019Lightweight,
     title = {A lightweight approach to academic research group management using online tools: Spend more time on research and less on management},
     author = {Eaton, Eric},
     booktitle = {Proceedings of the Educational Advances in Artificial Intelligene (EAAI) Symposium},
     pages = {9644--9647},
     year = {2019},
   }

5. Rapid multi-objective optimization with multi-year future weather condition and decision-making support for building retrofit

   Pengyuan Shen, William Braham, Yun Kyu Yi, and 1 more author

   Energy, Apr 2019

   Abstract Bib PDF Website  

   A method of fast multi-objective optimization and decision-making support for building retrofit planning is developed, and lifecycle cost analysis method taking into account of future climate condition is used in evaluating the retrofit performance. In order to resolve the optimization problem in a fast manner with recourse to non-dominate sorting differential evolution algorithm, the simplified hourly dynamic simulation modeling tool SimBldPy is used as the simulator for objective function evaluation. Moreover, the generated non-dominated solutions are treated and rendered by a layered scheme using agglomerative hierarchical clustering technique to make it more intuitive and sense making during the decision-making process as well as to be better presented. The suggested optimization method is implemented to the retrofit planning of a campus building in UPenn with various energy conservation measures (ECM) and costs, and more than one thousand Pareto fronts are obtained and being analyzed according to the proposed decision-making framework. Twenty ECM combinations are eventually selected from all generated Pareto fronts. It is manifested that the developed decision-making support scheme shows robustness in dealing with retrofit optimization problem and is able to provide support for brainstorming and enumerating various possibilities during the decision-making process.

   @article{Shen2019Rapid,
     title = {Rapid multi-objective optimization with multi-year future weather condition and decision-making support for building retrofit},
     keywords = {Building retrofit, Climate change, Heuristic method, Hierarchical clustering, Optimization, Pareto fronts},
     author = {Shen, Pengyuan and Braham, William and Yi, {Yun Kyu} and Eaton, Eric},
     year = {2019},
     month = apr,
     volume = {172},
     pages = {892--912},
     journal = {Energy},
     publisher = {Elsevier Limited},
   }

6. Transfer learning via minimizing the performance gap between domains

   Boyu Wang, Jorge Mendez, Mingbo Cai, and 1 more author

   In Advances in Neural Information Processing Systems, Apr 2019

   Abstract Bib PDF Supplement

   We propose a new principle for transfer learning, based on a straightforward intuition: if two domains are similar to each other, the model trained on one domain should also perform well on the other domain, and vice versa. To formalize this intuition, we define the performance gap as a measure of the discrepancy between the source and target domains. We derive generalization bounds for the instance weighting approach to transfer learning, showing that the performance gap can be viewed as an algorithm-dependent regularizer, which controls the model complexity. Our theoretical analysis provides new insight into transfer learning and motivates a set of general, principled rules for designing new instance weighting schemes for transfer learning. These rules lead to gapBoost, a novel and principled boosting approach for transfer learning. Our experimental evaluation on benchmark data sets shows that gapBoost significantly outperforms previous boosting-based transfer learning algorithms.

   @incollection{Wang2019Transfer,
     title = {Transfer learning via minimizing the performance gap between domains},
     author = {Wang, Boyu and Mendez, Jorge and Cai, Mingbo and Eaton, Eric},
     booktitle = {Advances in Neural Information Processing Systems},
     volume = {32},
     pages = {10645--10655},
     year = {2019},
     bib2html_dl_code = {https://github.com/bwang-ml/gapBoost},
     supplement = {papers/Wang2019Transfer-Supplement.pdf}
   }

7. SAR image classification using few-shot cross-domain transfer learning

   Mohammad Rostami, Soheil Kolouri, Eric Eaton, and 1 more author

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Jun 2019

   Abstract Bib PDF

   Data-driven classification algorithms based on deep convolutional neural networks (CNNs) have reached human-level performance for many tasks within Electro-Optical (EO) computer vision.Despite being the prevailing visual sensory data, EO imaging is not effective in applications such as environmental monitoring at extended periods, where data collection at occluded weather is necessary.Synthetic Aperture Radar (SAR) is an effective imaging tool to circumvent these limitations and collect visual sensory information continually. However, replicating the success of deep learning on SAR domains is not straightforward. This is mainly because training deep networks requires huge labeled datasets anddata labeling is a lot more challenging in SAR domains. We develop an algorithm to transfer knowledge from EO domains to SAR domains to eliminate the need for huge labeled data points in the SAR domains. Our idea is to learn a shared domain-invariant embedding for cross-domain knowledge transfer such that the embedding is discriminative for two related EO and SAR tasks, while the latent data distributions for both domains remain similar. As a result, a classifier learned using mostly EO data can generalize well on the related task for the EO domain.

   @inproceedings{Rostami2019SAR,
     author = {Rostami, Mohammad and Kolouri, Soheil and Eaton, Eric and Kim, Kyungnam},
     title = {{SAR} image classification using few-shot cross-domain transfer learning},
     booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
     month = jun,
     year = {2019},
   }

8. Deep transfer learning for few-shot SAR image classification

   Mohammad Rostami, Soheil Kolouri, Eric Eaton, and 1 more author

   Remote Sensing, Jun 2019

   Abstract Bib PDF Website  

   The emergence of Deep Neural Networks (DNNs) has lead to high-performance supervised learning algorithms for the Electro-Optical (EO) domain classification and detection problems. This success is because generating huge labeled datasets has become possible using modern crowdsourcing labeling platforms such as Amazon’s Mechanical Turk that recruit ordinary people to label data. Unlike the EO domain, labeling the Synthetic Aperture Radar (SAR) domain data can be much more challenging, and for various reasons, using crowdsourcing platforms is not feasible for labeling the SAR domain data. As a result, training deep networks using supervised learning is more challenging in the SAR domain. In the paper, we present a new framework to train a deep neural network for classifying Synthetic Aperture Radar (SAR) images by eliminating the need for a huge labeled dataset. Our idea is based on transferring knowledge from a related EO domain problem, where labeled data are easy to obtain. We transfer knowledge from the EO domain through learning a shared invariant cross-domain embedding space that is also discriminative for classification. To this end, we train two deep encoders that are coupled through their last year to map data points from the EO and the SAR domains to the shared embedding space such that the distance between the distributions of the two domains is minimized in the latent embedding space. We use the Sliced Wasserstein Distance (SWD) to measure and minimize the distance between these two distributions and use a limited number of SAR label data points to match the distributions class-conditionally. As a result of this training procedure, a classifier trained from the embedding space to the label space using mostly the EO data would generalize well on the SAR domain. We provide a theoretical analysis to demonstrate why our approach is effective and validate our algorithm on the problem of ship classification in the SAR domain by comparing against several other competing learning approaches.

   @article{Rostami2019Deep,
     title = {Deep transfer learning for few-shot {SAR} image classification},
     author = {Rostami, Mohammad and Kolouri, Soheil and Eaton, Eric and Kim, Kyungnam},
     journal = {Remote Sensing},
     volume = {11},
     pages = {1374},
     year = {2019},
   }

2018

1. Lifelong inverse reinforcement learning

   Jorge A. Mendez, Shashank Shivkumar, and Eric Eaton

   Neural Information Processing Systems, Jun 2018

   Abstract Bib PDF

   Methods for learning from demonstration (LfD) have shown success in acquiring behavior policies by imitating a user. However, even for a single task, LfD may require numerous demonstrations. For versatile agents that must learn many tasks via demonstration, this process would substantially burden the user if each task were learned in isolation. To address this challenge, we introduce the novel problem of lifelong learning from demonstration, which allows the agent to continually build upon knowledge learned from previously demonstrated tasks to accelerate the learning of new tasks, reducing the amount of demonstrations required. As one solution to this problem, we propose the first lifelong learning approach to inverse reinforcement learning, which learns consecutive tasks via demonstration, continually transferring knowledge between tasks to improve performance.

   @article{Mendez2018Lifelong,
     title = {Lifelong inverse reinforcement learning},
     author = {Mendez, Jorge A. and Shivkumar, Shashank and Eaton, Eric},
     journal = {Neural Information Processing Systems},
     year = {2018},
   }

2. Multi-agent distributed lifelong learning for collective knowledge acquisition

   Mohammad Rostami, Soheil Kolouri, Kyungnam Kim, and 1 more author

   In Proceedings of the Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-18), Jun 2018

   Abstract Bib PDF

   Lifelong machine learning methods acquire knowledge over a series of consecutive tasks, continually building upon their experience. Current lifelong learning algorithms rely upon a single learning agent that has centralized access to all data. In this paper, we extend the idea of lifelong learning from a single agent to a network of multiple agents that collectively learn a series of tasks. Each agent faces some (potentially unique) set of tasks; the key idea is that knowledge learned from these tasks may benefit other agents trying to learn different (but related) tasks. Our Collective Lifelong Learning Algorithm (CoLLA) provides an efficient way for a network of agents to share their learned knowledge in a distributed and decentralized manner, while eliminating the need to share locally observed data. We provide theoretical guarantees for robust performance of the algorithm and empirically demonstrate that CoLLA outperforms existing approaches for distributed multi-task learning on a variety of datasets.

   @inproceedings{Rostami2018MultiAgent,
     title = {Multi-agent distributed lifelong learning for collective knowledge acquisition},
     author = {Rostami, Mohammad and Kolouri, Soheil and Kim, Kyungnam and Eaton, Eric},
     booktitle = {Proceedings of the Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-18)},
     year = {2018},
   }

3. Modeling consecutive task learning with task graph agendas

   David Isele, Eric Eaton, Mark Roberts, and 1 more author

   In Proceedings of the Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-18), Jun 2018

   Abstract Bib PDF

   @inproceedings{Isele2018Modeling,
     title = {Modeling consecutive task learning with task graph agendas},
     author = {Isele, David and Eaton, Eric and Roberts, Mark and Aha, David},
     booktitle = {Proceedings of the Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-18)},
     year = {2018},
   }

2017

1. Lifelong machine learning with Gaussian processes

   Christopher Clingerman and Eric Eaton

   In Proceedings of the European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD-17), Jun 2017

   Abstract Bib PDF

   Recent developments in lifelong machine learning have demonstrated that it is possible to learn multiple tasks consecutively, transferring knowledge between those tasks to accelerate learning and improve performance. However, these methods are limited to using linear parametric base learners, substantially restricting the predictive power of the resulting models. We present a lifelong learning algorithm that can support nonparametric models, focusing on Gaussian processes. To enable efficient online transfer between Gaussian process models, our approach assumes a factorized formulation of the covariance functions, and incrementally learns a shared sparse basis for the models’ parameterizations. We show that this lifelong learning approach is highly computationally efficient, and outperforms existing methods on a variety of data sets.

   @inproceedings{Clingerman2017Lifelong,
     title = {Lifelong machine learning with Gaussian processes},
     author = {Clingerman, Christopher and Eaton, Eric},
     booktitle = {Proceedings of the European Conference on Machine Learning \& Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD-17)},
     year = {2017},
   }

2. Estimating 3D trajectories from 2D projections via disjunctive factored four-way conditional restricted Boltzmann machines

   Decebal Constantin Mocanu, Haitham Bou Ammar, Luis Puig, and 2 more authors

   Pattern Recognition, Sep 2017

   Abstract Bib Website  

   Estimation, recognition, and near-future prediction of 3D trajectories based on their two dimensional projections available from one camera source is an exceptionally difficult problem due to uncertainty in the trajectories and environment, high dimensionality of the specific trajectory states, lack of enough labeled data and so on. In this article, we propose a solution to solve this problem based on a novel deep learning model dubbed disjunctive factored four-way conditional restricted Boltzmann machine (DFFW-CRBM). Our method improves state-of-the-art deep learning techniques for high dimensional time-series modeling by introducing a novel tensor factorization capable of driving forth order Boltzmann machines to considerably lower energy levels, at no computational costs. DFFW-CRBMs are capable of accurately estimating, recognizing, and performing near-future prediction of three-dimensional trajectories from their 2D projections while requiring limited amount of labeled data. We evaluate our method on both simulated and real-world data, showing its effectiveness in predicting and classifying complex ball trajectories and human activities.

   @article{Mocanu2017Estimating,
     title = {Estimating 3D trajectories from 2D projections via disjunctive factored four-way conditional restricted Boltzmann machines},
     author = {Mocanu, Decebal Constantin and Ammar, Haitham Bou and Puig, Luis and Eaton, Eric and Liotta, Antonio},
     journal = {Pattern Recognition},
     volume = {69},
     pages = {325--335},
     month = sep,
     year = {2017},
   }

3. Teaching integrated AI through interdisciplinary project-driven courses

   Eric Eaton

   AI Magazine, Sep 2017

   Abstract Bib PDF Website  

   Different subfields of AI (such as vision, learning, reasoning, planning, and others) are often studied in isolation, both in individual courses and in the research literature. This promulgates the idea that these different AI capabilities can easily be integrated later, whereas, in practice, developing integrated AI systems remains an open challenge for both research and industry. Interdisciplinary project-driven courses can fill this gap in AI education, providing challenging problems that require the integration of multiple AI methods. This article explores teaching integrated AI through two project-driven courses: a capstone-style graduate course in advanced robotics, and an undergraduate course on computational sustainability and assistive computing. In addition to studying the integration of AI techniques, these courses provide students with practical applications experience and exposure to social issues of AI and computing. My hope is that other instructors find these courses as useful examples for constructing their own project-driven courses to teach integrated AI.

   @article{Eaton2017Teaching,
     title = {Teaching integrated {AI} through interdisciplinary project-driven courses},
     author = {Eaton, Eric},
     journal = {AI Magazine},
     volume = {38},
     number = {2},
     pages = {13--21},
     year = {2017},
   }

2016

1. MediBoost: a Patient Stratification Tool for Interpretable Decision Making in the Era of Precision Medicine

   Gilmer Valdes, Jose Marcio Luna, Eric Eaton, and 3 more authors

   Scientific Reports, Nov 2016

   Abstract Bib PDF Website  

   Machine learning algorithms that are both interpretable and accurate are essential in applications such as medicine where errors can have a dire consequence. Unfortunately, there is currently a tradeoff between accuracy and interpretability among state-of-the-art methods. Decision trees are interpretable and are therefore used extensively throughout medicine for stratifying patients. Current decision tree algorithms, however, are consistently outperformed in accuracy by other, less-interpretable machine learning models, such as ensemble methods. We present MediBoost, a novel framework for constructing decision trees that retain interpretability while having accuracy similar to ensemble methods, and compare MediBoost’s performance to that of conventional decision trees and ensemble methods on 13 medical classification problems. MediBoost significantly outperformed current decision tree algorithms in 11 out of 13 problems, giving accuracy comparable to ensemble methods. The resulting trees are of the same type as decision trees used throughout clinical practice but have the advantage of improved accuracy. Our algorithm thus gives the best of both worlds: it grows a single, highly interpretable tree that has the high accuracy of ensemble methods.

   @article{Valdes2016MediBoost,
     title = {MediBoost: a Patient Stratification Tool for Interpretable Decision Making in the Era of Precision Medicine},
     author = {Valdes, Gilmer and Luna, Jose Marcio and Eaton, Eric and {Simone II}, Charles B. and Ungar, Lyle H. and Solberg, Timothy D.},
     journal = {Scientific Reports},
     volume = {6},
     pages = {37854},
     year = {2016},
     month = nov,
     bib2html_dl_code = {http://www.mediboostml.com},
     supplement = {http://www.nature.com/article-assets/npg/srep/2016/161130/srep37854/extref/srep37854-s1.pdf}
   }

2. Lifelong Learning for Disturbance Rejection on Mobile Robots

   David Isele, Jose Marcio Luna, Eric Eaton, and 4 more authors

   In Proceedings of the International Conference on Intelligent Robots and Systems (IROS-16), Oct 2016

   Abstract Bib PDF

   No two robots are exactly the same—even for a given model of robot, different units will require slightly different controllers. Furthermore, because robots change and degrade over time, a controller will need to change over time to remain optimal. This paper leverages lifelong learning in order to learn controllers for different robots. In particular, we show that by learning a set of control policies over robots with different (unknown) motion models, we can quickly adapt to changes in the robot, or learn a controller for a new robot with a unique set of disturbances. Furthermore, the approach is completely model-free, allowing us to apply this method to robots that have not, or cannot, be fully modeled.

   @inproceedings{Isele2016Lifelong,
     author = {Isele, David and Luna, Jose Marcio and Eaton, Eric and {de la Cruz}, Gabriel V. and Irwin, James and Kallaher, Brandon and Taylor, Matthew E.},
     year = {2016},
     title = {Lifelong Learning for Disturbance Rejection on Mobile Robots},
     booktitle = {Proceedings of the International Conference on Intelligent Robots and Systems (IROS-16)},
     month = oct,
     publisher = {IEEE/RSJ},
     bib2html_dl_video = {https://youtu.be/u7pkhLx0FQ0}
   }

3. Using task features for zero-shot knowledge transfer in lifelong learning

   David Isele, Mohammad Rostami, and Eric Eaton

   In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-16), Jul 2016

   Award   Abstract Bib PDF Poster   Slides  

   IJCAI-16 Distinguished Student Paper (sole Best Student Paper)

   Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of the inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong reinforcement learning method based on coupled dictionary learning that incorporates high-level task descriptors to model the inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of dynamical control problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict the task policy through zero-shot learning using the coupled dictionary, eliminating the need to pause to gather training data before addressing the task.

   @inproceedings{Isele2016Using,
     author = {Isele, David and Rostami, Mohammad and Eaton, Eric},
     year = {2016},
     title = {Using task features for zero-shot knowledge transfer in lifelong learning},
     booktitle = {Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-16)},
     month = jul,
     bib2html_dl_slides = {papers/Isele2016Using-slides.pdf},
     bib2html_dl_poster = {papers/Isele2016Using-poster.pdf}
   }

4. Design of a low-cost platform for autonomous mobile service robots

   Eric Eaton, Caio Mucchiani, Mayumi Mohan, and 3 more authors

   In IJCAI-16 Workshop on Autonomous Mobile Service Robots, Jul 2016

   Abstract Bib PDF Poster   Slides  

   Most current autonomous mobile service robots are either expensive commercial platforms or custom manufactured for research environments, limiting their availability. We present the design for a low-cost service robot based on the widely used TurtleBot 2 platform, with the goal of making service robots affordable and accessible to the research, educational, and hobbyist communities. Our design uses a set of simple and inexpensive modifications to transform the TurtleBot 2 into a 4.5ft (1.37m) tall tour-guide or telepresence-style robot, capable of performing a wide variety of indoor service tasks. The resulting platform provides a shoulder-height touchscreen and 3D camera for interaction, an optional low-cost arm for manipulation, enhanced onboard computation, autonomous charging, and up to 6 hours of runtime. The resulting platform can support many of the tasks performed by significantly more expensive service robots. For compatibility with existing software packages, the service robot runs the Robot Operating System (ROS).

   @inproceedings{Eaton2016Design,
     author = {Eaton, Eric and Mucchiani, Caio and Mohan, Mayumi and Isele, David and Luna, Jose Marcio and Clingerman, Christopher},
     year = {2016},
     title = {Design of a low-cost platform for autonomous mobile service robots},
     booktitle = {IJCAI-16 Workshop on Autonomous Mobile Service Robots},
     month = jul,
     bib2html_dl_slides = {papers/Eaton2016Design-slides.pdf},
     bib2html_dl_poster = {papers/Eaton2016Design-poster.pdf}
   }

5. Work in Progress: Lifelong Learning for Disturbance Rejection on Mobile Robots

   David Isele, Jose Marcio Luna, Eric Eaton, and 4 more authors

   In Proceedings of the AAMAS’16 Workshop on Adaptive Learning Agents, May 2016

   Superseded by the IROS-16 paper Lifelong Learning for Disturbance Rejection on Mobile Robots.

   Abstract Bib PDF Slides  

   No two robots are exactly the same – even for a given model of robot, different units will require slightly different controllers. Furthermore, because robots change and degrade over time, a controller will need to change over time to remain optimal. This paper leverages lifelong learning in order to learn controllers for different robots. In particular, we show that by learning a set of control policies over robots with different (unknown) motion models, we can quickly adapt to changes in the robot, or learn a controller for a new robot with a unique set of disturbances. Further, the approach is completely model-free, allowing us to apply this method to robots that have not, or cannot, be fully modeled. These preliminary results are an initial step towards learning robust fault-tolerant control for arbitrary robots.

   @inproceedings{Isele2016Work,
     author = {Isele, David and Luna, Jose Marcio and Eaton, Eric and {de la Cruz}, Gabriel V. and Irwin, James and Kallaher, Brandon and Taylor, Matthew E.},
     year = {2016},
     title = {Work in Progress: Lifelong Learning for Disturbance Rejection on Mobile Robots},
     booktitle = {Proceedings of the AAMAS'16 Workshop on Adaptive Learning Agents},
     month = may,
     bib2html_dl_slides = {papers/Isele2016Work-slides.pdf}
   }

2015

1. Safe policy search for lifelong reinforcement learning with sublinear regret

   Haitham Bou Ammar, Rasul Tutunov, and Eric Eaton

   In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), Jul 2015

   Abstract Bib PDF Poster   Slides  

   Lifelong reinforcement learning provides a promising framework for developing versatile agents that can accumulate knowledge over a lifetime of experience and rapidly learn new tasks by building upon prior knowledge. However, current lifelong learning methods exhibit non-vanishing regret as the amount of experience increases, and include limitations that can lead to suboptimal or unsafe control policies. To address these issues, we develop a lifelong policy gradient learner that operates in an adversarial setting to learn multiple tasks online while enforcing safety constraints on the learned policies. We demonstrate, for the first time, sublinear regret for lifelong policy search, and validate our algorithm on several benchmark dynamical systems and an application to quadrotor control.

   @inproceedings{BouAmmar2015Safe,
     author = {{Bou Ammar}, Haitham and Tutunov, Rasul and Eaton, Eric},
     year = {2015},
     title = {Safe policy search for lifelong reinforcement learning with sublinear regret},
     booktitle = {Proceedings of the 32nd International Conference on Machine Learning (ICML-15)},
     month = jul,
     bib2html_dl_slides = {papers/slides-BouAmmar2015Safe.pdf},
     bib2html_dl_poster = {papers/poster-BouAmmar2015Safe.pdf}
   }

2. Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning

   Haitham Bou Ammar, Eric Eaton, Jose Marcio Luna, and 1 more author

   In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-15), Jul 2015

   Award   Abstract Bib PDF

   Finalist for IJCAI-15 Distinguished Paper

   Online multi-task learning is an important capability for lifelong learning agents, enabling them to acquire models for diverse tasks over time and rapidly learn new tasks by building upon prior experience. However, recent progress toward lifelong reinforcement learning (RL) has been limited to learning from within a single task domain. For truly versatile lifelong learning, the agent must be able to autonomously transfer knowledge between different task domains. A few methods for cross-domain transfer have been developed, but these methods are computationally inefficient for scenarios where the agent must learn tasks consecutively. In this paper, we develop the first cross-domain lifelong RL framework. Our approach efficiently optimizes a shared repository of transferable knowledge and learns projection matrices that specialize that knowledge to different task domains. We provide rigorous theoretical guarantees on the stability of this approach, and empirically evaluate its performance on diverse dynamical systems. Our results show that the proposed method can learn effectively from interleaved task domains and rapidly acquire high performance in new domains.

   @inproceedings{BouAmmar2015Autonomous,
     author = {{Bou Ammar}, Haitham and Eaton, Eric and Luna, Jose Marcio and Ruvolo, Paul},
     year = {2015},
     title = {Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning},
     booktitle = {Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-15)},
     month = jul,
   }

3. Unsupervised cross-domain transfer in policy gradient reinforcement learning via manifold alignment

   Haitham Bou Ammar, Eric Eaton, Paul Ruvolo, and 1 more author

   In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15), Jan 2015

   Abstract Bib PDF Poster   Slides  

   The success of applying policy gradient reinforcement learning (RL) to difficult control tasks hinges crucially on the ability to determine a sensible initialization for the policy. Transfer learning methods tackle this problem by reusing knowledge gleaned from solving other related tasks. In the case of multiple task domains, these algorithms require an inter-task mapping to facilitate knowledge transfer across domains. However, there are currently no general methods to learn an inter-task mapping without requiring either background knowledge that is not typically present in RL settings, or an expensive analysis of an exponential number of inter-task mappings in the size of the state and action spaces. This paper introduces an autonomous framework that uses unsupervised manifold alignment to learn intertask mappings and effectively transfer samples between different task domains. Empirical results on diverse dynamical systems, including an application to quadrotor control, demonstrate its effectiveness for cross-domain transfer in the context of policy gradient RL.

   @inproceedings{BouAmmar2015Unsupervised,
     author = {{Bou Ammar}, Haitham and Eaton, Eric and Ruvolo, Paul and Taylor, Matthew E.},
     title = {Unsupervised cross-domain transfer in policy gradient reinforcement learning via manifold alignment},
     booktitle = {Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15)},
     month = jan,
     year = {2015},
     bib2html_dl_slides = {papers/slides-BouAmmar2015Unsupervised.pdf},
     bib2html_dl_poster = {papers/poster-BouAmmar2015Unsupervised.pdf}
   }

2014

1. On the degree distribution of Pólya urn graph processes

   Rasul Tutunov, Haitham Bou Ammar, Ali Jadbabaie, and 1 more author

   arXiv:1410.8515 Preprint, Oct 2014

   Abstract Bib PDF

   This paper presents a tighter bound on the degree distribution of arbitrary Polya urn graph processes, proving that the proportion of vertices with degree d obeys a power-law distribution P(d) proportional to d^(-gamma) for d <= n^(1/6 - epsilon) for any epsilon > 0, where n represents the number of vertices in the network. Previous work by Bollobas et al. formalized the well-known preferential attachment model of Barabasi and Albert, and showed that the power-law distribution held for d <= n^(1/15) with gamma = 3. Our revised bound represents a significant improvement over existing models of degree distribution in scale-free networks, where its tightness is restricted by the Azuma-Hoeffding concentration inequality for martingales. We achieve this tighter bound through a careful analysis of the first set of vertices in the network generation process, and show that the newly acquired is at the edge of exhausting Bollobas model in the sense that the degree expectation breaks down for other powers.

   @article{Tutunov2014DegreeDistribution,
     author = {Tutunov, Rasul and {Bou Ammar}, Haitham and Jadbabaie, Ali and Eaton, Eric},
     year = {2014},
     title = {On the degree distribution of P\'{o}lya urn graph processes},
     journal = {arXiv:1410.8515 Preprint},
     month = oct,
   }

2. An automated measure of MDP similarity for transfer in reinforcement learning

   Haitham Bou Ammar, Eric Eaton, Matthew E. Taylor, and 4 more authors

   In Proceedings of the AAAI’14 Workshop on Machine Learning for Interactive Systems, Jul 2014

   Abstract Bib PDF Poster   Slides  

   Transfer learning can improve the reinforcement learning of a new task by allowing the agent to reuse knowledge acquired from other source tasks. Despite their success, transfer learning methods rely on having relevant source tasks; transfer from inappropriate tasks can inhibit performance on the new task. For fully autonomous transfer, it is critical to have a method for automatically choosing relevant source tasks, which requires a similarity measure between Markov Decision Processes (MDPs). This issue has received little attention, and is therefore still a largely open problem. This paper presents a data-driven automated similarity measure for MDPs. This novel measure is a significant step toward autonomous reinforcement learning transfer, allowing agents to: (1) characterize when transfer will be useful and, (2) automatically select tasks to use for transfer. The proposed measure is based on the reconstruction error of a restricted Boltzmann machine that attempts to model the behavioral dynamics of the two MDPs being compared. Empirical results illustrate that this measure is correlated with the performance of transfer and therefore can be used to identify similar source tasks for transfer learning.

   @inproceedings{BouAmmar2014Automated,
     author = {Ammar, Haitham Bou and Eaton, Eric and Taylor, Matthew E. and Mocanu, Decebal and Driessens, Kurt and Weiss, Gerhard and Tuyls, Karl},
     year = {2014},
     title = {An automated measure of {MDP} similarity for transfer in reinforcement learning},
     booktitle = {Proceedings of the AAAI'14 Workshop on Machine Learning for Interactive Systems},
     month = jul,
     bib2html_dl_slides = {papers/slides-BouAmmar2014Automated.pdf},
     bib2html_dl_poster = {papers/poster-BouAmmar2014Automated.pdf}
   }

3. Online Multi-Task Learning for Policy Gradient Methods

   Haitham Bou Ammar, Eric Eaton, Paul Ruvolo, and 1 more author

   In Proceedings of the 31st International Conference on Machine Learning (ICML-14), Jun 2014

   Abstract Bib PDF

   Policy gradient algorithms have shown considerable recent success in solving high-dimensional sequential decision-making (SDM) tasks, particularly in robotics. However, these methods often require extensive experience in a domain to achieve high performance. To make agents more sample-efficient, we developed a multi-task policy gradient method to learn SDM tasks consecutively, transferring knowledge between tasks to accelerate learning. Our approach provides robust theoretical guarantees and we show empirically that it dramatically accelerates learning on a variety of dynamical systems, including an application to quadcopter control.

   @inproceedings{BouAmmar2014Online,
     author = {Ammar, Haitham Bou and Eaton, Eric and Ruvolo, Paul and Taylor, Matthew E.},
     title = {Online Multi-Task Learning for Policy Gradient Methods},
     booktitle = {Proceedings of the 31st International Conference on Machine Learning (ICML-14)},
     year = {2014},
     month = jun,
   }

4. Online Multi-Task Learning via Sparse Dictionary Optimization

   Paul Ruvolo and Eric Eaton

   In Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI-14), Jul 2014

   Abstract Bib PDF Poster   Slides  

   This paper develops an efficient online algorithm for learning multiple consecutive tasks based on the K-SVD algorithm for sparse dictionary optimization. We first derive a batch multi-task learning method that builds upon K-SVD, and then extend the batch algorithm to train models online in a lifelong learning setting. The resulting method has lower computational complexity than other current lifelong learning algorithms while maintaining nearly identical model performance. Additionally, the proposed method offers an alternate formulation for lifelong learning that supports both task and feature similarity matrices.

   @inproceedings{Ruvolo2014Online,
     author = {Ruvolo, Paul and Eaton, Eric},
     title = {Online Multi-Task Learning via Sparse Dictionary Optimization},
     booktitle = {Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI-14)},
     year = {2014},
     month = jul,
     bib2html_dl_slides = {papers/slides-Ruvolo2014Online.pdf},
     bib2html_dl_poster = {papers/poster-Ruvolo2014Online.pdf}
   }

5. Online Multi-Task Gradient Temporal-Difference Learning

   Vishnu Purushothaman Sreenivasan, Haitham Bou Ammar, and Eric Eaton

   In Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI-14), Jul 2014

   [Student Abstract]

   Abstract Bib PDF

   We develop an online multi-task formulation of model-based gradient temporal-difference (GTD) reinforcement learning. Our approach enables an autonomous RL agent to accumulate knowledge over its lifetime and efficiently share this knowledge between tasks to accelerate learning. Rather than learning a policy for an RL task tabula rasa, as in standard GTD, our approach rapidly learns a high performance policy by building upon the agent’s previously learned knowledge. Our preliminary results on controlling different mountain car tasks demonstrates that GTD-ELLA significantly improves learning over standard GTD(0).

   @inproceedings{VishnuPS2014Online,
     author = {Sreenivasan, Vishnu Purushothaman and Ammar, Haitham Bou and Eaton, Eric},
     title = {Online Multi-Task Gradient Temporal-Difference Learning},
     note = {[Student Abstract]},
     booktitle = {Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI-14)},
     year = {2014},
     month = jul,
   }

6. Special Issue of AI Magazine on Computational Sustainability

   Jun 2014

   Bib Website  

   @book{Eaton2014AIMagazine,
     editor = {Eaton, Eric and Gomes, Carla and Williams, Brian},
     title = {Special Issue of AI Magazine on Computational Sustainability},
     series = {AI Magazine},
     volume = {35},
     number = {2-3},
     month = jun,
     year = {2014},
     publisher = {AAAI Press},
     url = {http://www.aaai.org/ojs/index.php/aimagazine/issue/view/206}
   }

7. Computational Sustainability

   Eric Eaton, Carla Gomes, and Brian Williams

   AI Magazine, Jun 2014

   Abstract Bib PDF

   Computational sustainability problems, which exist in dynamic environments with high amounts of uncertainty, provide a variety of unique challenges to artificial intelligence research and the opportunity for significant impact upon our collective future. This editorial provides an overview of artificial intelligence for computational sustainability, and introduces this special issue of AI Magazine.

   @article{Eaton2014CompSustainability,
     author = {Eaton, Eric and Gomes, Carla and Williams, Brian},
     title = {Computational Sustainability},
     journal = {AI Magazine},
     volume = {35},
     number = {2},
     pages = {3--7},
     year = {2014},
     month = jun,
   }

8. Multi-view constrained clustering with an incomplete mapping between views

   Eric Eaton, Marie desJardins, and Sara Jacob

   Knowledge and Information Systems, Jan 2014

   Abstract Bib PDF

   Multi-view learning algorithms typically assume a complete bipartite mapping between the different views in order to exchange information during the learning process. However, many applications provide only a partial mapping between the views, creating a challenge for current methods. To address this problem, we propose a multi-view algorithm based on constrained clustering that can operate with an incomplete mapping. Given a set of pairwise constraints in each view, our approach propagates these constraints using a local similarity measure to those instances that can be mapped to the other views, allowing the propagated constraints to be transferred across views via the partial mapping. It uses co-EM to iteratively estimate the propagation within each view based on the current clustering model, transfer the constraints across views, and then update the clustering model. By alternating the learning process between views, this approach produces a unified clustering model that is consistent with all views. We show that this approach significantly improves clustering performance over several other methods for transferring constraints and allows multi-view clustering to be reliably applied when given a limited mapping between the views. Our evaluation reveals that the propagated constraints have high precision with respect to the true clusters in the data, explaining their benefit to clustering performance in both single- and multi-view learning scenarios.

   @article{Eaton2012MultiView,
     author = {Eaton, Eric and desJardins, Marie and Jacob, Sara},
     title = {Multi-view constrained clustering with an incomplete mapping between views},
     journal = {Knowledge and Information Systems},
     volume = {38},
     number = {1},
     pages = {231--257},
     month = jan,
     year = {2014},
     notes = {Published online November 2012, <a href="http://link.springer.com/content/pdf/10.1007%2Fs10115-012-0577-7">doi 10.1007/s10115-012-0577-7</a>.}
   }

2013

1. Active Task Selection for Lifelong Machine Learning

   Paul Ruvolo and Eric Eaton

   In Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI-13), Jul 2013

   Abstract Bib PDF Poster   Slides  

   In a lifelong learning framework, an agent acquires knowledge incrementally over consecutive learning tasks, continually building upon its experience. Recent lifelong learning algorithms have achieved nearly identical performance to batch multi-task learning methods while reducing learning time by three orders of magnitude. In this paper, we further improve the scalability of lifelong learning by developing curriculum selection methods that enable an agent to actively select the next task to learn in order to maximize performance on future learning tasks. We demonstrate that active task selection is highly reliable and effective, allowing an agent to learn high performance models using up to 50% fewer tasks than when the agent has no control over the task order. We also explore a variant of transfer learning in the lifelong learning setting in which the agent can focus knowledge acquisition toward a particular target task.

   @inproceedings{Ruvolo2013Active,
     author = {Ruvolo, Paul and Eaton, Eric},
     title = {Active Task Selection for Lifelong Machine Learning},
     booktitle = {Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI-13)},
     year = {2013},
     month = jul,
     bib2html_dl_slides = {papers/slides-Ruvolo2013Active.pdf},
     bib2html_dl_poster = {papers/poster-Ruvolo2013Active.pdf},
     bib2html_dl_code = {software/ELLAv1.0.zip},
   }

2. ELLA: An Efficient Lifelong Learning Algorithm

   Paul Ruvolo and Eric Eaton

   In Proceedings of the 30th International Conference on Machine Learning (ICML-13), Jun 2013

   Abstract Bib PDF Poster   Slides  

   The problem of learning multiple consecutive tasks, known as \emphlifelong learning, is of great importance to the creation of intelligent, general-purpose, and flexible machines. In this paper, we develop a method for online multi-task learning in the lifelong learning setting. The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks. We show that ELLA has strong connections to both online dictionary learning for sparse coding and state-of-the-art batch multi-task learning methods, and provide robust theoretical performance guarantees. We show empirically that ELLA yields nearly identical performance to batch multi-task learning while learning tasks sequentially in three orders of magnitude (over 1,000x) less time.

   @inproceedings{Ruvolo2013ELLA,
     author = {Ruvolo, Paul and Eaton, Eric},
     title = {ELLA: An Efficient Lifelong Learning Algorithm},
     booktitle = {Proceedings of the 30th International Conference on Machine Learning (ICML-13)},
     year = {2013},
     month = jun,
     bib2html_dl_slides = {papers/slides-Ruvolo2013ELLA.pdf},
     bib2html_dl_poster = {papers/poster-Ruvolo2013ELLA.pdf},
     bib2html_dl_code = {software/ELLAv1.0.zip},
   }

3. Online Multi-Task Learning based on K-SVD

   Paul Ruvolo and Eric Eaton

   In Proceedings of the ICML 2013 Workshop on Theoretically Grounded Transfer Learning, Jun 2013

   Superseded by the AAAI-14 paper Online Multi-Task Learning via Sparse Dictionary Optimization.

   Abstract Bib PDF

   This paper develops an efficient online algorithm based on K-SVD for learning multiple consecutive tasks. We first derive a batch multi-task learning method that builds upon the K-SVD algorithm, and then extend the batch algorithm to train models online in a lifelong learning setting. The resulting method has lower computational complexity than other current lifelong learning algorithms while maintaining nearly identical performance. Additionally, the proposed method offers an alternate formulation for lifelong learning that supports both task and feature similarity matrices.

   @inproceedings{Ruvolo2013Online,
     author = {Ruvolo, Paul and Eaton, Eric},
     title = {Online Multi-Task Learning based on K-SVD},
     booktitle = {Proceedings of the ICML 2013 Workshop on Theoretically Grounded Transfer Learning},
     year = {2013},
     month = jun,
   }

4. Lifelong Machine Learning: Proceedings of the 2013 AAAI Spring Symposium

   Eric Eaton (chair)

   May 2013

   Bib Website  

   @book{Eaton2013AAAISSS,
     author = {(chair), Eric Eaton},
     title = {Lifelong Machine Learning: Proceedings of the 2013 AAAI Spring Symposium},
     series = {AAAI Technical Report SS-13-05},
     month = may,
     year = {2013},
     publisher = {AAAI Press},
     isbn = {ISBN 978-1-57735-602-8},
     url = {http://www.aaai.org/Press/Reports/Symposia/Spring/ss-13-05.php}
   }

5. Scalable Lifelong Learning with Active Task Selection

   Paul Ruvolo and Eric Eaton

   In Proceedings of the AAAI 2013 Spring Symposium on Lifelong Machine Learning, Stanford, CA, Mar 2013

   Superseded by the AAAI-13 paper Active Task Selection for Lifelong Machine Learning.

   Abstract Bib PDF

   The recently developed Efficient Lifelong Learning Algorithm (ELLA) acquires knowledge incrementally over a sequence of tasks, learning a repository of latent model components that are sparsely shared between models. ELLA shows strong performance in comparison to other multi-task learning algorithms, achieving nearly identical performance to batch multi-task learning methods while learning tasks sequentially in three orders of magnitude (over 1,000x) less time. In this paper, we evaluate several curriculum selection methods that allow ELLA to actively select the next task for learning in order to maximize performance on future learning tasks. Through experiments with three real and one synthetic data set, we demonstrate that active curriculum selection allows an agent to learn up to 50% more efficiently than when the agent has no control over the task order.

   @inproceedings{Ruvolo2013Scalable,
     author = {Ruvolo, Paul and Eaton, Eric},
     title = {Scalable Lifelong Learning with Active Task Selection},
     booktitle = {Proceedings of the AAAI 2013 Spring Symposium on Lifelong Machine Learning},
     year = {2013},
     location = {Stanford, CA},
     month = mar,
   }

2012

1. A spin-glass model for semi-supervised community detection

   Eric Eaton and Rachael Mansbach

   In Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada, Jul 2012

   Abstract Bib PDF Poster  

   Current modularity-based community detection methods show decreased performance as relational networks become increasingly noisy. These methods also yield a large number of diverse community structures as solutions, which is problematic for applications that impose constraints on the acceptable solutions or in cases where the user is focused on specific communities of interest. To address both of these problems, we develop a semi-supervised spin-glass model that enables current community detection methods to incorporate background knowledge in the forms of individual labels and pairwise constraints. Unlike current methods, our approach shows robust performance in the presence of noise in the relational network, and the ability to guide the discovery process toward specific community structures. We evaluate our algorithm on several benchmark networks and a new political sentiment network representing cooperative events between nations that was mined from news articles over six years.

   @inproceedings{Eaton2012SpinGlass,
     author = {Eaton, Eric and Mansbach, Rachael},
     title = {A spin-glass model for semi-supervised community detection},
     booktitle = {Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI-12)},
     month = jul,
     location = {Toronto, Canada},
     publisher = {AAAI Press},
     pages = {900--906},
     year = {2012},
     bib2html_dl_poster = {papers/poster-Eaton2012SpinGlass.pdf},
     bib2html_dl_code = {software/ICD-2011v0.5.zip}
   }

2. Incorporating computational sustainability into AI education through a freely-available, collectively-composed supplementary lab text

   Douglas Fisher, Bistra Dilkina, Eric Eaton, and 1 more author

   In Proceedings of the 3rd AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-12), Toronto, Canada, Jul 2012

   Abstract Bib PDF Poster  

   We introduce a laboratory text on environmental and societal sustainability applications that can be a supplemental resource for any undergraduate AI course. The lab text, entitled Artificial Intelligence for Computational Sustainability: A Lab Companion, is brand new and incomplete; freely available through Wikibooks; and open to community additions of projects, assignments, and explanatory material on AI for sustainability. The project adds to existing educational efforts of the computational sustainability community, encouraging the flow of knowledge from research to education and public outreach. Besides summarizing the laboratory book, this paper touches on its implications for integration of research and education, for communicating science to the public, and other broader impacts.

   @inproceedings{Fisher2012IncorporatingEAAI,
     author = {Fisher, Douglas and Dilkina, Bistra and Eaton, Eric and Gomes, Carla},
     title = {Incorporating computational sustainability into AI education through a freely-available, collectively-composed supplementary lab text},
     booktitle = {Proceedings of the 3rd AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-12)},
     month = jul,
     location = {Toronto, Canada},
     publisher = {AAAI Press},
     year = {2012},
     bib2html_dl_poster = {papers/poster-Fisher2012IncorporatingEAAI.pdf}
   }

3. Incorporating computational sustainability into AI education through a freely-available, collectively-composed supplementary lab text [Oral Presentation]

   Douglas Fisher, Bistra Dilkina, Eric Eaton, and 1 more author

   In Proceedings of the 3rd International Conference on Computational Sustainability (CompSust’12), Copenhagen, Denmark, Jul 2012

   Bib PDF

   @inproceedings{Fisher2012IncorporatingCompSust,
     author = {Fisher, Douglas and Dilkina, Bistra and Eaton, Eric and Gomes, Carla},
     title = {Incorporating computational sustainability into AI education through a freely-available, collectively-composed supplementary lab text [Oral Presentation]},
     booktitle = {Proceedings of the 3rd International Conference on Computational Sustainability (CompSust'12)},
     month = jul,
     location = {Copenhagen, Denmark},
     year = {2012},
   }

4. Inferring tasks for improved network structure discovery

   Diane Oyen, Eric Eaton, and Terran Lane

   In Working Notes of the Snowbird Learning Workshop, Snowbird, Utah, Apr 2012

   Bib PDF

   @inproceedings{Oyen2012Inferring,
     author = {Oyen, Diane and Eaton, Eric and Lane, Terran},
     title = {Inferring tasks for improved network structure discovery},
     booktitle = {Working Notes of the Snowbird Learning Workshop},
     month = apr,
     location = {Snowbird, Utah},
     year = {2012},
   }

2011

1. Selective Transfer Between Learning Tasks Using Task-Based Boosting

   Eric Eaton and Marie desJardins

   In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI-11), San Francisco, CA, Aug 2011

   Abstract Bib PDF Supplement

   The success of transfer learning on a target task is highly dependent on the selected source data. Instance transfer methods reuse data from the source tasks to augment the training data for the target task. If poorly chosen, this source data may inhibit learning, resulting in negative transfer. The current most widely used algorithm for instance transfer, TrAdaBoost, performs poorly when given irrelevant source data. We present a novel task-based boosting technique for instance transfer that selectively chooses the source knowledge to transfer to the target task. Our approach performs boosting at both the instance level and the task level, assigning higher weight to those source tasks that show positive transferability to the target task, and adjusting the weights of individual instances within each source task via AdaBoost. We show that this combination of task- and instance-level boosting significantly improves transfer performance over existing instance transfer algorithms when given a mix of relevant and irrelevant source data, especially for small amounts of data on the target task.

   @inproceedings{Eaton2011Selective,
     author = {Eaton, Eric and desJardins, Marie},
     title = {Selective Transfer Between Learning Tasks Using Task-Based Boosting},
     booktitle = {Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI-11)},
     month = aug,
     location = {San Francisco, CA},
     publisher = {AAAI Press},
     pages = {337--342},
     year = {2011},
     supplement = {papers/Eaton2011Selective-Supplement.pdf}
   }

2. The Importance of Selective Knowledge Transfer for Lifelong Learning

   Eric Eaton and Terran Lane

   In AAAI-11 Workshop on Lifelong Learning from Sensorimotor Data, San Francisco, CA, Aug 2011

   Abstract Bib PDF

   As knowledge transfer research progresses from single transfer to lifelong learning scenarios, it becomes increasingly important to properly select the source knowledge that would best transfer to the target task. In this position paper, we describe our previous work on selective knowledge transfer and relate it to problems in lifelong learning. We also briefly discuss our ongoing work to develop lifelong learning methods capable of continual transfer between tasks and the incorporation of guidance from an expert human user.

   @inproceedings{Eaton2011Importance,
     author = {Eaton, Eric and Lane, Terran},
     title = {The Importance of Selective Knowledge Transfer for Lifelong Learning},
     booktitle = {AAAI-11 Workshop on Lifelong Learning from Sensorimotor Data},
     month = aug,
     location = {San Francisco, CA},
     publisher = {AAAI Press},
     year = {2011},
   }

2010

1. Multi-View Clustering with Constraint Propagation for Learning with an Incomplete Mapping Between Views

   Eric Eaton, Marie desJardins, and Sara Jacob

   In Proceedings of the Conference on Information and Knowledge Management (CIKM’10), Toronto, Ontario, Canada, Oct 2010

   Abstract Bib PDF Slides  

   Multi-view learning algorithms typically assume a complete bipartite mapping between the different views in order to exchange information during the learning process. However, many applications provide only a partial mapping between the views, creating a challenge for current methods. To address this problem, we propose a multi-view algorithm based on constrained clustering that can operate with an incomplete mapping. Given a set of pairwise constraints in each view, our approach propagates these constraints using a local similarity measure to those instances that can be mapped to the other views, allowing the propagated constraints to be transferred across views via the partial mapping. It uses co-EM to iteratively estimate the propagation within each view based on the current clustering model, transfer the constraints across views, and update the clustering model, thereby learning a unified model for all views. We show that this approach significantly improves clustering performance over several other methods for transferring constraints and allows multi-view clustering to be reliably applied when given a limited mapping between the views.

   @inproceedings{Eaton2010MultiView,
     author = {Eaton, Eric and desJardins, Marie and Jacob, Sara},
     title = {Multi-View Clustering with Constraint Propagation for Learning with an Incomplete Mapping Between Views},
     booktitle = {Proceedings of the Conference on Information and Knowledge Management (CIKM'10)},
     month = oct,
     location = {Toronto, Ontario, Canada},
     publisher = {ACM Press},
     pages = {389--398},
     year = {2010},
     bib2html_dl_slides = {papers/Eaton2010MultiView-Slides.pdf}
   }

2. Interactive Learning using Manifold Geometry

   Eric Eaton, Gary Holness, and Daniel McFarlane

   In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, Jul 2010

   Abstract Bib PDF

   We present an interactive learning method that enables a user to iteratively refine a regression model. The user examines the output of the model, visualized as the vertical axis of a 2D scatterplot, and provides corrections by repositioning individual data instances to the correct output level. Each repositioned data instance acts as a control point for altering the learned model, using the geometry underlying the data. We capture the underlying structure of the data as a manifold, on which we compute a set of basis functions as the foundation for learning. Our results show that manifold-based interactive learning improves performance monotonically with each correction, outperforming alternative approaches.

   @inproceedings{Eaton2010Interactive,
     author = {Eaton, Eric and Holness, Gary and McFarlane, Daniel},
     title = {Interactive Learning using Manifold Geometry},
     booktitle = {Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI-10)},
     month = jul,
     location = {Atlanta, GA},
     publisher = {AAAI Press},
     pages = {437--443},
     year = {2010},
     bib2html_dl_slides = {papers/Eaton2010Interactive-Slides.ppt}
   }

3. Modeling and learning user preferences over sets

   Kiri Wagstaff, Marie desJardins, and Eric Eaton

   Journal of Experimental & Theoretical Artificial Intelligence, Sep 2010

   Abstract Bib PDF

   Although there has been significant research on modeling and learning user preferences for various types of objects, there has been relatively little work on the problem of representing and learning preferences over sets of objects. We introduce a representation language, DD-PREF, that balances preferences for particular objects with preferences about the properties of the set. Specifically, we focus on the depth of objects (i.e. preferences for specific attribute values over others) and on the diversity of sets (i.e. preferences for broad vs. narrow distributions of attribute values). The DD-PREF framework is general and can incorporate additional object- and set-based preferences. We describe a greedy algorithm, DD-Select, for selecting satisfying sets from a collection of new objects, given a preference in this language. We show how preferences represented in DD-PREF can be learned from training data. Experimental results are given for three domains: a blocks world domain with several different task-based preferences, a real-world music playlist collection, and rover image data gathered in desert training exercises.

   @article{Wagstaff2010Modeling,
     author = {Wagstaff, Kiri and desJardins, Marie and Eaton, Eric},
     year = {2010},
     title = {Modeling and learning user preferences over sets},
     journal = {Journal of Experimental & Theoretical Artificial Intelligence},
     volume = {22},
     number = {3},
     pages = {237--268},
     month = sep,
   }

2009

1. Set-Based Boosting for Instance-level Transfer

   Eric Eaton and Marie desJardins

   In Proceedings of the International Conference on Data Mining Workshop on Transfer Mining, Miami, FL, Dec 2009

   Superseded by the AAAI-11 paper Selective Transfer Between Learning Tasks Using Task-Based Boosting.

   Abstract Bib PDF

   The success of transfer to improve learning on a target task is highly dependent on the selected source data. Instance-based transfer methods reuse data from the source tasks to augment the training data for the target task. If poorly chosen, this source data may inhibit learning, resulting in negative transfer. The current best performing algorithm for instance-based transfer, TrAdaBoost, performs poorly when given irrelevant source data. We present a novel set-based boosting technique for instance-based transfer. The proposed algorithm, TransferBoost, boosts both individual instances and collective sets of instances from each source task. In effect, TransferBoost boosts each source task, assigning higher weight to those source tasks which show positive transferability to the target task, and then adjusts the weights of the instances within each source task via AdaBoost. The results demonstrate that TransferBoost significantly improves transfer performance over existing instance-based algorithms when given a mix of relevant and irrelevant source data.

   @inproceedings{Eaton2009SetBased,
     author = {Eaton, Eric and desJardins, Marie},
     title = {Set-Based Boosting for Instance-level Transfer},
     booktitle = {Proceedings of the International Conference on Data Mining Workshop on Transfer Mining},
     location = {Miami, FL},
     publisher = {IEEE Press},
     pages = {422--428},
     month = dec,
     year = {2009},
   }

2. Interactive Learning using Manifold Geometry

   Eric Eaton, Gary Holness, and Daniel McFarlane

   In Proceedings of the AAAI Fall Symposium on Manifold Learning and Its Applications (AAAI Technical Report FS-09-04), Arlington, VA, Nov 2009

   Superseded by the AAAI-10 conference paper Interactive Learning using Manifold Geometry.

   Bib

   @inproceedings{Eaton2009Interactive,
     author = {Eaton, Eric and Holness, Gary and McFarlane, Daniel},
     title = {Interactive Learning using Manifold Geometry},
     booktitle = {Proceedings of the AAAI Fall Symposium on Manifold Learning and Its Applications (AAAI Technical Report FS-09-04)},
     month = nov,
     location = {Arlington, VA},
     publisher = {AAAI Press},
     pages = {10--17},
     year = {2009},
   }

3. Selective Knowledge Transfer for Machine Learning

   Eric Eaton

   University of Maryland, Baltimore County, Nov 2009

   Abstract Bib

   Knowledge transfer from previously learned tasks to a new task is a fundamental component of human learning. Recent work has shown that knowledge transfer can also improve machine learning, enabling more rapid learning or higher levels of performance. Transfer allows learning algorithms to reuse knowledge from a set of previously learned source tasks to improve learning on new target tasks. Proper selection of the source knowledge to transfer to a given target task is critical to the success of knowledge transfer. Poorly chosen source knowledge may reduce the effectiveness of transfer, or hinder learning through a phenomenon known as negative transfer. This dissertation proposes several methods for source knowledge selection that are based on the transferability between learning tasks. Transferability is introduced as the change in performance on a target task between learning with and without transfer. These methods show that transferability can be used to select source knowledge for two major types of transfer: instance-based transfer, which reuses individual data instances from the source tasks, and model-based transfer, which transfers components of previously learned source models. For selective instance-based transfer, the proposed TransferBoost algorithm uses a novel form of set-based boosting to determine the individual source instances to transfer in learning the target task. TransferBoost reweights instances from each source task based on their collective transferability to the target task, and then performs regular boosting to adjust individual instance weights. For model-based transfer, the learning tasks are organized into a directed network based on their transfer relationships to each other. Tasks that are close in this network have high transferability, and tasks that are far apart have low transferability. Model-based transfer is equivalent to learning a labeling function on this network. This dissertation proposes the novel Spectral Graph Labeling algorithm that constrains the smoothness of the learned function using the graph’s Laplacian eigenvalues. This method is then applied to the task transferability network to learn a transfer function that automatically determines the model parameter values to transfer to a target task. Experiments validate the success of these methods for selective knowledge transfer, demonstrating significantly improved performance over existing methods.

   @phdthesis{Eaton2009Selective,
     author = {Eaton, Eric},
     title = {Selective Knowledge Transfer for Machine Learning},
     school = {University of Maryland, Baltimore County},
     year = {2009},
   }

4. Analysis of Complex Data Using Heterogeneous Relational Models

   Eric Eaton, Dan McFarlane, and Martin Hofmann

   Mar 2009

   Bib

   @techreport{Eaton2009Analysis,
     author = {Eaton, Eric and McFarlane, Dan and Hofmann, Martin},
     year = {2009},
     title = {Analysis of Complex Data Using Heterogeneous Relational Models},
     institution = {Lockheed Martin Advanced Technology Laboratories},
     number = {#DS-104-421-1610WP},
     pages = {6 pgs},
     month = mar,
   }

5. Situational Awareness through Interactive Learning

   Eric Eaton, Gary Holness, and Dan McFarlane

   Mar 2009

   Bib

   @techreport{Eaton2009Situational,
     author = {Eaton, Eric and Holness, Gary and McFarlane, Dan},
     year = {2009},
     title = {Situational Awareness through Interactive Learning},
     institution = {Lockheed Martin Advanced Technology Laboratories},
     number = {#DS-104-421-1607WP},
     pages = {4 pgs},
     month = mar,
   }

6. Dynamic Ensemble Planning for Tactical Hierarchies

   Meghann Lomas, Daniel McFarlane, Eric Eaton, and 2 more authors

   Mar 2009

   Bib

   @techreport{Lomas2009Dynamic,
     author = {Lomas, Meghann and McFarlane, Daniel and Eaton, Eric and Szczerba, Robert and Franke, Jerry},
     year = {2009},
     title = {Dynamic Ensemble Planning for Tactical Hierarchies},
     institution = {Lockheed Martin Advanced Technology Laboratories},
     number = {#DS-104-421-1604WP},
     pages = {4 pgs},
     month = mar,
   }

7. Predicting and Verifying Effects of Cyber Operations from Indirect Observations

   Eric Eaton, Katherine Guo, and Martin Hofmann

   Jan 2009

   Bib

   @techreport{Eaton2009Predicting,
     author = {Eaton, Eric and Guo, Katherine and Hofmann, Martin},
     year = {2009},
     title = {Predicting and Verifying Effects of Cyber Operations from Indirect Observations},
     institution = {Lockheed Martin Advanced Technology Laboratories},
     number = {#DS-105-421-1598WP},
     pages = {5 pgs},
     month = jan,
   }

2008

1. Multimodal and Temporal Learning using Relational Networks

   Eric Eaton, Katherine Guo, and Martin Hofmann

   Nov 2008

   Bib

   @techreport{Eaton2008Multimodal,
     author = {Eaton, Eric and Guo, Katherine and Hofmann, Martin},
     year = {2008},
     title = {Multimodal and Temporal Learning using Relational Networks},
     institution = {Lockheed Martin Advanced Technology Laboratories},
     number = {#DS-105-421-1583RFI},
     pages = {7 pgs},
     month = nov,
   }

2. Modeling Transfer Relationships Between Learning Tasks for Improved Inductive Transfer

   Eric Eaton, Marie desJardins, and Terran Lane

   In Proceedings of the 19th European Conference on Machine Learning (ECML), Antwerp, Belgium, Nov 2008

   Abstract Bib PDF

   In this paper, we propose a novel graph-based method for knowledge transfer. We model the transfer relationships between source tasks by embedding the set of learned source models in a graph using transferability as the metric. Transfer to a new problem proceeds by mapping the problem into the graph, then learning a function on this graph that automatically determines the parameters to transfer to the new learning task. This method is analogous to inductive transfer along a manifold that captures the transfer relationships between the tasks. We demonstrate improved transfer performance using this method against existing approaches in several real-world domains.

   @inproceedings{Eaton2008Modeling,
     author = {Eaton, Eric and desJardins, Marie and Lane, Terran},
     title = {Modeling Transfer Relationships Between Learning Tasks for Improved Inductive Transfer},
     booktitle = {Proceedings of the 19th European Conference on Machine Learning (ECML)},
     year = {2008},
     pages = {317--332},
     publisher = {Springer-Verlag},
     location = {Antwerp, Belgium},
     address = {Berlin, Heidelberg},
   }

3. Gridworld Search and Rescue: A Project Framework for a Course in Artificial Intelligence

   Eric Eaton

   In Proceedings of the AAAI-08 AI Education Colloquium, Jul 2008

   Abstract Bib PDF Website  

   This paper describes the Gridworld Search and Rescue simulator: freely available educational software that allows students to develop an intelligent agent for a search and rescue application in a partially observable gridworld. It permits students to focus on high-level AI issues for solving the problem rather than low-level robotic navigation. The complexity of the search and rescue problem supports a wide variety of solutions and AI techniques, including search, logical reasoning, planning, and machine learning, while the high-level GSAR simulator makes the complex problem manageable. The simulator represents a 2D disaster-stricken building for multiple rescue agents to explore and rescue autonomous injured victims. It was successfully used as the semester project for CMSC 471 (Artificial Intelligence) in Fall 2007 at UMBC.

   @inproceedings{Eaton2008Gridworld,
     author = {Eaton, Eric},
     title = {Gridworld Search and Rescue: A Project Framework for a Course in Artificial Intelligence},
     booktitle = {Proceedings of the AAAI-08 AI Education Colloquium},
     year = {2008},
     month = jul,
     address = {Chicago, IL},
     keywords = {education, project framework, search and rescue, gridworld},
   }

4. Gridworld Search and Rescue Software

   Eric Eaton

   Jul 2008

   Bib Website  

   @misc{Eaton2008GridworldSoftware,
     author = {Eaton, Eric},
     year = {2008},
     title = {Gridworld Search and Rescue Software},
     howpublished = {Available online at: http://maple.cs.umbc.edu/~ericeaton/searchandrescue/},
   }

5. Using functions on a model graph for inductive transfer

   Eric Eaton, Marie desJardins, and Terran Lane

   In Proceedings of the Northeast Student Colloquium on Artificial Intelligence (NESCAI-08), May 2008

   Superseded by the ECML-08 paper Modeling Transfer Relationships Between Learning Tasks for Improved Inductive Transfer.

   Abstract Bib PDF

   In this paper, we propose a novel graph-based method for knowledge transfer. We embed a set of learned background models in a graph that captures the transferability between the models. We then learn a function on this graph that automatically determines the parameters to transfer to each learning task. Transfer to a new problem proceeds by mapping the problem into the graph, then using the function to determine the parameters to transfer in learning the new model. This method is analogous to inductive transfer along a manifold that captures the transfer relationships between the tasks.

   @inproceedings{Eaton2008Using,
     author = {Eaton, Eric and desJardins, Marie and Lane, Terran},
     title = {Using functions on a model graph for inductive transfer},
     booktitle = {Proceedings of the Northeast Student Colloquium on Artificial Intelligence (NESCAI-08)},
     year = {2008},
     month = may,
     address = {Ithaca, NY},
   }

2007

1. Using multiresolution learning for transfer in image classification

   Eric Eaton, Marie desJardins, and John Stevenson

   In Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI), May 2007

   [Student Abstract]

   Abstract Bib PDF Poster  

   Our work explores the transfer of knowledge at multiple levels of abstraction to improve learning. By exploiting the similarities between objects at various levels of detail, multiresolution learning can facilitate transfer between image classification tasks. We extract features from images at multiple levels of resolution, then use these features to create models at different resolutions. Upon receiving a new task, the closest-matching stored model can be generalized (adapted to the appropriate resolution) and transferred to the new task.

   @inproceedings{Eaton2007Using,
     author = {Eaton, Eric and desJardins, Marie and Stevenson, John},
     year = {2007},
     title = {Using multiresolution learning for transfer in image classification},
     booktitle = {Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI)},
     address = {Vancouver, British Columbia, Canada},
     publisher = {AAAI Press},
     note = {[Student Abstract]},
     bib2html_dl_poster = {papers/poster-Eaton2007Using.pdf}
   }

2006

1. Multi-Resolution Learning for Knowledge Transfer

   Eric Eaton

   In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), Jul 2006

   [Doctoral Consortium]

   Abstract Bib PDF

   Related objects may look similar at low-resolutions; differences begin to emerge naturally as the resolution is increased. By learning across multiple resolutions of input, knowledge can be transfered between related objects. My dissertation develops this idea and applies it to the problem of multitask transfer learning.

   @inproceedings{Eaton2006MultiResolution,
     author = {Eaton, Eric},
     title = {Multi-Resolution Learning for Knowledge Transfer},
     booktitle = {Proceedings of the 21st National Conference on Artificial Intelligence (AAAI)},
     year = {2006},
     address = {Boston, MA},
     month = jul,
     publisher = {AAAI Press},
     note = {[Doctoral Consortium]},
     keywords = {knowledge transfer, multiresolution learning, aaai doctoral consortium},
   }

2. Knowledge Transfer with a Multiresolution Ensemble of Classifiers

   Eric Eaton and Marie desJardins

   In Proceedings of the ICML-06 Workshop on Structural Knowledge Transfer for Machine Learning, Jun 2006

   Abstract Bib PDF

   We demonstrate transfer via an ensemble of classifiers, where each member focuses on one resolution of data. Lower-resolution ensemble members are shared between tasks, providing a medium for knowledge transfer.

   @inproceedings{Eaton2006Knowledge,
     author = {Eaton, Eric and desJardins, Marie},
     title = {Knowledge Transfer with a Multiresolution Ensemble of Classifiers},
     booktitle = {Proceedings of the ICML-06 Workshop on Structural Knowledge Transfer for Machine Learning},
     year = {2006},
     address = {Pittsburgh, PA},
     month = jun,
     keywords = {knowledge transfer, multiresolution learning},
   }

3. Learning user preferences for sets of objects

   Marie desJardins, Eric Eaton, and Kiri Wagstaff

   In Proceedings of the 23rd International Conference on Machine Learning (ICML), Jun 2006

   Awarded recognition as a NASA Tech Brief in 2008

   Abstract Bib PDF

   Most work on preference learning has focused on pairwise preferences or rankings over individual items. In this paper, we present a method for learning preferences over sets of items. Our learning method takes as input a collection of positive examples–that is, one or more sets that have been identified by a user as desirable. Kernel density estimation is used to estimate the value function for individual items, and the desired set diversity is estimated from the average set diversity observed in the collection. Since this is a new learning problem, we introduce a new evaluation methodology and evaluate the learning method on two data collections: synthetic blocks-world data and a new real-world music data collection that we have gathered.

   @inproceedings{desJardins2006Learning,
     author = {desJardins, Marie and Eaton, Eric and Wagstaff, Kiri},
     title = {Learning user preferences for sets of objects},
     booktitle = {Proceedings of the 23rd International Conference on Machine Learning (ICML)},
     year = {2006},
     month = jun,
     address = {Pittsburgh, PA},
     bib2html_dl_code = {software/DDPref.zip}
   }

4. DDPref Software: Learning preferences for sets of objects

   Eric Eaton, Marie desJardins, and Kiri Wagstaff

   Jun 2006

   Bib

   @misc{Eaton2006DDPrefSoftware,
     author = {Eaton, Eric and desJardins, Marie and Wagstaff, Kiri},
     year = {2006},
     title = {DDPref Software: Learning preferences for sets of objects},
     howpublished = {Available online at: http://maple.cs.umbc.edu/~ericeaton/software/DDPref.zip},
     bib2html_dl_code = {software/DDPref.zip}
   }

2005

1. Clustering with Propagated Constraints

   Eric Eaton

   University of Maryland Baltimore County, Jun 2005

   Abstract Bib PDF

   Background knowledge in the form of constraints can dramatically improve the quality of generated clustering models. In constrained clustering, these constraints typically specify the relative cluster membership of pairs of points. They are tedious to specify and expensive from a user perspective, yet are very useful in large quantities. Existing constrained clustering methods perform well when given large quantities of constraints, but do not focus on performing well when given very small quantities. This thesis focuses on providing a high-quality clustering with small quantities of constraints. It proposes a method for propagating pairwise constraints to nearby instances using a Gaussian function. This method takes a few easily specified constraints, and propagates them to nearby pairs of points to constrain the local neighborhood. Clustering with these propagated constraints can yield superior performance with fewer constraints than clustering with only the original user-specified constraints. The experiments compare the performance of clustering with propagated constraints to that of established constrained clustering algorithms on several real-world data sets.

   @mastersthesis{Eaton2005MastersThesis,
     author = {Eaton, Eric},
     title = {Clustering with Propagated Constraints},
     school = {University of Maryland Baltimore County},
     year = {2005},
   }

2. A context-sensitive and user-centric approach to developing personal assistants

   Marie desJardins, Eric Eaton, and Kiri Wagstaff

   In Proceedings of the AAAI Spring Symposium on Persistent Assistants, Mar 2005

   Abstract Bib PDF

   Several ongoing projects in the MAPLE (Multi-Agent Planning and LEarning) lab at UMBC and the Machine Learning Systems Group at JPL focus on problems that we view as central to the development of persistent agents. This position paper describes our current research in this area, focusing on four topics in particular: effective use of observational and active learning, utilizing repeated behavioral contexts, clustering with annotated constraints, and learning user preferences.

   @inproceedings{desJardins2005ContextSensitive,
     author = {desJardins, Marie and Eaton, Eric and Wagstaff, Kiri},
     title = {A context-sensitive and user-centric approach to developing personal assistants},
     booktitle = {Proceedings of the AAAI Spring Symposium on Persistent Assistants},
     year = {2005},
     month = mar,
     address = {Stanford, CA},
     pages = {98--100},
   }