My research focuses on the design of efficient algorithms for modeling and predicting multiagent system behavior through game theory and reinforcement learning. I am now curious in approaches that leverage deep learning methods (hypernetworks, diffusion-based RL-policies) and population-based learning (PSRO, EGTA) especially how they apply to multiagent analysis of LLM-based systems.
Research Interests
Where I Think the Field is Going
Just like research, I tried to make this page (hopefully) appealing and fun. I'm not saying my job is happy and dandy all the time. I just think that "serious" things deserve love sometimes, especially things we care about.
In-Progress
This paper presents a novel offline adaptation of PSRO, where game-solving is restricted to a fixed dataset of gameplay. Offline model-based "discrepancy" is used to encourage conservatism when training new strategies and determining response targets within PSRO.
This project extends PSRO to skew strategy exploration towards high-welfare equilibria. Drawing inspiration from behavior regularization in offline RL, Ex2PSRO (Espresso) skews best-responses towards behavior described by a dataset of trajectories gathered during online exploration.
A good friend of mine and I built upon the recent Decision Transformer (DT) for an NLP class at UMich. We adapted DT to online contexts, training DT tabula rasa with a SOTA online RL algorithm for exploration.
A group of friends and I gave our take on adversarial text generation. We used an RL method that adversarially swapped words in input texts to fool state-of-the-art text classification models.
Another group of friends and I applied machine learning to a compiler problem: register allocation.
If the problem can be reduced to a graph coloring problem, why not apply machine learning?
This project created RL agents that imitate how ants communicate through pheromones to induce a desired joint behavior: decentralized coordination through indirect communication. Agents learned how to modify the environment for traversal through single-agent RL and a coordination algorithm to have multiple agents cooperatively navigate to a goal location.
A mechanism of indirect coordination through the environment, characterized by traces left in the environment by agents to stimulate the performance of succeeding actions by the same or different agents.
For UC Berkeley's Deep Reinforcement Learning Class (CS 285), a few friends and I combined two recent RL approaches to create a conservative single-agent RL algorithm, maximizing the worst-case return within a specificed lower percentile (CVar). Cool project.
Sometimes optimizing for the "worst-case" scenario is valuable to us. For example, it is unlikely to encounter road hazards while driving. However, agents that optimize for the average case may not account for this situation, whereas more risk-averse agents would be more prepared.
I tend to be quite responsive over email. Please feel free to reach out anytime. It doesn't even have to be research-related.