What I Care About

I generally consider myself a utilitarian and a long-termist. I care a lot about AI safety. My P(doom) is a tentative 35%. I am interested in working on technical AI safety research. Right now, I am particularly interested in conceptual writing and alignment research. This is based on some mixture of my aptitudes and the kinds of approaches that I am bullish on in terms of impact.

Credentialing

I am an undergraduate studying computer science at Georgia Tech. I am a Pivotal Research Fellow. I am the Fellowship Lead for the AI Safety Initiative at Georgia Tech and previously co-led the Effective Altruism club at Georgia Tech. I completed the Machine Learning Safety Scholars program in 2022. See also current research projects.

Skills

I think I am good at thinking about abstract problems (like math/philosophy-ish things). For example, I think I am good at reasoning about goals and probabilities. I think I am also decent at computer science (I would estimate around average for a GT CS grad, which is a little below average for a MATS technical scholar). I think I am conscientious, agentic, and work well with others.

Current Research Projects

RL Adaptations for Neutrality+

I am working with Elliott Thornley on an RL implementation of Neutrality+ (a part of his POST-Agency Proposal). Neutrality+ agents are theoretically shutdownable because their preferences are represented by an average utility across trajectory lengths rather than an expected utility across trajectory lengths (no weighting by probability). This means they are indifferent to shifting probability across trajectory lengths, which results in them not taking costly actions to avoid shutdown. Our RL implementation uses empirical results for probabilities of trajectory lengths in a batch as an estimator for objective probabilities in order to implement the Neutrality+ objective function.

Evaluation-Conditioned Training (ECT)

I am working with Kasey Corra, Archie Chaudhury, Yixiong Hao, Leo Richter, and Chris Harig on Evaluation-Conditioned Training (ECT), a post-training framework for improving performance under imperfect feedback. ECT conditions training examples on natural language descriptions of the evaluator, then conditions the model on a stronger evaluation regime at deployment. It works as an add-on to methods such as supervised fine-tuning and PPO. In two proof-of-concept experiments using mis-specified feedback, ECT increased political even-handedness under biased training data and reduced sycophancy on arithmetic tasks relative to standard training.

LLMs for Automated Red Teaming Using Steering Vectors

I am working with Dr. Kartik Goyal, Andrew Wei, and Sarvesh Tiku on using LLMs for automated red teaming using steering vectors. We find steering vectors for refusal and then train LLMs to create prompts that either over- or under-trigger these vectors in order to create over-refusal and jailbreak prompts.