I'm currently working with Michael Black on preference optimization for large language models (LLMs). Preference optimization is a key technique in aligning LLMs with human feedback, ensuring that the model’s outputs better match desired behaviors, whether for safety, accuracy, or usability. Our work focuses on refining these methods to improve model performance while maintaining efficiency.