I am a computer science PhD student at the Massachusetts Institute of Technology (MIT) — studying artificial intelligence through natural language processing and machine learning. I am lucky to be advised by Jacob Andreas.
I work on improving sequence modeling for language processing and understanding. Languages exhibit some notion of compositionality (productivity and systematicity) whereas current neural language learners lack required inductive biases to achieve this data-efficiently. My recent work aims at understanding simple biases that will enable neural sequence models to achieve types of generalization that humans do
I am also interested in language supervision/grounding and worked on two recent projects: (i) using language to guide image classifiers to learn representations that enable learning of new classes (only with few samples) without forgetting the old ones, (ii) using language models to guide policy learning in a virtual home environment.
Previously,
- I was student researcher at Google Research Brain Team and working with Tengyu Ma, Denny Zhou and Dale Shuurmans on understanding in-context learning.
- I had a summer internship at Google Research and worked with Kelvin Guu and Keith Hall on fact attribution by using large language models.
- As a visiting student at @MIT CSAIL, I worked with Prof. Alan Edelman on linear algebraic formulation of backpropagation to facilitate existing matrix operations on neural computation graphs;
- I worked with John W. Fischer on efficient distributed algorithms for non-bayesian parametric;
- I was a part of KUIS AI Lab during my undergraduate education and I worked with Prof. Deniz Yuret on natural language processing.
- I received my Bachelor’s degrees in Electrical&Electronics Engineering and in Physics from Koç University in 2019.
Publications
What learning algorithm is in-context learning? Investigations with linear models
Ekin Akyürek, Dale Schuurmans, Jacob Andreas*, Tengyu Ma*, Denny Zhou*
International Conference on Learning Representations, ICLR 2023 (oral)
Compositional Semantic Parsing with Large Language Models
Andrew Drozdov*, Nathanael Schärli*, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, Denny Zhou
International Conference on Learning Representations, ICLR 2023
Notes on Teaching GPT-3 Adding Numbers
Ekin Akyürek, Afra Feyza Akyürek
Blog Post
Tracing Knowledge in Language Models Back to the Training Data
Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu
Findings of the Conference on Empirical Methods in Natural Language Processing, EMNLP, 2022
Compositionality as Lexical Symmetry
Ekin Akyürek, Jacob Andreas
preprint
Pre-Trained Language Models for Interactive Decision-Making
Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, Yuke Zhu
Neural Information Processing Systems, Neurips, 2022 (oral)
Backpropagation through Back substitution with a Backslash
Ekin Akyürek, Alan Edelman, Bernie Wang (2021)
Neurips Differential Programming Workshop, 2021
Subspace Regularizers for Few-Shot Class Incremental Learning
Afra Feyza Akyürek, Ekin Akyürek, Derry Wijaya, Jacob Andreas (2021)
International Conference on Learning Representations, ICLR 2022
Lexicon Learning for Few-Shot Neural Sequence Modeling
Ekin Akyürek, Jacob Andreas (2021)
Annual Meeting of the Association for Computational Linguistics, ACL 2021 (oral)
Learning To Recombine and Resample Data For Compositional Generalization
Ekin Akyürek, Afra Feyza Akyürek, Jacob Andreas (2020)
International Conference on Learning Representations, ICLR 2021
Morphological Analysis Using Sequence Decoder
Ekin Akyürek*, Erenay Dayanık*, Deniz Yuret (2019)
Transactions of the Association for Computational Linguistics, TACL (also presented at EMNLP 2019)
Through the Glance Mug: A Familiar Artefact to Support Opportunistic Search in Meetings
Ahmet Börütecene, İdil Bostan, Ekin Akyürek, Alpay Sabuncuoğlu, İlker Temuzkuşu, Çağlar Genç, Tilbe Göksun, Oğuzhan Özcan (2018)
In Proceedings of the 12th International Conference on Tangible, Embedded and Embodied Interaction, TEI 2018, ACM
Locales




