What do you (want to) work on?

October 2026 · Simran Khanuja

This is a question I get asked ever so often these days, whenever I mention that I'm going to be on the job market. A few months ago I had a very long response to it, something along the lines of:

After multiple conversations like this, and advice from friends who care about me and my future :'), I realized that keywords matter: they're how employers screen and figure out where someone might fit. I did do my due diligence (check out my website!) but in the process, it just didn't feel right to define myself as a pre-training / mid-training / post-training / SFT / RL / alignment / interpretability / synthetic data / RSI / evals / safety / AI-for-science / agents ... person.

I see myself as more of a problem person. I have a few criteria I care about when choosing a problem, shaped by my experiences so far, where I'm situated (industry/academia have different problems you can/need to work on), and what I value most in my life right now. We're also fortunate to be in an era where the foundations are strong enough to execute ideas quickly, so I see choosing well becoming an even more important skill with time. My mentors early on would often say that research is a lot about asking the right questions, among other things. What "right" means is subjective though, and I've watched its definition change for me over the last few years. When I was in industry, the right problems were kinda defined for me: ones inspired by a product need that could eventually make their way back into the product. In hindsight, I liked that definition, especially when your solution actually ends up in something used by a lot of people. Going into my PhD, I had a very romanticized notion of what I wanted to do; my definition of right was to work on something unique, out of the box, not low-hanging fruit, something at the edge of innovation (if y'all have seen this graph [1], I wanted to make that small, but large, dent :')). I wanted to be intellectually stimulated, and was surprisingly (to me now) unbothered by hype-y things (probably because I didn't yet appreciate that things usually get popular because they actually work). But I still wanted the problems to come from real-world needs, probably an artifact of spending my early research years in industry, and of an advisor who really values this (he always asks, "who do you wish will use this?").

Today, I think I still hold the definition of "right" in that romanticized light, but it's wayyy more nuanced. After many bitter lessons along the way, I've learnt to also make sure it's feasible to work on with the technology we have today, or that we can build the tech reliably. I deeply care for real-world impact and relatability, something I can simply explain to my parents and they can immediately appreciate or recognize the problem. My own little ikigai lies somewhere in here:

That's also what settled my decision to go to industry after the PhD, to be closer to products people use (I was really confused about this last year hehe).

So that's the TL;DR of where I'm at. It comes from reflecting on my research journey since 2017. The rest of this blog is divided into four parts: the first covers my thought process pre-PhD, when I was in industry; the second covers how and why I chose the problems I did during my PhD (till the end of my third year, i.e., summer 2025); the third covers how I've been thinking about research since then; and the last just closes out with more details on where I'm at today. A lot of the way I think has also changed because of the exponential progress we've made since 2022, when I started my PhD, so I think this would probably have felt different if I'd been in a similar situation a few years earlier. The stuff below is also partly me reflecting on my own journey thus far, as a checkpoint for my future self to come back to (wonder what she'll think about this).

2019–2022: when right problems were ones that could improve products people use

My research journey started in 2017. I still remember sitting in the library at BITS Goa, my undergrad (by the window close to where Mongi's previously was, iykyk), watching a Magenta demo [2] that had just come out, and being blown away. I had a vague understanding of machine learning at the time, but after that demo I was convinced I wanted to do research in AI for music. When I started looking for research opportunities the following summer, though, practically all I could find were internships in NLP and CV. I picked NLP, partly because it felt closest to music: you're processing long sequences of tokens, and the input space overlaps more than it does with vision. I thought it would be easier to pivot (still waiting on that pivot :/).

Being in India, most of the problems I worked on were about making language technology work for India: building multilingual models that work well with very little data, and code-mixing, a very common phenomenon in multilingual communities. From 2019 to 2020 I was at Microsoft Research India. It was more of an academic lab than a product one, but it was my first experience doing research full-time, and I was really fortunate to learn from great mentors. My mentor had the idea of building the first benchmark for code-mixed NLP, on the principle that evaluation is what drives progress on any problem, and that became GLUECoS [3]. While we were building the benchmark, I wanted to repurpose a dataset of Bollywood movie conversations into an NLI dataset [4]. I pitched it to my mentors; they asked me to run a pilot with my friends at the lab first; the pilot went well; and we did data annotation at scale after. I was really proud of this back then because it was the first time I pitched my own idea and saw it come to fruition.

Then, from 2020 to 2022, I was at Google Research India. My first project was also somewhat defined for me: build a multilingual model for Indian languages and open-source it for the community. That became MuRIL [5], and it was so much fun to build. First time dealing with Google infra, training on TPUs, coding in TF1 -_-, lots of data collection and cleaning, benchmarking on a lot of evals. I learned so much, especially about engineering and optimization, and I trained what was a large model at the time (550M params, be kind, this was large back then :')). I also learned how to ask the community for help; somewhere inside Google there's a very, very long thread on multilingual pretraining best practices from 2020 (anyone there reading this can probably find it?). The release was exciting! People started fine-tuning it for all sorts of tasks, which was a great feeling after building it.

After MuRIL, we wanted to merge it into a production system, and that need shaped my next project, MergeDistill [6]. The idea was to do the research with open datasets and then use it to integrate our system into the product, which we did. This was a fulfilling loop: the problem came from a product need, we did research on it, and then we shipped it and it made the product better. I also dabbled in some other projects I found exciting at the time (a notable one was speech-text cross-modal retrieval, with a potential application of searching for relevant video snippets on YT given text queries; a small part of this became part of the FLEURS paper [7]).

Something I also learned there was that your contribution can be as small as a one-line change in a codebase (I changed AdamW to LAMB as the optimizer we were using to train nightly production models and it gave us an almost 10x speedup because LAMB worked well with large batch sizes) or as big as pretraining a model from scratch. The scale of impact it creates matters more to your peers. A lot of colleagues working in production valued the first one way more than the second.

By the end of my time there I could see how much people around me, in an applied research setting, valued this closing of the loop. I'm not sure I fully appreciated the beauty of it back then; I value it much more today. At the time I was mostly just excited to start my PhD and do my own thing, and downstream use didn't mean as much to me as it does now. Somehow, during the PhD, I came back around to it.

2022–2025: when right problems were ones that nobody had thought about, but have impact

When I was starting my PhD, like I mentioned, I had this romanticized idea of what it would look like: going deep into one topic, proposing novel ideas, coming up with interesting hypotheses while working on real-world problems. I wanted to be intellectually stimulated, and in hindsight I wasn't very open-minded about what that could mean.

I remember, somewhere in my first year, telling my advisor that I would read a lot of papers and think of cool things to do. His advice was that if I wanted to find what to work on, I shouldn't look for it inside papers but outside, in the real world: observe what people need (he also sent me an article by an economist on how he looks for problems [8], which is where a lot of this comes from). He would also advise anyone building models or systems: inspect the outputs, find the errors, look for patterns in them, and let that tell you where the model is going wrong.

All of this took me back to a conversation from my industry days, with an organization that publishes children's storybooks in many languages. One of their stories (Ammachi's Amazing Machines) was about a grandmother wearing a sari, making an Indian dessert for her grandson, told in pictures and text. The stories' text had been translated into many other languages by humans, but they noticed that children were not reading the translated stories. One hypothesis was that, despite translating the text, the images were highly contextualized to where the illustrators were from or what the story was about; in this specific case, children reading the story in Spanish cannot relate to a woman in a sari as their grandmother. Disney and Pixar pay a lot of attention to this. In the Japanese release of Inside Out they made a bunch of visual changes, like swapping broccoli for bell peppers as the vegetable kids hate. Some ad companies do it too; Coca-Cola localizes its ad imagery for wherever an ad is released. But machine translation had always been confined to dealing with language, in speech or text.

I spent a lot of time reading about how localization is done in several industries outside of AI. It's extremely prevalent and important in movies, ads, education, gaming, websites, and so on. Scholars in translation studies challenge the notion of simply translating words, highlighting that visuals, music, and other elements contribute equally to meaning. While each modality carries its own information, interaction between modalities creates deeper, emergent meanings. Partial translation disturbs this multimodal interaction and causes cognitive dissonance to the receptor. The whole practice goes by the term transcreation, and the basic goal is to evoke the same emotions with the translated content as the source does. The book I read at the time, Media Across Borders [9], gave me a lot of ideas for how to operationalize something like this for machine learning.

I also liked how relatable the problem was. No matter what someone's professional background was, the problem made sense to them. They'd often say, "Oh yeah, that makes sense. Has nobody worked on it yet?" and sometimes give me new examples they'd noticed in the wild, some of which would end up in the intros of my papers :)

I first published this at EMNLP 2024 [10], and did some follow-up work after that. I was extremely grateful to even receive academic recognition for it, post which I thought a lot of people would start working on it. I mean, the problem just makes sense, and it's so cool, right? But they didn't, and this is where I learned my bitter lessons :') It's also when I realized that what actually matters to me more is people actually using what I research or build.

The main issue was that evaluation wasn't scalable. We relied on human evaluation. We defined a few dimensions along which a culturally transcreated image can be judged, but there's still no reliable way to do it automatically. VLMs can do a decent job, but they have their own limitations, and we couldn't come up with an automatic eval that was scalable, reliable, verifiable, and still correlated well with the downstream task. I really liked the design of the test set we released: it had a subset derived from math worksheets and storybooks, i.e., application data, and even now frontier models can't transcreate those images well (some examples are in these slides).

What this taught me is that even with academic recognition, even if a problem is intellectually stimulating, has potential for real-world impact, and is simple enough that a lot of people understand and relate to it, that isn't enough. With where the field was two years ago, it wasn't feasible to build the best technology for it, and without that practicality, the potential impact can't be realized. That's when my desire to build something people are actually going to use got a lot stronger (and also when I started appreciating making tasks verifiable and building great evals).

2025–now: when right problems became ones that solve a real-world problem and are feasible to work on

Based on everything so far, when it came to deciding on proposed work, I just wanted to build a model that works really well and that people actually use. That's what drove my latest project, T-DIG [11].

Diversity in image generation is a known problem, and I started with the methods that seemed interesting to me from a research & learning perspective. I wanted to come up with better sampling algorithms that explore the latent space of diffusion models. I played around with a few prior methods, but they often produced undesired artifacts or broke the model in unexpected ways. It felt a lot like trying to steer LLMs with mech interp techniques, which I'd dabbled with in my internship project [12], where modifying internal activations would similarly start producing gibberish or incoherent text at times. Even though it seemed like a fun way to work on the problem, it felt like it might not generalize in the wild, and hence I decided to pivot.

The other option was to post-train, but with the way I'd defined diversity (conditioned on a user's prompt), I didn't have a good metric to optimize for. That was true until two months ago, when TPIPS [13] came out. They had somewhat magically defined the task in a very similar way to how I had, and proposed a really good eval metric for it, trained on a million human judgments. With access to that metric, we trained T-DIG: an LLM policy that writes prompts for a text-to-image model, trained with a reward that tries to maximize diversity but carries alignment penalties, length penalties, and a bunch of other terms to keep reward hacking in check. In our human studies the feedback was very wholesome: people actually enjoyed playing around with the model, and it looked like it added something to their workflows. We're releasing it this week and I'm pretty excited about it!

Soooo... what do you want to work on?

I think there's an explore/exploit tradeoff in how you spend your research years, same as in the models we train. When you feel really strongly about something, it's best to exploit: go deep, build something and see it through. When you don't, it's a good time to step back and explore, and it's also a good opportunity to catch up on everything you didn't keep track of while you were busy exploiting.

Right now, I'm excited to enjoy a short exploration phase and am looking forward to joining industry somewhere where I can maximize learning, specifically learning how to build frontier models at scale. Longer term, problems in AI for science seem really exciting to me; with what we have today, I think there's a lot of impact to be made in drug discovery and healthcare in particular (it's something I'm passionate about outside of my work too). Although I'm just starting out and exploring the space :)

I do believe that problem selection in research is an art, and one must be thoughtful about it and align it to your values from time to time. Going forward, I still want to work on things that are at the edge of innovation and impactful to people, but feasible to solve and scale. With all the foundations the community has built up to now, I really don't know what that will look like even a year from now, but I'm so excited for the future ahead!

This will change

This is subject to change, and most likely will, as I gather more experiences. It reflects what I've learned so far and a few things I'm most excited about right now, not everything. I'd love for this to keep evolving with new conversations, ideas, results and people. If something here strongly resonates with you, or doesn't, I'd love to chat :)

Note: everything in this post is written by me. I used Claude as a last pass to catch grammar and typos, to make the ikigai figure, to help link the references, and for the frontend design. This post also includes a lot of advice from the wonderful mentors I've had the opportunity to learn from (Sunayana Sitaram, Monojit Choudhury, Partha Talukdar, Sebastian Ruder, and Graham Neubig) throughout my research journey, and I'm very grateful for all of it.

References

1. The Illustrated Guide to a Ph.D. Matt Might. 2010. ↩
2. NSynth: Neural Audio Synthesis. Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi. Magenta, Google Brain, 2017. ↩
3. GLUECoS: An Evaluation Benchmark for Code-Switched NLP. Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan, Sunayana Sitaram, Monojit Choudhury. ACL 2020. ↩
4. A New Dataset for Natural Language Inference from Code-mixed Conversations. Simran Khanuja, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury. CALCS @ LREC 2020. ↩
5. MuRIL: Multilingual Representations for Indian Languages. Simran Khanuja, Diksha Bansal, Sarvesh Mehtani, Savya Khosla, Atreyee Dey, Balaji Gopalan, Dilip Kumar Margam, Pooja Aggarwal, Rajiv Teja Nagipogu, Shachi Dave, Shruti Gupta, Subhash Chandra Bose Gali, Vish Subramanian, Partha Talukdar. arXiv 2021. Model on Hugging Face. ↩
6. MergeDistill: Merging Pre-trained Language Models using Distillation. Simran Khanuja, Melvin Johnson, Partha Talukdar. Findings of ACL 2021. ↩
7. FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, Ankur Bapna. SLT 2022 (Best Paper). ↩
8. How to Build an Economic Model in Your Spare Time. Hal R. Varian. The American Economist, 2016. ↩
9. Media Across Borders: Localising TV, Film and Video Games. Andrea Esser, Miguel Á. Bernal-Merino, Iain Robert Smith (eds.). Routledge, 2016. ↩
10. An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance. Simran Khanuja, Sathyanarayanan Ramamoorthy, Yueqi Song, Graham Neubig. EMNLP 2024 (Best Paper). ↩
11. T-DIG: Text-Conditioned Diversity for Image Generation. Simran Khanuja, Rohit Gandikota, Gaurav Parmar, Graham Neubig. arXiv 2026. ↩
12. Steering LLMs for Culturally Localized Generation. Simran Khanuja, Hongbin Liu, Shujian Zhang, John Lambert, Mingqing Chen, Rajiv Mathews, Lun Wang. COLM 2026. ↩
13. The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric. Sheng-Yu Wang, Yotam Nitzan, Aaron Hertzmann, Jun-Yan Zhu, Eli Shechtman, Alexei A. Efros, Richard Zhang. arXiv 2026. ↩