Hi, I'm Tyler Romero.

Tyler Romero I am a Member of Technical Staff on the MAI Superintelligence Team. Previously, I was a Research Engineer at the Allen Institute (Ai2), where I worked on open language modeling.

Before Ai2, I was Lead ML Engineer at Groundlight, a startup building multimodal question-answering systems. Prior to that, I worked on large-scale recommender systems at Twitter, where I developed ranking models and ran the A/B tests for the experimentally successful but short-lived downvote button. I also researched, trained, and shipped model architecture improvements for ranking Twitter's home timeline and conversation reply trees. Some of my work at Twitter is now open source. The public repository uses sanitized git history. Earlier in my career, I also worked as a Research Scientist at Microsoft, where I built new machine-learning systems from the ground up.

I studied CS and ML at Stanford and computer engineering at Texas A&M, where I researched parallel algorithms in C / Cilk. Outside of work, I contribute to open-source ML projects and write about what I’m learning here.

Posts

Publications & Preprints

  • Jun 2026
    MAI-Thinking-1 Watercolor circles arranged as a thought bubble for MAI-Thinking-1 — The Microsoft AI Team.
    Microsoft AI technical report.

    MAI-Thinking-1 is a 35B active / 1T total parameter MoE reasoning model trained from scratch (without distillation from third-party models). We present a scaling-focused pre-training framework paired with a robust RL recipe and infrastructure that sustains long, log-linear improvement. It is competitive with frontier models of similar size on STEM reasoning and coding.

  • Mar 2026
    Olmo Hybrid Olmo Hybrid expressivity Venn diagram — William Merrill*, Yanhong Li*, Tyler Romero*, Anej Svete*, Caia Costello*, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Chuan Li, Kyle Lo, Saumya Malik, DJ Matusz, Benjamin Minixhofer, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish Sabharwal*.
    arXiv preprint arXiv:2604.03444.

    We provide evidence for the advantages of hybrid models (mixing attention and linear RNN layers) over pure transformers on several fronts: theoretically, we show that hybrid models can express tasks beyond both transformers and linear RNNs individually; empirically, we find that hybrids scale more efficiently. We train Olmo Hybrid, a 7B model with sliding window layers replaced by Gated DeltaNet layers, and show it outperforms Olmo 3 across standard pretraining and long context evaluations.

  • Nov 2025
    Olmo 3 Olmo 3 32B training loss curve — Team Olmo (incl. Tyler Romero, core contributor).
    arXiv preprint arXiv:2512.13961.

    We introduce Olmo 3, a family of state-of-the-art, fully-open language models at the 7B and 32B parameter scales. Olmo 3 model construction targets long-context reasoning, function calling, coding, instruction following, general chat, and knowledge recall. This release includes the entire model flow, i.e., the full lifecycle of the family of models, including every stage, checkpoint, data point, and dependency used to build it. Our flagship model, Olmo 3 32B Think, is the strongest fully-open thinking model released to-date.

Projects & Open Source

  • greyhound

    I've been using greyhound as a place to build and experiment with CuTe DSL kernels for PyTorch, with an eye toward making training and inference workloads faster and easier to hack on. So far it includes kernels for cross-entropy and chunked linear losses, selective log-softmax, and causal Conv1D; see the greyhound documentation and benchmarks.

  • olmo-core

    Building blocks for OLMo modeling and distributed training. I did substantial work in OLMo-core on pretraining and model development, including MFU improvements, scaling ladders, and the training infrastructure used for OLMo 3 and OLMo Hybrid.

  • Liger-Kernel

    Recently I've been contributing to Liger-Kernel, a collection of custom Triton Kernels for efficient LLM training. I've found these kernels very useful for training LLMs/VLMs on my RTX 4090. My contributions, as well as those of other top collaborators, were recently featured in LinkedIn Engineering's overview of the Liger Kernel ecosystem.

  • microR1

    A micro-scale DeepSeek-R1 reproduction in the style of Karpathy's nanoGPT. Intended to be easy to understand and to hack on top of.

Favorite Reads

Fun Stuff