Hi, I'm Tyler Romero.

Tyler Romero I am a Member of Technical Staff on the MAI Superintelligence Team, where I work on post-training. Previously, I was a Research Engineer at the Allen Institute (Ai2), where I worked on pretraining research for open language models.

Before that, I led ML at Groundlight and built large-scale recommender systems at Twitter, some of which is now open source. I studied CS and ML at Stanford and computer engineering at Texas A&M.

Posts

Publications & Preprints

  • Jun 2026
    MAI-Thinking-1 Watercolor circles arranged as a thought bubble for MAI-Thinking-1 — The Microsoft AI Team.
    Microsoft AI technical report.

    MAI-Thinking-1 is a 35B active / 1T total parameter MoE reasoning model trained from scratch (without distillation from third-party models). We present a scaling-focused pre-training framework paired with a robust RL recipe and infrastructure that sustains long, log-linear improvement. It is competitive with frontier models of similar size on STEM reasoning and coding.

  • Mar 2026
    Olmo Hybrid Olmo Hybrid expressivity Venn diagram — William Merrill*, Yanhong Li*, Tyler Romero*, Anej Svete*, Caia Costello*, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Chuan Li, Kyle Lo, Saumya Malik, DJ Matusz, Benjamin Minixhofer, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish Sabharwal*.
    arXiv preprint arXiv:2604.03444.

    We provide evidence for the advantages of hybrid models (mixing attention and linear RNN layers) over pure transformers on several fronts: theoretically, we show that hybrid models can express tasks beyond both transformers and linear RNNs individually; empirically, we find that hybrids scale more efficiently. We train Olmo Hybrid, a 7B model with sliding window layers replaced by Gated DeltaNet layers, and show it outperforms Olmo 3 across standard pretraining and long context evaluations.

  • Nov 2025
    Olmo 3 Olmo 3 32B training loss curve — Team Olmo (incl. Tyler Romero, core contributor).
    arXiv preprint arXiv:2512.13961.

    We introduce Olmo 3, a family of state-of-the-art, fully-open language models at the 7B and 32B parameter scales. Olmo 3 model construction targets long-context reasoning, function calling, coding, instruction following, general chat, and knowledge recall. This release includes the entire model flow, i.e., the full lifecycle of the family of models, including every stage, checkpoint, data point, and dependency used to build it. Our flagship model, Olmo 3 32B Think, is the strongest fully-open thinking model released to-date.

Projects & Open Source

  • greyhound

    I've been using greyhound as a place to build and experiment with CuTe DSL kernels for PyTorch, with an eye toward making training and inference workloads faster and easier to hack on. So far it includes kernels for cross-entropy and chunked linear losses, selective log-softmax, and causal Conv1D; see the greyhound documentation and benchmarks.

  • olmo-core

    Building blocks for OLMo modeling and distributed training. I did substantial work in OLMo-core on pretraining and model development, including MFU improvements, scaling ladders, and the training infrastructure used for OLMo 3 and OLMo Hybrid.

  • Liger-Kernel

    I contribute to Liger-Kernel, a collection of custom Triton kernels for efficient LLM training. I've found these kernels very useful for training LLMs/VLMs on my RTX 4090. My contributions, as well as those of other top collaborators, were featured in LinkedIn Engineering's overview of the Liger-Kernel ecosystem.

  • microR1

    A micro-scale DeepSeek-R1 reproduction in the style of Karpathy's nanoGPT. Intended to be easy to understand and to hack on top of.

Favorite Reads

Fun Stuff