The TSMC of AI Factories, Fireworks

TL;DR

Fireworks is an open-model and first-party-model Specialized Intelligence platform founded in 2022 by former Meta (including the PyTorch core founding team) / Google engineers, headquartered in San Mateo, California.

Its stack, from the bottom up, is six modules: Infrastructure (Cloud + Sovereign) → Training → Fireworks Lab → Models (in-house / open library / customer weights) → Inference → Model ensembling (Nexus + API).

Fireworks discloses that of the 40 trillion+ (40T+) tokens it processes each day, more than 95% are not direct calls to public open weights, but come from proprietary models fine-tuned, reinforcement-learned (RL), or parameter-optimized on customers’ private data. If 90% of a platform’s traffic is running public Llama or DeepSeek, it is a public-compute wholesaler; when 95% of output is proprietary models custom-processed for different customers, it has the attributes of TSMC (Foundry).

Figure 1. Token mix at Series D. Company disclosure, July 2026.

Team and development

Funding history

Series C snapshot (2025-10): disclosed 10,000+ companies served; > 10T tokens/day; ARR > $280M. Series D snapshot (2026-07): latest disclosed figures already > 40T tokens/day; ARR > $1B+; > 95% of tokens from models specialized on customer proprietary data or optimized for a specific job.

Founding team and core stable team

Seven co-founders: Lin Qiao, Dmytro Dzhulgakov, Benny Chen, Chenyu Zhao, Dmytro Ivchenko, James Reed, Pawel Garbacki. About half came from the original PyTorch team.

Lin Qiao — Co-Founder, CEO

• Education: Fudan University, B.S. and M.S. in computer science; Ph.D. in computer science, University of California, Santa Barbara (UCSB).
• Career: IBM from about 2003 (including Almaden Research’s BLU analytic query engine); 2011–2015 LinkedIn, Tech Lead / Manager, on the Espresso distributed data platform, Gobblin ingestion, and data lifecycle management; joined Meta in 2015-07, Data Infrastructure engineering manager, then Senior Director of Engineering from 2018-02 to 2022-09. The team grew from about 5 people to 300+, covering Caffe2, PyTorch, and Meta’s train/serve production stack. Public interview line: the team kept the PyTorch front end and rebuilt the back end (including TorchScript); what was expected to be a 6-month library migration became about 5 years of train-serve stack reconstruction, with a peak narrative of more than 5 trillion inferences a day.

Dmytro “Dima” Dzhulgakov — Co-Founder, CTO

• Education / background: B.S. and M.S. in applied mathematics, NTU Kharkiv Polytechnic Institute; Ukrainian. Earlier founded QBit, programming education for teenagers, and built an industrial-control IDE at NPF Informatica. Interned at Google Zurich and Facebook. His X bio: “Ex-Pro Competitive Programmer.”
• Meta (about 2011-12 to 2022-09): Ads personalization and large-scale event prediction; core developer of Caffe2; co-initiator of Facebook’s first company-wide production AI platform; PyTorch core maintainer, pushing research (FAIR) and product toolchains together; ONNX; co-author of the 2019 Deep Learning Recommendation Model (DLRM) paper. Third-party résumé mirrors say a 400+ person engineering org in a Tech Executive phase (Med).

Benny Chen (Benny Yufei Chen) — Co-Founder

• Education / background: Born in New Zealand. Open Source CEO interview and MindMakers podcast: UCLA computer science / Computer Science and Engineering (about 2015).
• Meta (about 8–9 years, intern to Principal): Ads infrastructure lead. In multiple interviews: responsible for ads-model inference on GPU and ASIC (self-described moving ads models onto ASIC from 2017, still a V100 world); later did capacity planning for ads infrastructure, saw internal GPU demand go “up and to the right,” the direct signal to leave Meta and start a company.

Chenyu Zhao — Co-Founder (the only Google hire)

• Education: B.S. in Electrical Engineering and Computer Science, UC Berkeley (Equilar).
• Google (about a decade): Official /team title: Google Vertex AI Lead. Slush 2026 speaker bio: senior engineering manager, Google Cloud AI platform. Third-party sketch: helped build Vertex AI, managed an ~50-person MLOps / AutoML engineering team (Med). In a public AMD × Crusoe conversation he said he came from GCP.

Dmytro Ivchenko — Co-Founder, Engineer

• Education / background: M.S. in systems analysis, Igor Sikorsky Kyiv Polytechnic Institute; with Dzhulgakov, one of the two Ukrainian co-founders named in press.
• Career: Software development at Bloomberg and FatWire; LinkedIn Senior Staff, led implementation of the Galene search engine and worked on Recruiter; from 2016 at Meta, Staff / Senior Staff / Principal. Official /team: PyTorch for ranking lead.

James Reed (James K. Reed) — Co-Founder

• Education: B.S. in computer engineering, Virginia Tech, Summa Cum Laude. Interned at Google and Qualcomm.
• Meta: PyTorch compiler. Staff Software Engineer; led or was deep in torch.fx, PiPPy (Pipeline Parallelism for PyTorch, later merged into PyTorch core), TorchScript, ONNX. MLSys 2022 paper torch.fx: Practical Program Capture and Transformation for Deep Learning in Python (with Zach DeVito, Jason Ansel, and others). Patents: predication in computational models (USPTO 10,553,207 and others).

Pawel Garbacki (Paweł Garbacki) — Co-Founder, Researcher

• Education / background: Born in Warsaw. B.S./M.S. in mathematics and computer science, University of Warsaw; M.S. in computer science, VU Amsterdam; Ph.D. in computer science, Delft University of Technology + VU Amsterdam (advisor Dick Epema, OpenReview self-report 2003–2008). Early papers in P2P / Tribler systems.
• Career: Research intern, IBM T.J. Watson; Google Staff / Tech Lead; 2013–2016 Pinterest, founding member of the Ads org; 2016-10 to 2022-11 Meta Principal Engineer, News Feed Core Machine Learning.

Core product lines and business model

Six core business modules

The company’s own positioning: not a closed Frontier Lab; an operating system for enterprise-owned Specialized Intelligence — open inference + fine-tune/RL + orchestration, with cost and control aimed at closed APIs.

Module 1: Infrastructure

Base compute is not tightly bound to Fireworks’ own cloud capacity.

• Fireworks Cloud (global managed GPU pool): 30+ regions, latest NVIDIA + AMD cards, Foundry first-class citizen can offset Azure commit, ZDR / regional residency / SOC 2 Type II / ISO 27001/42001
• Fireworks Sovereign (Fireworks on your cloud): inference stays on the customer’s K8s / cloud / data center; same API, no code change; can overflow to Cloud.

Fireworks Cloud (global managed GPU pool)

A Virtual Cloud leased across 20+ suppliers. Training is Available in only 5 single regions (Virginia / Ohio / Minnesota / Malaysia / Taiwan). A100 remains in Illinois; three AMD zones have no public hourly price. Supply sources: Microsoft Foundry (counts toward Azure MACC); AWS Marketplace / SageMaker / Bedrock AgentCore; Google Cloud Marketplace.

Fireworks Sovereign (Enterprise Private Preview)

The same serving engine on customer-owned / already-contracted GPUs; the customer keeps architecture and policy; Fireworks does qualification, deploy, upgrade, monitoring, and incident response.

Sovereign equals managed BYOC; Airgap is the harder isolation SKU.

Module 2: Training

Enterprises used to call giant closed general models such as Claude Opus or GPT-5/6 directly. Now they use Fireworks to fine-tune / RL a mid-size open model (such as 30B Muse Glimmer or GLM 5.3 Flash) on a single in-house job and put it into production: smaller, more specialized, inference cost falls off a cliff, latency improves. Because training and inference sit on the same platform (no exporting hundreds of GB of weights onto a second inference cluster), iteration that took weeks shortens to hours.

• Cursor (Federico Cassano): the resource-allocation strategy for AI-native companies — scarce top algorithm engineers should spend their time writing code sandbox environments (Environment) and reward functions (Reward), not CUDA inference kernels and GPU clusters.
• Vercel (Malte Ubl): why Compound AI — the front-end generation tool v0 is not one giant model doing everything, but a router + a codegen model + a small Auto-fix model together; base models turn over every week, and with Fireworks a training Recipe can move onto the latest SOTA open base at any time.
• Genspark (Kay Zhu): from renting an API to owning the weights can close the loop in 4 weeks.

Any training (especially RL) has two motions — ① Rollout / Sample (run trajectories: the model answers or calls tools as if solving problems) and ② Train (update: score, take gradients, update weights). Fireworks offers four training paths by who writes the control Loop and how GPUs are used.

Training results

Module 3: Fireworks Lab

Fireworks Lab is a high-ARPU, low-GPU-COGS embedding used to lock a large customer’s core budget and leave evaluators and training loops behind. GPUs still bill on the public SKUs of Module 2 Training / Module 5 Inference; Lab bills researcher time.

Project delivery: the customer keeps the model, eval harness, data pipes, training loop, and recipes — when the base gets stronger, the standard moves with it. High-ARPU, low-GPU-COGS embedding used to lock a large customer’s core budget and leave evaluators and training loops behind.

• Cycle about 6–10 weeks, a time-boxed build counted backward from a target date. (Genspark Deep Research, Heidi clinical scribe).
• Opening diagnosis: capability, baseline, success criteria, scope.
• Work packages (Training page “We build with you”):
  • Data readiness: rights, volume, whether it is fit for production; confirm “we can win before we spend more.”
  • Eval design: write evals with the customer’s experts; put own models and other models on the same ruler.
  • Training strategy: base, method, serving economics, retrain on new traces / a new base.
• Shares the same train-serve stack as Module 2 Training; Lab sells researcher time and method, not a GPU SKU. When algorithm staff is missing, Lab is the service layer on top of Managed / Training API.

Module 4: Models

Model supply is three lanes:

• Fireworks models (in-house)
• Model library (open labs)
• Your models (customer weights)

Fireworks models (in-house)

Ember-1 is a short-reasoning specialist built on Kimi K3. Motive: K3 sometimes spends >90% of generated tokens on internal reasoning; multi-turn agents feed early reasoning back in, so context grows roughly quadratically; turning K3 effort down costs quality. The team ran 50+ Serverless Training experiments and 200+ evals, without customer data.

External line: about 40% fewer tokens, quality close to K3; on five industry benchmarks about 1 point from Claude Opus 5, about 1/5 the cost per task (marketing / internal eval). On Doximity Bedside Bench (500 clinical cases / 10 specialties) it claims a cost/task Pareto frontier versus GPT-5.6 Sol, GPT-6 Astra, and Claude Opus 5.

Public comparison table (company self-test)

Two customer production coding A/Bs: about -35% tokens / task, quality flat; one already cut over. Internal traffic: developers did not notice the swap. An internal table also shows reasoning tokens -71.3%, output 49.3K → 29.9K, score 0.751 → 0.753. Currently a Research Preview: new research models get two weeks of Serverless, then graduate on community demand. Ember-1 training is open; customers can specialize further. Serverless and Dedicated both serve it.

Model library

Open-model picks by scene

Closed-model supply versus open substitutes

Your models (customer weights)

• Training output is 100% the customer’s, subject to the base license; firectl model download exports Hugging Face PEFT; compatible externally trained adapters / custom weights can also be uploaded.
• A single LoRA can live-merge into the base, inference speed close to the base; Multi-LoRA about 100 addons share one Dedicated deploy.
• Specialized Model Inference (Public Preview, Forge 2026): hang LoRAs on shared Serverless, billed at the base token price, no idle GPU. Currently only GLM 5.3 Flash (accounts/fireworks/models/glm-5p3-flash), at most 3 adapters per base, rank ≤ 64; a fourth returns ResourceExhausted. Flow: Train/import → Load < 5 minutes → first request < 1 minute leaves the LoRA in cache → thereafter near-base latency. Requires firectl ≥ 1.8.15. Other bases still need Dedicated. <a href="https://docs.fireworks.ai/serverless/specialized-model-inference/overview.md">https://docs.fireworks.ai/serverless/specialized-model-inference/overview.md</a>

Pricing

Module 5: Inference

Fireworks inference products, as customer traffic grows, are three ramps: Serverless (token carpool) → Reserved Throughput (buy dollars-per-minute of spend to hold SLA) → Dedicated / GPU Reserve (rent whole cards by the second / by the year).

• Serverless: multi-tenant shared pool, no cold start, no GPU picker. OpenAI- and Anthropic-compatible, billed by token.
• Reserved Throughput (Serverless SLA): buy dollars/minute of quota, use-it-or-lose-it by the minute, does not accumulate. Pricing page writes throughput SLA + 99.9% uptime terms (docs write only throughput SLA; the full credits language is not public).
• GPU Reserve / Reserved capacity: enterprise-typical 1 year; guaranteed capacity, higher quotas, lower GPU-hour, Pre-GA regions / new hardware.

Module 6: Model ensembling (Nexus + API)

Two faces: the human tool (Fireworks Nexus) and the system API (Fireworks API). Product-page line: keep ChatGPT / Claude / the existing coding harness; everyday turns go open, hardest turns stay closed; claimed team bill −50%+; API side “change one base URL,” OpenAI / Anthropic compatible. Nexus FAQ is explicit: it is not merely an LLM router — CLI plus tools, Auto per turn, Settings with a budget; the three together make a “multi-model organization.”

Nexus Auto

Closed + open together through Nexus: so developers can still use Claude Opus and GPT inside Claude Code, Cursor, or OpenCode, Fireworks uses a client/gateway pass-through in Nexus (formerly FireRouter) — you put in your own Anthropic/OpenAI Key; hard turns go straight to official Anthropic/OpenAI servers; routine turns route to open models or Ember-1 / Qwen Plus in Fireworks’ own rooms. That keeps a compute supplier on the right side of closed-model terms and cuts the customer’s closed-model bill by more than 50%.

Fireworks API (system face)

Fireworks API (system face) is essentially the Serverless Inference API already unpacked above (plus the matching Routers / deploy-management interfaces).

Versus OpenRouter

On the surface, Fireworks Nexus and OpenRouter both sit in “unified multi-model access and intelligent routing (Model Routing),” and both can plug into Cursor, Claude Code, OpenCode; their underlying business models, routing logic, and place in the chain are different — and on the compute supply chain, Fireworks itself is one of OpenRouter’s most important upstream open-model inference suppliers.

• OpenRouter is a “neutral model aggregation bazaar (horizontal B2D Marketplace)”: the job is “find a model across vendors, compare price, fail over.” It does not run a fleet; it makes a unified gateway over all models and takes top-up fees or wholesale spread.
• Fireworks Nexus is an “agent cost-cut add-on bound to its own inference engine (vertical integration funnel)”: the job is “without changing how engineers use Claude Code / Cursor / OpenCode, offload 70% of routine tasks from expensive closed models onto Fireworks’ own open models, per turn,” in order to fill its own high-margin Serverless/Dedicated engines.
  • When a developer runs nexus claude or nexus codex, the tool still keeps the original Claude login or the developer’s own OpenAI/Anthropic API Key (keys stay in the local keychain, not uploaded)
  • The enterprise does not have to throw away existing Anthropic/OpenAI contracts or ChatGPT/Claude desktop subscriptions, and can stretch a full-year AI budget that would otherwise burn in 4 months (the Uber engineering pain in public write-ups) by 2 to 3×.

Core engineering moat

Early open-model clouds generally rented bare-metal GPUs and wrapped open-source vLLM/SGLang, and slid into a homogenized price war. Fireworks’ claim to hold gross margin and lock in 95%+ custom-model traffic is that it closed three industrial gaps in getting large models into production:

1. Close the “train–serve numerical skew” gap: In post-training and RL of giant MoE models (Kimi K3, DeepSeek V4.1), the traditional path trains in Megatron/FSDP at BF16, then exports to vLLM at FP8/FP4. Operator differences, low-precision rounding, and the MoE gating router activating different experts under tiny numerical noise produce “reward curve up in training, production accuracy collapse after serve.” Fireworks built a train-serve-identical kernel stack (FireAttention), natively aligned to BF16, block FP8, and NVIDIA’s latest NVFP4, and MoE Router Replay, forcing backprop and sampling onto the same expert path, holding serve-vs-train KL at KLD < 0.007 (token reject rate < 3%).
2. Close the “async RL compute idle” gap: In 2026 the mainstream enterprise custom-model paradigm has moved from static SFT to Agentic RL / RFT. In the RL loop, 70%–85% of GPU time is Rollout (sampling huge Trajectory sets in code or tool environments), not the gradient step. If train and serve clusters are split, syncing a full multi-hundred-GB Checkpoint each time leaves GPUs waiting. Fireworks RL Rollouts and Serverless Training API use an async architecture; Delta-compressed Weight Updates send about 2% of the weight change each time, with background pipeline prefetch and decode, for sub-minute (< 1 minute) GPU-memory hot-swap, stated at 10,000+ GPU concurrent RL.
3. Close the “multi-turn agent long-context cost blow-up” gap (Session Affinity + Nexus): A coding-agent job often spans tens of turns, context swelling to 100K–1M tokens. Fireworks implements Session Affinity at the cluster scheduler, pinning consecutive requests of one Agent session onto the physical node that already holds that session’s KV Cache, stated hit rate up to 99.6% (no cache-write fee; reads default half price or lower). On the terminal, Nexus as a “task-difficulty and cache-aware router” dumps file search, ordinary rewrites, and unit-test generation — more than 70% of daily development — onto Fireworks’ fast open models, and only the hardest architectural reasoning pass-through to a closed-model API, cutting the team coding-AI bill 50%–72%.

Why Fireworks？

Versus TSMC

The semiconductor industry’s move from IDM monopoly to Fabless (no wafer design) + Foundry (specialized manufacturing).

• Early semiconductor IDM ↔ closed Frontier Lab (OpenAI / Anthropic): early on there was only an IDM giant like Intel, designing the architecture, building the fab, selling Intel-branded standard parts. That maps to today’s OpenAI and Anthropic — pretrain a closed general model, serve it on an internal cluster, sell a black-box API.
• Rise of Fabless ↔ 10,000+ vertical application companies that own private data: Apple, NVIDIA, AMD, Qualcomm did not each spend hundreds of billions on a fab; they drew the chip and sent it to TSMC. In the AI era, Cursor, Harvey, Notion, Uber, Vercel are the AI-era Fabless — they hold the core workflow, vertical private data, and RL reward functions (the “drawings” of a specialized model), but they do not want a closed API to lock margin and latency, and they will not staff a hundred-person CUDA-kernel and 10,000-GPU distributed-systems team to own a “token line.”
• Birth of Foundry ↔ Fireworks AI: Fireworks is the specialized foundry that takes those tens of thousands of “specialized-model drawings” and turns them into high-throughput, low-cost production token streams.

Engineering comparison

Versus vLLM, SGLang

If you only look at getting an open model running on a single box, continuous batching, and paged memory, Fireworks’ inference engine and vLLM (PagedAttention and general high throughput) and SGLang (RadixAttention prefix-tree reuse and structured JSON constrained decode) really are solving the same class of problem.

But vLLM/SGLang stop at pure inference, while Fireworks opened train-serve identity and online RL.