Staff/Principal Engineer – Core Engineering

<p><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>About TrueFoundry</strong></span><br><br>Every production AI system whether it's powering customer support, writing code, analyzing financial data, or diagnosing medical conditions needs the same foundational infrastructure.A way to route between models. A way to manage tools and integrate them securely. A way to orchestrate agents and enforce governance. A unified compute layer to run it all.</p> <p><strong>That infrastructure layer is being built right now.</strong></p> <p>We're TrueFoundry, and we're building it. We're looking for a Staff/Principal Engineer our Core Engineering to join the team.</p> <h2><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>The Problem We're Solving</strong></span></h2> <p>Companies are moving beyond simple chatbots to production agentic systems. These systems route between OpenAI, Anthropic, Google, and self-hosted models. They integrate dozens of tools via protocols like MCP. They orchestrate multi-agent workflows where agents coordinate with other agents.</p> <p>The infrastructure to support this doesn't exist yet. You can't just duct-tape together a few API calls and call it production-ready.</p> <p>You need a control plane that handles:</p> <ul> <li>Intelligent routing with observability, cost policies, and fallback logic</li> <li>Centralized tool and MCP server management with security and lifecycle controls</li> <li>Agent orchestration with governance and guardrails</li> <li>A unified compute layer to run self-hosted models, custom tools, and agents</li> </ul> <p>We've built two products to solve this:</p> <p><strong>AI Gateway</strong> is the control plane of five composable components (Prompts, LLM Gateway, MCP Gateway, Guardrails, Agent Gateway) that handle routing, orchestration, and governance.</p> <p><strong>AI Deploy</strong> is a compute layer of a Kubernetes-based platform that abstracts ML workloads as standard software primitives, so everything runs on unified infrastructure.</p> <p>We're Series A, backed by Intel Capital and Sequoia. Companies like CVS, Mastercard, Siemens, Paytm, Synopsys, and Zscaler run production AI workloads on our platform.</p> <p>We're looking for<strong> </strong>an <strong>Engineer</strong> who is passionate about scaling deep learning workloads, optimizing multi-GPU training, and shipping production-grade solutions.&nbsp;</p> <h3><strong>The Role: </strong>We are seeking a <strong>Staff / Principal Engineer</strong> to join our <strong>Core Engineering team</strong>.<br>You will:</h3> <ul> <li>Solve some of the most complex Engineering problems and drive it alongside a team of engineers &amp; ML researchers.</li> <li>Build a <strong>deep, holistic understanding</strong> of the TrueFoundry platform across all components and shape the product vision and implementation.</li> <li>Act as the <strong>technical face of engineering</strong> for customer-related discussions and escalations</li> <li>Guide and <strong>unblock engineers</strong> across projects in the US region</li> <li>Partner closely with our <strong>CTO and India-based engineering team</strong> to drive system design, architecture, and implementation of complex products</li> <li>Lead <strong>technical design</strong>, <strong>critical customer problem-solving</strong>, and <strong>platform scalability initiatives</strong> end-to-end</li> </ul> <p>This is a <strong>high-ownership</strong>, <strong>high-impact</strong> role designed for an engineer who loves combining <strong>world-class systems thinking</strong> with <strong>real-world execution</strong>.</p> <h3><strong>What You’ll Do:</strong></h3> <ul> <li>Develop deep expertise across <strong>TrueFoundry’s platform stack</strong>&nbsp; infrastructure, deployment systems, LLM/ML orchestration, observability, cost optimization, and more</li> <li>Drive the <strong>system architecture and design</strong> for complex, distributed, cloud-native systems</li> <li>Act as the <strong>technical point-of-contact</strong> for enterprise customer engineering needs and escalations</li> <li>Lead and participate in <strong>design reviews, code reviews, and critical incident responses</strong><strong><br></strong></li> <li>Collaborate closely with the <strong>CTO</strong> on architectural decisions, scaling strategies, and technical roadmap prioritization</li> <li>Guide and mentor <strong>US-based engineers</strong> across multiple initiatives, helping them deliver high-quality, scalable systems</li> <li>Identify and drive <strong>technical debt cleanup</strong>, <strong>performance improvements</strong>, and <strong>resilience upgrades</strong> across the platform</li> <li>Bring a <strong>product engineering mindset</strong>, ensuring that customer needs and feedback translate into scalable engineering solutions</li> </ul> <h3><strong>Who You Are:</strong></h3> <ul> <li>8+ years of <strong>strong backend/systems engineering</strong> experience at top technology companies or startups</li> <li>Deep expertise in <strong>distributed systems</strong>, <strong>cloud-native architectures</strong>, and <strong>scalable system design</strong><strong><br></strong></li> <li>Strong working knowledge of <strong>Kubernetes</strong>, <strong>containerized workloads</strong>, and <strong>infrastructure engineering</strong><strong><br></strong></li> <li>Practical experience building or deploying <strong>ML/GenAI applications</strong> (or closely working with ML/DS teams)</li> <li>Skilled in programming languages such as <strong>Python</strong>, <strong>Go</strong>, or <strong>typescript</strong><strong><br></strong></li> <li>Solid understanding of <strong>system observability</strong>, <strong>resiliency design</strong>, and <strong>SRE practices</strong><strong><br></strong></li> <li>Strong technical leadership and communication skills — able to work with both <strong>customers</strong> and <strong>engineering teams</strong><strong><br></strong></li> <li>Ability to <strong>think strategically</strong> while also executing hands-on when required</li> </ul> <h3>Bonus: Experience supporting enterprise deployments of <strong>AI/ML infrastructure</strong>, <strong>model training</strong>, or <strong>inference systems</strong><strong><br></strong></h3> <h3><strong>Why Join TrueFoundry?</strong></h3> <ul> <li>Work directly with <strong>ex-Facebook engineers</strong> and <strong>founders from IIT Kharagpur, UC Berkeley, and Y Combinator alumni</strong>.</li> <li>First-hand exposure to building and scaling a <strong>deep-tech startup</strong>—insights you’ll carry if you want to start your own one day.</li> <li>Be part of a <strong>fearlessly experimental culture</strong> focused on customer success and long-term impact.</li> </ul> <p>Flexible hours, learning credits, and the opportunity to work <strong>shoulder-to-shoulder with the co-founders</strong> (Abhishek &amp; Nikunj).</p>