GenAI Systems Engineer

About Modular

About the role:

What you will do:

• Leverage a broad understanding of available libraries and concurrency techniques to inform high impact architecture decisions
• Identify and implement architecture-level optimizations in complex distributed systems
• Architect and implement building blocks and APIs to accelerate the development of advanced distributed optimizations
• Lead cross-functional projects spanning multiple teams and multiple layers of a deep tech stack
• Build beautiful abstractions to seamlessly weave async RESTful layers with intensive data processing layers
• Collaborate with cloud inference team to maximize flexibility in scalable cluster deployments
• Develop extensible customization interfaces to support open source community models and features
• Develop detailed and intuitive metrics, logging, and profiling tools

What you bring to the table:

• Expert-level Python programming with deep understanding of asyncio and event loops
• 5+ years of systems programming experience with focus on performance and concurrency
• Hands on experience with robust low-latency applications running production workloads
• Extensive experience designing software architecture, interfaces, and collaboration
• Deep understanding of the fundamentals of profiling, benchmarking, and performance optimization
• Creativity and curiosity for learning and solving complex distributed systems problems

 Helpful, but not required

• Experience working inside high-performance ML inference systems (e.g. vLLM, SGLang, etc.)
• Experience with Kubernetes, containers, microservices, and cloud-native architectures
• Experience with graph based (e.g. dataflow, actors) programming models and runtimes
• Experience with distributed runtimes such as Ray, Open MPI, Dask, Spark, etc

What Modular brings to the table:

• Amazing Team. We are a progressive and agile team with some of the industry’s best engineering and product leaders.
• World-class Benefits. In order to attract the best, we need to offer the best. Premier insurance plans, up to 5% 401k matching, flexible paid time off, and more are available to you! Please note that specific benefit packages may vary based on your location.
• Competitive Compensation. We offer very strong compensation packages, including stock options. We want people to be focused on their best work and believe in tailoring compensation plans to meet the needs of our workforce. 
• Team Building Events. We organize regular team onsites and local meetups in Los Altos, CA as well as different cities. Traveling 2-4 times a year is expected for all roles.