Senior/Principal Performance Engineer

<div class="content-intro"><p style="text-align: justify;"><span style="font-size: 14pt; font-family: helvetica, arial, sans-serif;"><strong>CIQ OVERVIEW</strong></span></p> <p style="text-align: justify;"><span style="font-size: 12pt;">CIQ builds the enterprise infrastructure that powers the world's most demanding workloads. From the operating system layer through AI infrastructure, high-performance computing, and cloud-native orchestration, CIQ delivers the speed, security, scalability, and sovereignty that major enterprises, government agencies, and research institutions depend on.</span></p> <p style="text-align: justify;"><span style="font-size: 12pt;">CIQ is the founding support and services partner of Rocky Linux and the developer of the RLC Pro family of Enterprise Linux distributions, Fuzzball workload orchestration, Warewulf Pro cluster provisioning, and Ascender Pro automation. Our customers include some of the largest and most technically sophisticated organizations in the world, working across HPC, AI/ML, defense, and regulated industries.</span></p> <p style="text-align: justify;"><span style="font-size: 12pt;">We are a company of builders, operators, and open source practitioners. If you want to do work that matters, at a company that is genuinely changing how enterprise infrastructure gets built and run, we want to talk.</span></p></div><p style="text-align: justify;"><span style="font-size: 12pt;">CIQ is seeking a highly experienced Senior or Principal Performance Engineer to own and drive system-level and application-level performance across our product portfolio. This is an AI-first role, both in methodology and focus area,&nbsp; and the right candidate will bring deep expertise in operating system internals, kernel and userspace performance, and the performance demands of modern AI workloads, HPC environments, general-purpose computing, and production service workloads.</span></p> <p style="text-align: justify;"><span style="font-size: 12pt;">In this role, you will be the standard-bearer for performance at CIQ. Our performance-focused solutions must always be the fastest in the industry, and you will be responsible for ensuring that remains true. You will be intimately involved with Fuzzball, CIQ's cloud-native computing platform, learning its architecture end-to-end and integrating workloads - both user-facing and CI/testing pipelines - directly through it.</span></p> <p style="text-align: justify;"><strong><span style="font-size: 14pt;">Position Summary</span></strong></p> <p style="text-align: justify;"><em><span style="font-size: 12pt;">This role is leveled as Senior or Principal based on qualifications and demonstrated capabilities.</span></em></p> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>Benchmarking &amp; Profiling</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Design, develop, and maintain comprehensive benchmarking frameworks spanning OS, kernel, and application layers.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Profile workloads across CPU, memory, I/O, network, and accelerator (GPU/NPU) subsystems to identify bottlenecks and optimization opportunities.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Establish and own performance baselines across CIQ's product and solutions portfolio.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Leverage AI-assisted tooling and agentic workflows to accelerate profiling, analysis, and root cause identification.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>Regression Detection &amp; Resolution</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Build and maintain automated performance regression-detection pipelines integrated into CI/CD workflows using Fuzzball.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Identify, triage, and resolve regressions across user space, kernel space, and application layers with urgency and rigor.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Collaborate across engineering teams to root-cause regressions introduced by upstream kernel changes, compiler updates, or library modifications.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>Proactive Performance Engineering</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Drive proactive performance improvements - not just reactive fixes - to keep CIQ solutions ahead of the competition across every layer of the stack.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Own core operating system performance: kernel subsystem tuning (scheduler, memory management, I/O, networking), system call overhead reduction, and user space library and runtime optimizations.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Identify and implement kernel-level enhancements, including patches, configuration changes, and upstream contributions that yield measurable performance gains for CIQ's customer workloads.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Optimize for AI inference and training workloads, including LLM serving, model parallelism, and accelerator utilization.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Tune performance for HPC workloads, including modeling, simulation, and tightly coupled parallel applications (MPI, OpenMP, etc.).</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Optimize general computing and service workloads - web services, databases, messaging systems, and other production software that runs on CIQ's OS platform.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Work at all levels of the stack: compiler flags, kernel parameters, scheduler tuning, NUMA topology, memory allocation, and application-level algorithmic improvements.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>AI-First Approach</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Champion an AI-first engineering philosophy - use AI tools, agents, and automation to accelerate your own productivity and the quality of performance insights.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Identify and prioritize optimization opportunities that directly impact AI training throughput and inference latency/cost.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Stay current on state-of-the-art techniques in ML system performance, including quantization, batching strategies, kernel fusion, and hardware-software co-design.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>Fuzzball Integration</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Develop deep expertise in CIQ's Fuzzball platform - its architecture, scheduling, and workload execution model.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Integrate performance benchmarks, regression tests, and user-facing workloads into Fuzzball-based pipelines.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Contribute to the performance characterization of Fuzzball itself, ensuring the platform adds minimal overhead and scales efficiently.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 12pt;"><strong><em>Cross-Functional Collaboration</em></strong></span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Develop broad familiarity with the full CIQ product portfolio — including Rocky Linux and RLC (and its variants), Fuzzball, Apptainer (formerly Singularity), and Warewulf - understanding how performance considerations span and interconnect across each.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Collaborate deeply with the engineering teams behind each product line to surface, prioritize, and deliver performance improvements that benefit customers across the entire CIQ ecosystem.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Partner with product and customer success teams to translate real-world performance pain points into engineering priorities and measurable outcomes.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Document and communicate findings clearly - from low-level profiling data to executive-level summaries.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Contribute to technical publications, conference presentations, and thought leadership that reinforces CIQ's reputation for performance excellence.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 14pt;"><strong>NEEDED TO SUCCEED</strong></span></p> <p style="text-align: justify;"><span style="font-size: 12pt;">Successful candidates will have:</span></p> <ul style="text-align: justify;"> <li style="font-size: 12pt;"><span style="font-size: 12pt;">A deep, principled understanding of operating system internals -&nbsp; Linux kernel scheduler, memory subsystem, I/O stack, and networking.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Proven experience identifying and resolving performance regressions across kernel and user space in production environments.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Hands-on expertise with profiling and tracing tools: perf, eBPF/bpftrace, Flamegraphs, VTune, Nsight, strace, ftrace, and similar.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Strong background in AI/ML workload performance - including inference optimization (TensorRT, ONNX, vLLM, or similar), training efficiency, and GPU/accelerator utilization.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Experience with HPC workloads: MPI, OpenMP, parallel filesystems, RDMA/InfiniBand, and job schedulers (Slurm, PBS, etc.).</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Familiarity with modern AI-first development workflows and comfort using LLM-based tools to accelerate engineering work.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Experience building automated performance testing and regression detection pipelines in CI/CD environments.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Excellent analytical skills -&nbsp; able to form hypotheses, design experiments, and draw actionable conclusions from complex data.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">Strong written and verbal communication skills; able to present findings to both deeply technical audiences and business stakeholders.</span></li> <li style="font-size: 12pt;"><span style="font-size: 12pt;">A collaborative, humble, and always-learning mindset -&nbsp; combined with the confidence to champion performance as a first-class engineering concern.</span></li> </ul> <p style="text-align: justify;"><span style="font-size: 14pt;"><strong>EDUCATION AND EXPERIENCE</strong></span></p> <ul> <li style="text-align: justify; font-size: 12pt;"><span style="font-size: 12pt;">PhD in Computer Science, Computer Engineering, or a related field strongly preferred; equivalent industry experience considered.</span></li> <li style="text-align: justify; font-size: 12pt;"><span style="font-size: 12pt;">15+ years of industry experience in systems performance engineering, OS development, or a closely related discipline.</span></li> <li style="text-align: justify; font-size: 12pt;"><span style="font-size: 12pt;">Demonstrated track record of measurable, published, or production-deployed performance improvements at scale.</span></li> <li style="text-align: justify; font-size: 12pt;"><span style="font-size: 12pt;">Experience working in or with open-source ecosystems (Linux kernel contributions, upstream community engagement) is a strong plus.</span></li> <li style="text-align: justify; font-size: 12pt;"><span style="font-size: 12pt;">Background with cloud-native, containerized, and/or HPC computing environments preferred.</span></li> </ul><div class="content-conclusion"><p style="text-align: justify;"><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"><strong>BENEFITS</strong></span></p> <div class="form-field"> <div class="form-field__input-wrapper"> <div class="textarea-box"> <div class="read-only-redactor ng-star-inserted"> <ul> <li style="font-family: helvetica, arial, sans-serif; text-align: justify; font-size: 12pt;"> <p style="line-height: 1;"><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Medical, dental, and vision insurance.</span></p> </li> <li style="line-height: 1; font-family: helvetica, arial, sans-serif; text-align: justify; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Flexible paid time off.</span></p> </li> <li style="line-height: 1; font-family: helvetica, arial, sans-serif; text-align: justify; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Employee stock options.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p style="line-height: 1; text-align: justify;"><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Remote work; no travel required for most positions.</span></p> </li> </ul> </div> </div> </div> </div> <div class="form-field">&nbsp;</div></div>