Applied ML Researcher

About Ironsite

The Role

What You'll Build

• Architect & Train Novel VLMs: Design, train, and iterate on general-purpose Vision-Language Models fine-tuned for "Construction Intelligence" using our massive, proprietary dataset of first-person video.
• Drive the Research Roadmap: Take a leading role in executing our research goals, including establishing baselines with state-of-the-art models and developing novel fine-tuning methodologies, long context architectures, and visual reasoning techniques.
• Build Scalable Pipelines: Develop and own the model training and evaluation pipelines, ensuring we can rapidly experiment, measure performance, and deploy models into production.
• Optimize for the Edge: Design and implement a hierarchical set of models for efficient, on-device inference on our wearable hardware as well as server-side inference. This includes developing lightweight, coarse classifiers for real-time analysis (e.g., safety event detection, info density classification), as well as heavy-weight server-side VLMs to deeply understand complex tasks.
• Define the Future of Spatial Intelligence: Spearhead the development and expansion of our "Construction Intelligence" benchmark, a comprehensive suite of tasks including video-question answering, temporal reasoning, activity recognition, and higher level data analysis reasoning across the construction site that will guide our research.
• Collaborate on System Design: Work closely with the hardware and data teams to explore model architectures such as a two-model system (lightweight segmenter, heavyweight insight extractor) and tool-use agents, to improve spatial understanding.

Technical Challenges You'll Solve

• Training large-scale models efficiently with limited compute budgets while maximizing performance
• Developing novel pre-training objectives that capture construction-specific knowledge and temporal reasoning
• Implementing efficient attention mechanisms and architectural innovations for long-context understanding of construction projects
• Designing evaluation metrics that measure real-world construction task performance beyond standard benchmarks
• Balancing model capability with deployment constraints for edge and mobile applications

What We're Looking For

• Fast learner eager to grow and expand their knowledge and capabilities, welcoming new challenges with a passion for their work
• Background in Computer Science, Machine Learning, AI, Robotics, Data Science or a related field with multiple relevant classes completed.
• Prior experience, internship, or personal project with hands-on experience designing and training deep learning models, particularly transformer-based architectures.
• Familiarity with one deep learning framework (e.g., PyTorch, TensorFlow, JAX).
• Proficiency in Python

Preferred Qualifications

• At least one publication in an AI/ML/CV conference.
• Experience doing research as part of a larger research lab or team
• Demonstrated experience with major deep learning frameworks (e.g., PyTorch, TensorFlow, JAX).
• Strong proficiency in Python and a solid foundation in software engineering principles.
• Experience working with and creating large-scale vision and/or language datasets.
• A strong interest in vision language models and the application of AI to solve real-world physical problems, including working with and understanding the day-to-day lives of construction workers.

Location & Compensation

• San Francisco Bay Area (on-site)
• Competitive salary and significant equity package
• Full benefits including health, dental, vision, and 401k +6% match
• Access to dedicated GPU compute resources for research and experimentation

Compensation