Member of Technical Staff - ML Systems & Inference

San Francisco, CA

Onsite

$200-$300K + Equity (experience + skills dependent)

They are hiring a Member of Technical Staff focused on ML systems and inference to design and build the systems that execute models end-to-end under real production constraints.

You’ll work across model architecture, inference runtimes, scheduling, memory management, and system performance to make inference faster, more predictable, and more cost-efficient at scale.

Responsibilities

Build and optimize end-to-end inference pipelines
Design inference runtimes that balance latency, throughput, and concurrency
Work on batching, queuing, scheduling, and tail-latency trade-offs
Manage KV cache allocation, reuse, placement, and eviction
Optimize prefill and decode paths, including attention and memory usage
Profile and debug performance across model, runtime, and system layers
Collaborate with compiler, kernel, networking, and distributed systems engineers

Requirements

Strong software engineering fundamentals
Experience with ML inference, model serving, or production ML systems
Strong understanding of latency, throughput, memory, and concurrency
Experience with Python and C++
Interest in working close to models, runtimes, and systems performance

Nice to Have

Experience with TensorRT-LLM, vLLM, Triton, or custom serving systems
Knowledge of modern model architectures and attention mechanisms
Experience with KV cache management, batching, scheduling, or GPU-backed systems
Background in distributed systems, compilers, kernels, or high-performance infrastructure

Why Join?

Backed by significant Series A funding
Already generating eight-figure revenue
Deployed with major enterprise and AI-native customers
Building core infrastructure for next-generation AI inference
Opportunity to work across the full AI stack: models, runtimes, compilers, kernels, orchestration, and hardware

Fresh Jobs. Delivered to Your Inbox.

Acceler8 Talent jobs

Machine Learning Engineer