Engineering Remote full time Remote-eligible
About the role
We serve inference at $/token margins that don't tolerate sloppy stacks. You'll own the serving layer — vLLM, TensorRT-LLM, Triton — and the benchmarking discipline that keeps it honest.
You'll own
- Serving-stack selection per workload (continuous batching vs. static, KV cache strategy, paged attention).
- Quantization (FP8, AWQ, GPTQ) and the eval harness that proves the trade-offs.
- The InferenceBench-style benchmarks that compare our serving against the field.
You probably have
- Shipped at least one production inference stack on H100s or A100s.
- Read a recent vLLM PR for fun.
- Strong opinions about speculative decoding.
Role reference: YJ-00003