Job Description

Summary

Seeking an experienced embedded /software engineer to design and optimize low-level compute kernels for Large Language Model (LLM) inference and training on custom RISC-V based silicon. You'll write high-performance, CUDA-like code that directly interfaces with custom hardware accelerators, maximizing throughput and minimizing latency for next-generation AI workloads. This role bridges the gap between ML algorithms and hardware capabilities, requiring deep systems thinking and hands-on optimization skills.

Core Responsibilities

• Kernel Development: Write optimized compute kernels in CUDA-like code, HIP, or proprietary ISAs that implement LLM operations (matrix multiplication, attention, normalization, quantization).
• Performance Optimization: Profile, benchmark, and optimize kernels for target hardware; identify and eliminate bottlenecks through memory hierarchy analysis and instruction-level tuning.
• Custom Hardware Integration: Collaborate with hardware design teams to understand hardware capabilities and constraints; optimize code for novel architectures and emerging hardware features.
• Algorithm Implementation: Translate cutting-edge LLM techniques (flash attention, quantized inference, mixture-of-experts) into efficient kernel implementations.
• Documentation & Knowledge Sharing: Maintain clear technical documentation of kernel designs and share optimization insights across the team.
• Testing & Validation: Develop rigorous testing frameworks; validate correctness and performance across diverse LLM models and hardware configurations.
• Production Readiness: Ensure kernels are stable, maintainable, and production-grade; handle edge cases and numerical precision requirements.

Preferred Qualifications

• Background in computer architecture or computing hardware design
• Familiarity with LLM inference frameworks (vLLM, TensorRT, Triton, etc.)
• Experience with heterogeneous computing environments (CPU + GPU + specialized accelerators)
• Understanding of compiler internals and code generation strategies, knowledge of compiler optimization and ISA design principles
• Hardware design background: RTL, hardware description languages (Verilog/SystemVerilog), or close collaboration with silicon design teams, or experience with custom ASICs or FPGAs for ML acceleration
• Published research or papers in systems optimization, numerical computing, or hardware-software co-design
• Multi-GPU/multi-accelerator optimization: distributed training, all-reduce operations, tensor parallelism
• Experience with quantization, sparsity, or other LLM optimization techniques at the kernel level

Core Technical Skills

• Languages: C/C++, CUDA/HIP, Assembly (RISC-V or custom ISA)
• Hardware Knowledge: GPU memory model, shared memory optimization, bank conflicts, warp divergence, thread coalescing
• Math: Linear algebra, matrix factorization, fast algorithms (FFT, polynomial operations)
• Tools: Performance profilers, simulation tools, hardware emulators