LanguageModelContinuousBatching

Language model that uses continuous batching.

PyTorchPaxML / Praxis

PyTorch

API: —

Strategy: Custom / Partial

PaxML / Praxis

API: paxml.layers.LanguageModelContinuousBatching

Strategy: Direct Mapping

Official Docs ↗