llmops

homepage

DJL is a framework for doing deep learning in the java programming language.

They also provide general purpose language model inference (LMI) Docker containers for interacting with vllm: see LMI containers. One of the selling points of these containers is that they are optimised for