GGUF is a model serialization format for Machine Learning models with built in quantization.
How Does GGUF Work?
There is an excellent guide to the format by Vicki Boykiss (mirror) that explains different file formats and the need for GGUF models.
GGUF is a model serialization format for Machine Learning models with built in quantization.
There is an excellent guide to the format by Vicki Boykiss (mirror) that explains different file formats and the need for GGUF models.