A Large Language Model (LLM) is a specific type of Machine Learning model that is particularly effective at Natural Language Processing. The term has come into prominence and wide use with the widespread adoption of models like ChatGPT which have 20 billion parameters1 in their neural network. However it is this author’s opinion that it’s perfectly reasonable to refer to models like BERT, boasting a paltry 340 million parameters 2 , large language models.
LLMs and Quantization
Large Language Models are particularly resource intensive and take many gigabytes of memory to run. Therefore,
Running LLMs
Tools like:
Evaluating LLMs
See Evaluating LLMs.
Monitoring LLMs
Tools like:
- https://ukgovernmentbeis.github.io/inspect_ai/
- lunary
- langfuse
Compute Requirements
See LLM Memory and Compute Requirements
LLMs vs LMMs
LLMs typically deal with input and output data that is encoded in natural language. More recently LMM models which can accept data in other modalities (for example images or audio) have begun to gain popularity. Quite often there isn’t much difference in architecture between these model types but instead the way that data is encoded and fed into the model is what makes the difference.