text-generation-ui

project page

text-generation-ui is a python gradio package that provides a web ui for LLM and, more recently, LMM inference. It can be used with any GGUF-formatted models.

replace None Bug

Getting the following error when trying to use text-generation-ui:

File "text-generation-webui/extensions/openai/script.py", line 137, in openai_chat_completions
    response = OAIcompletions.chat_completions(to_dict(request_data), is_legacy=is_legacy)
  File "text-generation-webui/extensions/openai/completions.py", line 536, in chat_completions
    return deque(generator, maxlen=1).pop()
  File "text-generation-webui/extensions/openai/completions.py", line 315, in chat_completions_common
    prompt = generate_chat_prompt(user_input, generate_params)
  File "text-generation-webui/modules/chat.py", line 97, in generate_chat_prompt
    user_bio=replace_character_names(state['user_bio'], state['name1'], state['name2']),
  File "text-generation-webui/modules/chat.py", line 636, in replace_character_names
    text = text.replace('{{user}}', name1).replace('{{char}}', name2)
AttributeError: 'NoneType' object has no attribute 'replace'

see solution here: