This prototype retrieves passages from a medical reference document and supplies them to a pretrained, quantized Llama 2 model. It is an engineering experiment in retrieval and interface design. It has not been clinically validated and is not intended for medical decision-making.

Image description

Try it for FREE:huggingface.co/Llama-2-GGML-Medical-Chatbot

Image description

Here are some of the features of the Llama-2-7B-Chat-GGML-Medical-Chatbot:

  • It uses the Llama-2-7B-Chat-GGML model, which is a large language model (LLM) that has been fine-tuned.
    • Name - llama-2-7b-chat.ggmlv3.q2_K.bin
    • Quant method - q2_K
    • Bits - 2
    • Size - 2.87 GB
    • Max RAM required - 5.37 GB
    • Use case - New k-quant method. Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors.
    • Model: Know more about model Llama-2-7B-Chat-GGML
  • It retrieves reference passages from the PDF The Gale Encyclopedia of Medicine, Volume 1, 2nd Edition, 637-page PDF, which is a comprehensive medical reference that provides information on a wide range of medical topics. This means that the chatbot is able to answer questions about a variety of medical topics.
  • This is a sophisticated medical chatbot, developed using Llama-2 7B and Sentence Transformers. Powered by Langchain and Chainlit, This bot operates on a powerful CPU computer that boasts a minimum of
    • Operating system: Linux, macOS, or Windows
    • CPU: Intel Core i3
    • RAM: 8 GB
    • Disk space: 7 GB
    • GPU: None (CPU only)
  • It is still under development, and has not been validated for clinical use.

Image description

Quickstart

  1. Open Git Bash.
  2. Change the current working directory to the location where you want the cloned directory.
  3. Type git clone, and then paste the URL you copied earlier. 
    git clone https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot.git

    Press Enter to create your local clone.

  4. Install the pip packages in requirements.txt 
    pip install -r requirements.txt
  5. Now run it! 
    chainlit run model.py -w

Image description

Conversation examples

Chainlit ver. on#v1.0.1.dev20230913

Image description

Streamlit ver. on#v2.0.1.dev20231230

ChatBot Conversession img-4

Video of Conversation

Video of Conversation

Long Chats

ChatBot Conversession img-2

ChatBot Conversession img-3

PDF

Conversation examples img-3.pdf

Image description

Important Notes

  • While powerful, this chatbot isn't a substitute for professional medical advice.