From e22cfa2e0986cef469975f9118cc72aacd23c321 Mon Sep 17 00:00:00 2001 From: Gustavo de Rosa Date: Wed, 13 Dec 2023 21:40:21 +0000 Subject: [PATCH] Update README.md (#2) - Update README.md (93093db3a799f04d6d2a4850749bc5fd6b896781) Co-authored-by: Mojan Javaheripi --- README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/README.md b/README.md index 2ebd4b7..0696371 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,16 @@ Our model hasn't been fine-tuned through reinforcement learning from human feedb ## Intended Uses +Below are example codes to load phi-2, we support two modes of execution for the model: + 1. loading in fp-16 format with flash-attention support: + ```python + model = AutoModelForCausalLM.from_pretrained('microsoft/phi-2', torch_dtype='auto', flash_attn=True, flash_rotary=True, fused_dense=True, trust_remote_code=True) + ``` + 2. loading in fp-16 without flash-attention + ```python + model = AutoModelForCausalLM.from_pretrained('microsoft/phi-2', torch_dtype='auto', trust_remote_code=True) + ``` + Phi-2 is intended for research purposes only. Given the nature of the training data, the phi-2 model is best suited for prompts using the QA format, the chat format, and the code format. #### QA format: