diff --git a/README.md b/README.md index 2ebd4b7..0696371 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,16 @@ Our model hasn't been fine-tuned through reinforcement learning from human feedb ## Intended Uses +Below are example codes to load phi-2, we support two modes of execution for the model: + 1. loading in fp-16 format with flash-attention support: + ```python + model = AutoModelForCausalLM.from_pretrained('microsoft/phi-2', torch_dtype='auto', flash_attn=True, flash_rotary=True, fused_dense=True, trust_remote_code=True) + ``` + 2. loading in fp-16 without flash-attention + ```python + model = AutoModelForCausalLM.from_pretrained('microsoft/phi-2', torch_dtype='auto', trust_remote_code=True) + ``` + Phi-2 is intended for research purposes only. Given the nature of the training data, the phi-2 model is best suited for prompts using the QA format, the chat format, and the code format. #### QA format: