├── Llama2_CodeGen_Fine_Tuning_LLama_2.ipynb
└── README.md


/README.md:
--------------------------------------------------------------------------------
  1 | 
  2 | # LlaMa2-CodeGen
  3 | This model is  [**LlaMa2-7b**](https://huggingface.co/meta-llama/Llama-2-7b) which is fine-tuned on the  [**CodeSearchNet dataset**](https://github.com/github/CodeSearchNet) by using the method  [**QLoRA**](https://github.com/artidoro/qlora) with [PEFT](https://github.com/huggingface/peft) library.
  4 | 
  5 | # Model Trained on Google Colab Pro Using AutoTrain, PEFT and QLoRA
  6 | 
  7 | ## [Implementation code](https://colab.research.google.com/drive/18sAFC7msV0gJ24wn5gl41nU0QRynfLqG?usp=sharing)    [![Open in Colab][Colab Badge]][RDP Notebook]
  8 | 
  9 | 
 10 | # You can load the LlaMa2-CodeGen model on google colab.
 11 | 
 12 | 
 13 | 
 14 | 
 15 | 
 16 | ### Example 
 17 | ```py
 18 | 
 19 | 
 20 | import torch
 21 | from peft import PeftModel, PeftConfig
 22 | from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
 23 | 
 24 | peft_model_id = "AhmedSSoliman/Llama2-CodeGen-PEFT-QLoRA"
 25 | config = PeftConfig.from_pretrained(peft_model_id)
 26 | model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, trust_remote_code=True, return_dict=True, load_in_4bit=True, device_map='auto')
 27 | tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
 28 | 
 29 | # Load the Lora model
 30 | model = PeftModel.from_pretrained(model, peft_model_id)
 31 | 
 32 | 
 33 | 
 34 | 
 35 | def create_prompt(instruction):
 36 |   system = "You are using the Llam2-CodeGen model, a coding assistant that will help the user to resolve the following instruction:\n"
 37 |   instruction = "### Input: " + instruction
 38 |   return system + "\n" + instruction + "\n\n" + "### Response:" + "\n"
 39 | 
 40 | def generate(
 41 |         instruction,
 42 |         max_new_tokens=128,
 43 |         temperature=0.1,
 44 |         top_p=0.75,
 45 |         top_k=40,
 46 |         num_beams=4,
 47 |         **kwargs,
 48 | ):
 49 |     prompt = create_prompt(instruction)
 50 |     print(prompt)
 51 |     inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
 52 |     #input_ids = inputs["input_ids"].to("cuda")
 53 |     #attention_mask = inputs["attention_mask"].to("cuda")
 54 | 
 55 |     generation_config = GenerationConfig(
 56 |         temperature=temperature,
 57 |         top_p=top_p,
 58 |         top_k=top_k,
 59 |         num_beams=num_beams,
 60 |         **kwargs,
 61 |     )
 62 |     with torch.no_grad():
 63 |         generation_output = model.generate(
 64 |             #input_ids=input_ids,
 65 |             #attention_mask=attention_mask,
 66 |             **inputs,
 67 |             generation_config=generation_config,
 68 |             return_dict_in_generate=True,
 69 |             output_scores=True,
 70 |             max_new_tokens=max_new_tokens,
 71 |             early_stopping=True
 72 |         )
 73 | 
 74 | 
 75 | 
 76 |     generated_response = tokenizer.decode(outputs[0], skip_special_tokens=True)
 77 |     stop_output = "### Input"
 78 |     gen_response = (generated_response.split(stop_output))[0]
 79 | 
 80 | 
 81 |     #s = generation_output.sequences[0]
 82 |     #output = tokenizer.decode(s, skip_special_tokens=True)
 83 |     #stop_output = "### Input"
 84 | 
 85 |     #gen_response = (output.split(stop_output))[0]
 86 | 
 87 | 
 88 |     #return output.split("### Response:")[1].lstrip("\n")
 89 |     return gen_response
 90 | 
 91 | 
 92 | 
 93 | 
 94 | 
 95 | 
 96 | instruction = """
 97 |  Write a python code for the name Ahmed to be in a reversed order
 98 | """
 99 | print(generate(instruction))
100 | ```
101 | 
102 | 
103 | [Colab Badge]:          https://colab.research.google.com/assets/colab-badge.svg
104 | [License-Badge]:        https://img.shields.io/badge/License-MIT-blue.svg
105 | [RDP Issues]:           https://img.shields.io/github/issues/PradyumnaKrishna/Colab-Hacks/Colab%20RDP?label=Issues
106 | [RDP Notebook]:         https://colab.research.google.com/drive/18sAFC7msV0gJ24wn5gl41nU0QRynfLqG?usp=sharing
107 | [Code Issues]:          https://img.shields.io/github/issues/PradyumnaKrishna/Colab-Hacks/Code%20Server?label=Issues
108 | [Code Notebook]:        https://colab.research.google.com/drive/18sAFC7msV0gJ24wn5gl41nU0QRynfLqG?usp=sharing
109 | 


--------------------------------------------------------------------------------