IsakNordgren commited on
Commit
54bf217
·
verified ·
1 Parent(s): 5ddf4bc

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +56 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - AI-Sweden-Models/Llama-3-8B-instruct
4
+ tags:
5
+ - merge
6
+ - mergekit
7
+ - lazymergekit
8
+ - AI-Sweden-Models/Llama-3-8B-instruct
9
+ ---
10
+
11
+ # OrpoLlama-3-8B-x-Llama-3-8B-instruct
12
+
13
+ OrpoLlama-3-8B-x-Llama-3-8B-instruct is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
14
+ * [AI-Sweden-Models/Llama-3-8B-instruct](https://huggingface.co/AI-Sweden-Models/Llama-3-8B-instruct)
15
+
16
+ ## 🧩 Configuration
17
+
18
+ ```yaml
19
+ models:
20
+ - model: mlabonne/OrpoLlama-3-8B
21
+ # No parameters necessary for base model
22
+ - model: AI-Sweden-Models/Llama-3-8B-instruct
23
+ parameters:
24
+ density: 0.53
25
+ weight: 0.6
26
+ merge_method: dare_ties
27
+ base_model: mlabonne/OrpoLlama-3-8B
28
+ parameters:
29
+ int8_mask: true
30
+ dtype: bfloat16
31
+ ```
32
+
33
+ ## 💻 Usage
34
+
35
+ ```python
36
+ !pip install -qU transformers accelerate
37
+
38
+ from transformers import AutoTokenizer
39
+ import transformers
40
+ import torch
41
+
42
+ model = "IsakNordgren/OrpoLlama-3-8B-x-Llama-3-8B-instruct"
43
+ messages = [{"role": "user", "content": "What is a large language model?"}]
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained(model)
46
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
47
+ pipeline = transformers.pipeline(
48
+ "text-generation",
49
+ model=model,
50
+ torch_dtype=torch.float16,
51
+ device_map="auto",
52
+ )
53
+
54
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
55
+ print(outputs[0]["generated_text"])
56
+ ```