TheTsar1209 commited on
Commit
1e0c549
·
verified ·
1 Parent(s): 02152e4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -5
README.md CHANGED
@@ -8,11 +8,14 @@ tags:
8
  - merge
9
 
10
  ---
11
- # output
 
 
 
 
12
 
13
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
 
15
- ## Merge Details
16
  ### Merge Method
17
 
18
  This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
@@ -21,7 +24,7 @@ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge m
21
 
22
  The following models were included in the merge:
23
  * [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
24
- * ../htwfinetune/qwen-carpmuscle-v0.3
25
 
26
  ### Configuration
27
 
@@ -29,7 +32,7 @@ The following YAML configuration was used to produce this model:
29
 
30
  ```yaml
31
  models:
32
- - model: ../htwfinetune/qwen-carpmuscle-v0.3
33
  parameters:
34
  weight: 1
35
  density: 1
@@ -44,7 +47,7 @@ parameters:
44
  density: 1
45
  normalize: true
46
  int8_mask: true
47
- tokenizer_source: ../htwfinetune/qwen-carpmuscle-v0.3
48
  dtype: bfloat16
49
 
50
  ```
 
8
  - merge
9
 
10
  ---
11
+ A Fishy Model
12
+
13
+ qwen-carpmuscle-r-v0.3 was made using Rombodawg's Shared Continuous Finetuning method.
14
+
15
+ qwen-carpmuscle-v0.3 was made using Unsloth's continuous pretraining on the ChatML format with 24k context and was merged with Qwen2.5-14B-Instruct and Qwen2.5-14B.
16
 
17
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
18
 
 
19
  ### Merge Method
20
 
21
  This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) as a base.
 
24
 
25
  The following models were included in the merge:
26
  * [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
27
+ * qwen-carpmuscle-v0.3
28
 
29
  ### Configuration
30
 
 
32
 
33
  ```yaml
34
  models:
35
+ - model: qwen-carpmuscle-v0.3
36
  parameters:
37
  weight: 1
38
  density: 1
 
47
  density: 1
48
  normalize: true
49
  int8_mask: true
50
+ tokenizer_source: qwen-carpmuscle-v0.3
51
  dtype: bfloat16
52
 
53
  ```