Update README.md
Browse files
README.md
CHANGED
@@ -3,4 +3,17 @@ license: apache-2.0
|
|
3 |
---
|
4 |
# MarinbadGPT
|
5 |
|
6 |
-
MarinbadGPT
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
4 |
# MarinbadGPT
|
5 |
|
6 |
+
MarinbadGPT is a language model based on HuggingFaceTB's SmolLM-135M architecture, finely trained on a corpus of **Marinbad** games. The aim of this model is to generate games of Marinbad and play them against a human player.
|
7 |
+
|
8 |
+
## Model Training
|
9 |
+
|
10 |
+
The training of MarinbadGPT was conducted on a high-performance computing infrastructure utilizing NVIDIA H100 GPUs, renowned for their power in the field of deep learning.
|
11 |
+
|
12 |
+
**Training Configuration:**
|
13 |
+
|
14 |
+
* **Infrastructure:** 2x NVIDIA H100 (80GB HBM3)
|
15 |
+
* **Duration:** 1 hour
|
16 |
+
* **Optimizer:** AdamW
|
17 |
+
* **Learning Rate:** 3e-4
|
18 |
+
* **Batch Size:** Micro batch size of 128, with gradient accumulation steps of 8, resulting in an effective batch size of 1024
|
19 |
+
* **Warmup Steps:** 100
|