Snowad commited on
Commit
960455b
·
verified ·
1 Parent(s): 26bba0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -1
README.md CHANGED
@@ -3,4 +3,17 @@ license: apache-2.0
3
  ---
4
  # MarinbadGPT
5
 
6
- MarinbadGPT est un modèle de langage basé sur l'architecture SmolLM-135M de HuggingFaceTB, finement entraîné sur un corpus de parties de **Marinbad**. L'objectif de ce modèle est de générer des parties de Marinbad et d'affronter un joueur humain
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
4
  # MarinbadGPT
5
 
6
+ MarinbadGPT is a language model based on HuggingFaceTB's SmolLM-135M architecture, finely trained on a corpus of **Marinbad** games. The aim of this model is to generate games of Marinbad and play them against a human player.
7
+
8
+ ## Model Training
9
+
10
+ The training of MarinbadGPT was conducted on a high-performance computing infrastructure utilizing NVIDIA H100 GPUs, renowned for their power in the field of deep learning.
11
+
12
+ **Training Configuration:**
13
+
14
+ * **Infrastructure:** 2x NVIDIA H100 (80GB HBM3)
15
+ * **Duration:** 1 hour
16
+ * **Optimizer:** AdamW
17
+ * **Learning Rate:** 3e-4
18
+ * **Batch Size:** Micro batch size of 128, with gradient accumulation steps of 8, resulting in an effective batch size of 1024
19
+ * **Warmup Steps:** 100