Update README.md
Browse files
README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
| 3 |
---
|
| 4 |
# GPT2-Medium-BG 2021
|
| 5 |
|
| 6 |
-
* GPT2-Medium
|
| 7 |
* The model was created and trained from scratch, using tensorflow in free Google Colab T4. The research experiment started in June 2021 and continued to September, the video explanation is uploaded on 17.9.2024.
|
| 8 |
* It is supposed to be run with the provided code here and in the notebook. Read the comments in gen_comments-1-2023-clean.py
|
| 9 |
* That was the biggest, as far as I knew, GPT/Transformer model in Bulgarian at the time, except one with unknown size, which was demoed for a few seconds in a video in Linkedin: a display of generation in Bulgarian by a startup called BAIHUI AI in mid 2019. I've written in my blog 1.5B, but I don't remember if they have mentioned a size and now it seems unlikely, they didn't have the resurces of OpenAI, three people, one ML engineer. https://artificial-mind.blogspot.com/2019/07/baihuiai-baihuiai-new-bulgarian-ai.html The company didn't live long, it was a show-off. Now it seems reasonable that their model was GPT2-SMALL, as that was the usual choice for demos even years later; I couldn't get info from the ML engineer of the project M.V. I found several other GPT2-SMALL models trained later here, one for poetry, one by Bulgarian Academy of Science in 2023.
|
|
|
|
| 3 |
---
|
| 4 |
# GPT2-Medium-BG 2021
|
| 5 |
|
| 6 |
+
* GPT2-Medium 345M for Bulgarian
|
| 7 |
* The model was created and trained from scratch, using tensorflow in free Google Colab T4. The research experiment started in June 2021 and continued to September, the video explanation is uploaded on 17.9.2024.
|
| 8 |
* It is supposed to be run with the provided code here and in the notebook. Read the comments in gen_comments-1-2023-clean.py
|
| 9 |
* That was the biggest, as far as I knew, GPT/Transformer model in Bulgarian at the time, except one with unknown size, which was demoed for a few seconds in a video in Linkedin: a display of generation in Bulgarian by a startup called BAIHUI AI in mid 2019. I've written in my blog 1.5B, but I don't remember if they have mentioned a size and now it seems unlikely, they didn't have the resurces of OpenAI, three people, one ML engineer. https://artificial-mind.blogspot.com/2019/07/baihuiai-baihuiai-new-bulgarian-ai.html The company didn't live long, it was a show-off. Now it seems reasonable that their model was GPT2-SMALL, as that was the usual choice for demos even years later; I couldn't get info from the ML engineer of the project M.V. I found several other GPT2-SMALL models trained later here, one for poetry, one by Bulgarian Academy of Science in 2023.
|