CyberBrain_Model

GitHub_Project_link

CyberBrain_Model is an advanced AI project designed for fine-tuning the model unsloth/DeepSeek-R1-Distill-Qwen-14B specifically for cyber security tasks. This repository provides tools and scripts for training and fine-tuning large language models efficiently using minimal hardware resources. The goal is to adapt the model for ethical cyber security applications, making it efficient even on devices with limited computational power, whether you have a low-end CPU or a GPU with limited VRAM.

In this project, we use technical content extracted from various cyber security sources as our primary training data. The raw text is processed into instruction-response pairs tailored for fine-tuning the model on cyber security scenarios. You can access the training data here.

📦 Project Structure

assest/                         # Assets, images, and other media files
Configure_Training_Arguments.py  # Script for configuring training arguments
DataSet/                         # Directory containing dataset files
Load_DataSet.py                  # Script to load the dataset
LoRA_Configuration.py            # Script for LoRA configuration
map.md                           # Documentation about mapping
Model_Loading_with_Unsloth.py    # Script to load the model using Unsloth
README.md                        # This file
requirements.txt                 # Required dependencies for the project
Table-Ways.md                    # Documentation about table ways
Train_Start.py                   # Script to start training the model

🚀 Installation

1. Clone the Repository

git clone https://github.com/YourUsername/CyberBrain_Model.git
cd CyberBrain_Model

2. Set Up the Environment

Create a new virtual environment (Python 3.11 is recommended):

python -m venv .env
# Activate the environment:
# On Linux/Mac:
source .env/bin/activate
# On Windows:
.env\Scripts\activate

3. Install Required Dependencies

pip install --upgrade pip
pip install -r requirements.txt
pip install torch==2.5.1+cu118 --index-url https://download.pytorch.org/whl/cu118
pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

🤖 Running the Project

• Model Loading: Run Model_Loading_with_Unsloth.py to load the model.
• Training: Run Train_Start.py to start the fine-tuning process.
• Configurations: Review LoRA_Configuration.py and Configure_Training_Arguments.py for training settings.

📄 Additional Documentation

Refer to the following files for more details:

• map.md
• Table-Ways.md

---

🚀 Quick Start on Google Colab

To quickly run CyberBrain_Model on Google Colab, follow these steps:

1. Open a New Colab Notebook
   Click here to open a new Colab notebook in your browser.

2. Clone the Repository
   In your Colab notebook, run:

   !git clone https://github.com/YourUsername/CyberBrain_Model.git
   %cd CyberBrain_Model
   

3. Install Dependencies
   Install the required packages by running:

   !pip install --upgrade pip
   !pip install -r requirements.txt
   !pip install torch==2.5.1+cu118 --index-url https://download.pytorch.org/whl/cu118
   !pip install torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
   

4. Open and Run main.ipynb
   Open the main.ipynb notebook in Colab. This notebook provides a step-by-step guide to:

   • Load the dataset from the DataSet directory.
   • Load the model using Model_Loading_with_Unsloth.py.
   • Configure training arguments via Configure_Training_Arguments.py.
   • Start training using Train_Start.py.
   • Evaluate the model and monitor training progress.

---

License

This project is licensed under the MIT License – see the LICENSE file for details.

Contact

For questions or contributions, feel free to open an issue or contact us directly through GitHub.

• Portfolio: peteradel.netlify.app
• LinkedIn: linkedin.com/in/1peteradel

⭐ Give a Star

If you find this project useful or interesting, please give it a star! Your support helps improve the project and motivates further development.

---

🤍 Thank you for checking out CyberBrain_Model! Happy training!

---

Uploaded model

• Developed by: PeterAdel
• License: apache-2.0
• Finetuned from model : unsloth/deepseek-r1-distill-qwen-14b-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.