Pythia Quantized Model for Sentiment Analysis

This repository hosts a quantized version of the Pythia model, fine-tuned for sentiment analysis tasks. The model has been optimized for efficient deployment while maintaining high accuracy, making it suitable for resource-constrained environments.

Model Details

Model Architecture: Pythia-410m
Task: Sentiment Analysis
Dataset: IMDb Reviews
Quantization: Float16
Fine-tuning Framework: HF中国镜像站 Transformers

The quantized model achieves comparable performance to the full-precision model while reducing memory usage and inference time.

Usage

Installation

pip install transformers torch

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AventIQ-AI/pythia-410m")
model = AutoModelForCausalLM.from_pretrained(model_name, low_cpu_mem_usage=True)

# Example usage
text = "This product is amazing!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Performance Metrics

Accuracy: 0.56
F1 Score: 0.56
Precision: 0.68
Recall: 0.56

Fine-Tuning Details

Dataset

The IMDb Reviews dataset was used, containing both positive and negative sentiment examples.

Training

Number of epochs: 3
Batch size: 8
evaluation_strategy= epoch
Learning rate: 2e-5

Quantization

Post-training quantization was applied using PyTorch's built-in quantization framework to reduce the model size and improve inference efficiency.

Repository Structure

.
├── model/               # Contains the quantized model files
├── tokenizer/           # Tokenizer configuration and vocabulary files
├── model.safensors/            # Fine Tuned Model
├── README.md            # Model documentation

Limitations

The model may not generalize well to domains outside the fine-tuning dataset.
Quantization may result in minor accuracy degradation compared to full-precision models.

Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.