Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
63
Yixin Song
yixinsong
AI & ML interests
None yet
Recent Activity
liked
a dataset
7 days ago
epfml/FineWeb2-HQ
liked
a dataset
14 days ago
bytedance-research/MAGACorpus
published
a model
18 days ago
PowerInfer/OPT-7B-predictor
Organizations
yixinsong's activity
upvoted
a
paper
about 2 months ago
upvoted
a
paper
3 months ago
upvoted
a
paper
6 months ago
upvoted
3
papers
7 months ago
Dolphin: Long Context as a New Modality for Energy-Efficient On-Device Language Models
Paper
•
2408.15518
•
Published
•
43
Scaling Synthetic Data Creation with 1,000,000,000 Personas
Paper
•
2406.20094
•
Published
•
97
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
Paper
•
2406.08464
•
Published
•
67
upvoted
2
papers
8 months ago
upvoted
3
papers
9 months ago
ReLU^2 Wins: Discovering Efficient Activation Functions for Sparse LLMs
Paper
•
2402.03804
•
Published
•
2
PowerInfer-2: Fast Large Language Model Inference on a Smartphone
Paper
•
2406.06282
•
Published
•
38
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
Paper
•
2406.05955
•
Published
•
27
upvoted
a
paper
11 months ago
upvoted
a
paper
about 1 year ago
upvoted
a
collection
about 1 year ago