Vibrato Model Archive

This repository hosts all models from the Vibrato(url) GitHub release. Models that were compressed using zstd have already been decompressed, so they are ready for direct download and use.

Important: This repository deliberately does not provide a unified license because it contains models under different licensing terms. While most models have been detected to use the BSD license, some do not have a standard license. It is recommended to check the license file in each folder before using the models.

For those seeking models under the BSD license, you can visit the following repository (ryan-minato/vibrato-models-bsdonly).

Available Models

  • bccwj-suw+unidic-cwj-3_1_1+compact-dual
  • bccwj-suw+unidic-cwj-3_1_1+compact
  • bccwj-suw+unidic-cwj-3_1_1-extracted+compact-dual
  • bccwj-suw+unidic-cwj-3_1_1-extracted+compact
  • bccwj-suw+unidic-cwj-3_1_1
  • ipadic-mecab-2_7_0-small
  • ipadic-mecab-2_7_0
  • jumandic-mecab-7_0
  • naist-jdic-mecab-0_6_3b
  • unidic-cwj-3_1_1+compact-dual
  • unidic-cwj-3_1_1+compact
  • unidic-cwj-3_1_1
  • unidic-mecab-2_1_2

Usage

from huggingface_hub import hf_hub_download
import vibrato

# Load tokenizer from `.cache/hf`
model_path = hf_hub_download("ryan-minato/vibrato-models", "<<model_name>>/system.dic")
with open(model_path, "rb") as f:
  tokenizer = vibrato.Vibrato(f.read())

text = """\
「四十二だと!」ルーンクォールが叫んだ。
「七百五十万年かけて、それだけか?」
「何度も徹底的に検算しました」コンピュータが応じた。
「まちがいなくそれが答えです。率直なところ、みなさんのほうで究極の疑問が何であるかわかっていなかったところに問題があるのです」
"""

tokenizer.tokenize(text)
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.