Update tokenizer

#9
by alvarobartt HF staff - opened

Description

Updates both tokenizer.json and special_tokens_map.json as in https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/commit/07eb05b21d191a58c577b4a45982fe0c049d0693

alvarobartt changed pull request title from Update special_tokens_map.json to Update tokenizer
alvarobartt changed pull request status to merged
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment