01-ai/Yi-6B
BAAI/Aquila-7B
BAAI/AquilaChat-7B
BAAI/AquilaChat2-7B
CohereForAI/c4ai-command-r-v01
EleutherAI/gpt-j-6B
EleutherAI/gpt-j-6b
EleutherAI/gpt-neo-1.3B
EleutherAI/gpt-neo-125m
EleutherAI/gpt-neo-2.7B
EleutherAI/gpt-neox-20b
EleutherAI/pythia-160m
GAIR/Abel-7B-002
OrionStarAI/Orion-14B-Base
PygmalionAI/pygmalion-6b
Qwen/Qwen-7B
Qwen/Qwen-7B-Chat
Qwen/Qwen1.5-0.5B
Qwen/Qwen1.5-1.8B
Qwen/Qwen1.5-7B
Qwen/Qwen1.5-7B-Chat
Qwen/Qwen1.5-MoE-A2.7B
Qwen/Qwen1.5-MoE-A2.7B-Chat
Qwen/Qwen2-7B
Qwen/Qwen2-7B-Instruct
Salesforce/codegen-350M-multi
Salesforce/codegen-350M-nl
Salesforce/codegen2-1b
# Salesforce/xgen-7b-8k-base: Transformers issue - Object of type method is not JSON serializable (https://huggingface.co/Salesforce/xgen-7b-8k-base/discussions/32)
THUDM/chatglm3-6b
TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ
TinyLlama/TinyLlama-1.1B-Chat-v0.6
TinyLlama/TinyLlama-1.1B-Chat-v1.0
TitanML/tiny-mixtral
WizardLMTeam/WizardCoder-15B-V1.0
allenai/OLMo-1B-hf
allenai/OLMo-7B-hf
baichuan-inc/Baichuan-7B
baichuan-inc/Baichuan2-7B-Base
baichuan-inc/Baichuan2-7B-Chat
berkeley-nest/Starling-LM-7B-alpha
bigcode/gpt_bigcode-santacoder
bigcode/starcoder2-3b
bigcode/starcoder2-7b
bigcode/starcoderbase-3b
bigscience/bloom-560m
bigscience/bloom-7b1
bigscience/bloomz-1b7
bigscience/bloomz-560m
bigscience/bloomz-7b1
cerebras/Cerebras-GPT-13B
core42/jais-13b
core42/jais-13b-chat
databricks/dolly-v1-6b
databricks/dolly-v2-3b
# deepseek-ai/deepseek-coder-33b-instruct: OpenVINO tokenizers - Cannot convert tokenizer of this type without `.model` file
# deepseek-ai/deepseek-coder-6.7b-instruct: OpenVINO tokenizers - Cannot convert tokenizer of this type without `.model` file
deepseek-ai/deepseek-moe-16b-base
deepseek-ai/DeepSeek-V3-Base
facebook/blenderbot-3B
facebook/incoder-1B
facebook/opt-1.3b
facebook/opt-125m
facebook/opt-2.7b
facebook/opt-350m
facebook/opt-6.7b
google/gemma-1.1-7b-it
google/gemma-2b
google/gemma-2b-it
google/gemma-7b
google/gemma-2-9b
google/pegasus-big_patent
google/pegasus-large
gpt2
gpt2-xl
internlm/internlm-chat-7b
internlm/internlm2-7b
lmsys/longchat-7b-v1.5-32k
lmsys/vicuna-7b-v1.3
lmsys/vicuna-7b-v1.5
meta-llama/CodeLlama-7b-hf
meta-llama/Llama-2-7b-chat-hf
meta-llama/Llama-2-7b-hf
meta-llama/Meta-Llama-3-8B-Instruct
meta-llama/Meta-Llama-3.1-8B
meta-llama/Meta-Llama-3.1-8B-Instruct
microsoft/DialoGPT-large
microsoft/DialoGPT-medium
microsoft/Orca-2-7b
microsoft/Phi-3-mini-128k-instruct
microsoft/Phi-3-mini-4k-instruct
microsoft/Phi-3-medium-128k-instruct
microsoft/Phi-3-small-8k-instruct
microsoft/Phi-3-small-128k-instruct
microsoft/Phi-3.5-MoE-instruct
# microsoft/biogpt: OpenVINO Tokenizers - openvino.runtime.exceptions.OVTypeError: Tokenizer type is not supported: <class 'transformers.models.biogpt.tokenization_biogpt.BioGptTokenizer'>
microsoft/phi-1_5
microsoft/phi-2
mistralai/Mistral-7B-Instruct-v0.1
mistralai/Mistral-7B-v0.1
mistralai/Mixtral-8x7B-Instruct-v0.1
mistralai/Mixtral-8x7B-v0.1
mosaicml/mpt-30b
mosaicml/mpt-7b
mosaicml/mpt-7b-chat
nomic-ai/gpt4all-j
nomic-ai/gpt4all-mpt
openai-community/gpt2
openai-community/gpt2-large
openai-community/gpt2-medium
openai-community/gpt2-xl
openbmb/MiniCPM-2B-dpo-bf16
openbmb/MiniCPM-2B-sft-bf16
openchat/openchat_3.5
openlm-research/open_llama_13b
openlm-research/open_llama_3b
openlm-research/open_llama_3b_v2
# replit/replit-code-v1-3b: OpenVINO Tokenizers - AttributeError: 'ReplitLMTokenizer' object has no attribute 'sp_model'
rinna/bilingual-gpt-neox-4b
rinna/youri-7b-chat
stabilityai/stable-code-3b
stabilityai/stable-zephyr-3b
stabilityai/stablelm-2-zephyr-1_6b
stabilityai/stablelm-3b-4e1t
tiiuae/falcon-7b
tiiuae/falcon-rw-7b
togethercomputer/RedPajama-INCITE-Chat-3B-v1
# xverse/XVERSE-7B-Chat: Transformers - Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 78 column 3
# xverse/XVERSE-MoE-A4.2B: Transformers - Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 78 column 3
Deci/DeciLM-7B
