Categories / Natural Language Processing
tiktoken-go / tokenizer
Pure Go implementation of OpenAI's tiktoken tokenizer
Last push 4 weeks ago
Created
Apr 2023
Total stars
463
Last 7 days
0
Last 30 days
+5
Last 12 months
+64
README badge
[](https://bestofgo.dev/tiktoken-go/tokenizer)