Either something like [thai segmenter](https://pypi.org/project/thai-segmenter/) or maybe [sentence piece](https://github.com/google/sentencepiece).
Either something like thai segmenter or maybe sentence piece.