View Single Post
Old 10-10-2018, 07:17 AM   #2965
fanfictioneer
Enthusiast
fanfictioneer doesn't litterfanfictioneer doesn't litter
 
Posts: 26
Karma: 110
Join Date: Oct 2018
Device: Kindle
Thanks!

That would certainly work, too - the Kindle have no issues with minor Zalgo-fication, but I suspect it runs into issues when it runs into several lines - then it starts to slow down and becomes unresponsive.

Something like this should work for your needs (warning: I am a Python n00b, and rusty on top of that):
Code:
import unicodedata

ZALGO_CHAR_CATEGORIES = ['Mn', 'Me']
maxZalgo = 0
currentZalgo = 0

def deZalgo(c):
	isZalgo = unicodedata.category(c) in ZALGO_CHAR_CATEGORIES
	if isZalgo:
		currentZalgo += 1
		if currentZalgo > maxZalgo:
			currentZalgo = 0
			return False
	return True

stripped = ''.join([c for c in unicodedata.normalize('NFD', text) if deZalgo(c)])

Last edited by fanfictioneer; 10-10-2018 at 07:48 AM.
fanfictioneer is offline   Reply With Quote