View Single Post
Old 06-11-2012, 01:52 PM   #11
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 8,878
Karma: 6120478
Join Date: Nov 2009
Device: many
Hi,

Resizing and converting images can be quite time consuming and memory intensive.

Also often long execution times in python are related to memory allocation/deallocation thrashing. All strings in Python are immutable. So any time you append or delete or modify a string in any way, python allocates new memory and deallocates the old. So appending/changing lots of little substrings in a long (large) string can result in massive thrashing of memory (the cpu is pegged just deallocating and reallocating memory and the paging and virtual memory that can be associated with that).

Walking and modifying long css and xhtml files can be quite costly is lots of little changes are needed on the large xhtml file.

The only "workaround" for the python approach of having strings be immutable is to create lists of string pieces and when finally ready allocating the required space once, and then copying all of the strings in the list (in order) into that newly allocated string.

This approach prevents the memory thrashing and can significantly speed up operations in python on memory limited platforms.

Perhaps these may be contributing to what you are seeing. Does your "benchmark" conversion properly resize and convert images and properly process/convert CSS entities, processing large xhtml files, etc?

KevinH
KevinH is offline   Reply With Quote