I'm not sure how I overlooked this the first time, but shortly after that last post I realized that the article directories are in numeric order, so a reverse sort will give a reliable newest-first order!
I've updated my tweaked script to:
- Use $(($x + 1)) instead of expr
- Create .processed files inside the article directories instead of a separate one
- Removed -d DAYS and the -mtime argument when finding directories
- Updated the directory iteration to always go in reverse numeric order
- Added -p ARTICLES to limit the number of articles that will be processed in a given run
- Added -s ARTICLES to limit the number of articles that will be processed OR skipped in a given run
- Added -r to force re-processing old articles
(those last two are probably more useful for debugging than regular use)
Code:
#!/bin/sh
usage()
{
cat << EOF
usage: $0 [ -r ] [ -s ARTICLES ] [ -p ARTICLES ]
Pocket Image Fix:
Converts gif, png, svg, and webp images in downloaded Pocket articles to jpeg
so that they render correctly in the Kobo reader
OPTIONS:
-r reprocess previously processed articles
-s ARTICLES scan at most ARTICLES articles
-p ARTICLES process at most ARTICLES articles
-h prints this help message
EOF
}
# Setup
ARTICLES="/mnt/onboard/.kobo/articles"
POCKET="/mnt/onboard/.adds/pocket"
CONVERT="${POCKET}/magick convert -limit time 60 -font ${POCKET}/fonts/DejaVuSans.ttf"
IDENTIFY="${POCKET}/magick identify -limit time 60 -quiet"
LD_LIBRARY_PATH="${POCKET}/lib:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH
FLAG_FILE=".processed"
# Parse arguments
unset ARG_PROCESS_LIMIT
unset ARG_SCAN_LIMIT
unset ARG_REPROCESS
while getopts 's:p:rh' c
do
case $c in
s)
if ! [ "$OPTARG" -gt 0 ] 2> /dev/null; then
echo "Invalid arg for -s option! Expected a positive integer, got: $OPTARG"
usage
exit 1
fi
ARG_SCAN_LIMIT=$OPTARG
;;
p)
if ! [ "$OPTARG" -gt 0 ] 2> /dev/null; then
echo "Invalid arg for -p option! Expected a positive integer, got: $OPTARG"
usage
exit 1
fi
ARG_PROCESS_LIMIT=$OPTARG
;;
r)
ARG_REPROCESS=1
;;
*)
usage
exit 0
;;
esac
done
if [ -n "$ARG_SCAN_LIMIT" ]; then
echo "Scanning up to $ARG_SCAN_LIMIT articles"
else
echo "Scanning all articles"
fi
if [ -n "$ARG_PROCESS_LIMIT" ]; then
echo "Processing up to $ARG_PROCESS_LIMIT articles"
else
echo "Processing all articles; this could take a while"
fi
if [ -n "$ARG_REPROCESS" ]; then
echo "Reprocessing previously processed articles"
fi
# Counters
skippedd=0
processedd=0
skippedf=0
convertedf=0
failedf=0
# Main Loop
echo "Started: $(date)"
for d in $(find $ARTICLES -mindepth 1 -type d | sort -Vr); do
if [ -z "$ARG_REPROCESS" ] && [ -f "$d/$FLAG_FILE" ]; then
skippedd=$(($skippedd + 1))
else
for i in $(find $d -type f -not \( -iname "*.html" -o -iname "$FLAG_FILE" \) ); do
FORMAT=$($IDENTIFY -format "%m" "$i")
if [ "$FORMAT" == "GIF" ] ||
[ "$FORMAT" == "PNG" ] ||
[ "$FORMAT" == "SVG" ] ||
[ "$FORMAT" == "WEBP" ]; then
$CONVERT "$i" "$i.jpg"
if [ $? -eq 0 ]; then
mv "$i.jpg" "${i%.jpg}"
convertedf=$(($convertedf + 1))
else
failedf=$(($failedf + 1))
fi
else
skippedf=$(($skippedf + 1))
fi
done
processedd=$(($processedd + 1))
touch "$d/$FLAG_FILE"
if [ "$processedd" -ge "$ARG_PROCESS_LIMIT" ] 2> /dev/null; then
break
fi
fi
if [ $(($processedd + $skippedd)) -ge "$ARG_SCAN_LIMIT" ] 2> /dev/null; then
break
fi
done
# Summary
echo "Finished: $(date)"
echo
echo "Articles Processed: $processedd"
echo "Articles Skipped: $skippedd"
echo
echo "Files Converted: $convertedf"
echo "Files Not Converted: $skippedf"
echo "Files Failed: $failedf"
And then I also slightly tweaked the arguments I use when calling the script:
Code:
menu_item :library :Pocket Images > Convert :cmd_spawn :quiet:/mnt/onboard/.adds/pocket/fix.sh -p 40 > /mnt/onboard/.adds/pocket/log
chain_success :dbg_toast :Started processing articles
menu_item :library :Pocket Images > Status :cmd_output :1000:tail -n 15 /mnt/onboard/.adds/pocket/log
If there's nothing new to process it returns almost instantly and the 40-article limit seems to do a decent job of preventing it from slowing down the device for too long at any one time.