View Single Post
Old 03-23-2023, 09:54 PM   #24
CasualBookworm
Junior Member
CasualBookworm doesn't litterCasualBookworm doesn't litter
 
Posts: 3
Karma: 168
Join Date: Mar 2023
Device: Kobo Aura Edition 2
I'm not sure how I overlooked this the first time, but shortly after that last post I realized that the article directories are in numeric order, so a reverse sort will give a reliable newest-first order!

I've updated my tweaked script to:
  • Use $(($x + 1)) instead of expr
  • Create .processed files inside the article directories instead of a separate one
  • Removed -d DAYS and the -mtime argument when finding directories
  • Updated the directory iteration to always go in reverse numeric order
  • Added -p ARTICLES to limit the number of articles that will be processed in a given run
  • Added -s ARTICLES to limit the number of articles that will be processed OR skipped in a given run
  • Added -r to force re-processing old articles
(those last two are probably more useful for debugging than regular use)

Code:
#!/bin/sh

usage()
{
cat << EOF
usage: $0 [ -r ] [ -s ARTICLES ] [ -p ARTICLES ]

Pocket Image Fix:
Converts gif, png, svg, and webp images in downloaded Pocket articles to jpeg
so that they render correctly in the Kobo reader

OPTIONS:
  -r               reprocess previously processed articles
  -s ARTICLES      scan at most ARTICLES articles
  -p ARTICLES      process at most ARTICLES articles
  -h               prints this help message
EOF
}

# Setup
ARTICLES="/mnt/onboard/.kobo/articles"
POCKET="/mnt/onboard/.adds/pocket"

CONVERT="${POCKET}/magick convert -limit time 60 -font ${POCKET}/fonts/DejaVuSans.ttf"
IDENTIFY="${POCKET}/magick identify -limit time 60 -quiet"

LD_LIBRARY_PATH="${POCKET}/lib:${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH

FLAG_FILE=".processed"

# Parse arguments
unset ARG_PROCESS_LIMIT
unset ARG_SCAN_LIMIT
unset ARG_REPROCESS

while getopts 's:p:rh' c
do
  case $c in
    s) 
        if ! [ "$OPTARG" -gt 0 ] 2> /dev/null; then
          echo "Invalid arg for -s option! Expected a positive integer, got: $OPTARG"
          usage
          exit 1
        fi
        ARG_SCAN_LIMIT=$OPTARG
        ;;
    p) 
        if ! [ "$OPTARG" -gt 0 ] 2> /dev/null; then
          echo "Invalid arg for -p option! Expected a positive integer, got: $OPTARG"
          usage
          exit 1
        fi
        ARG_PROCESS_LIMIT=$OPTARG
        ;;
    r) 
        ARG_REPROCESS=1
        ;;
    *)
        usage
        exit 0
        ;;
  esac
done

if [ -n "$ARG_SCAN_LIMIT" ]; then
  echo "Scanning up to $ARG_SCAN_LIMIT articles"
else
  echo "Scanning all articles"
fi

if [ -n "$ARG_PROCESS_LIMIT" ]; then
  echo "Processing up to $ARG_PROCESS_LIMIT articles"
else
  echo "Processing all articles; this could take a while"
fi

if [ -n "$ARG_REPROCESS" ]; then
  echo "Reprocessing previously processed articles"
fi

# Counters
skippedd=0
processedd=0

skippedf=0
convertedf=0
failedf=0

# Main Loop
echo "Started: $(date)"

for d in $(find $ARTICLES -mindepth 1 -type d | sort -Vr); do
  if [ -z "$ARG_REPROCESS" ] && [ -f "$d/$FLAG_FILE" ]; then
    skippedd=$(($skippedd + 1))
  else
    for i in $(find $d -type f -not \( -iname "*.html" -o -iname "$FLAG_FILE" \) ); do
      FORMAT=$($IDENTIFY -format "%m" "$i")
      if [ "$FORMAT" == "GIF" ] ||
         [ "$FORMAT" == "PNG" ] ||
         [ "$FORMAT" == "SVG" ] ||
         [ "$FORMAT" == "WEBP" ]; then
        $CONVERT "$i" "$i.jpg"
        if [ $? -eq 0 ]; then
          mv "$i.jpg" "${i%.jpg}"
          convertedf=$(($convertedf + 1))
        else
          failedf=$(($failedf + 1))
        fi
      else
        skippedf=$(($skippedf + 1))
      fi
    done
    processedd=$(($processedd + 1))
    touch "$d/$FLAG_FILE"

    if [ "$processedd" -ge "$ARG_PROCESS_LIMIT" ] 2> /dev/null; then
      break
    fi
  fi
  if [ $(($processedd + $skippedd)) -ge "$ARG_SCAN_LIMIT" ] 2> /dev/null; then
    break
  fi
done

# Summary
echo "Finished: $(date)"
echo
echo "Articles Processed: $processedd"
echo "Articles Skipped: $skippedd"
echo
echo "Files Converted: $convertedf"
echo "Files Not Converted: $skippedf"
echo "Files Failed: $failedf"
And then I also slightly tweaked the arguments I use when calling the script:
Code:
menu_item  :library  :Pocket Images > Convert  :cmd_spawn   :quiet:/mnt/onboard/.adds/pocket/fix.sh -p 40 > /mnt/onboard/.adds/pocket/log
  chain_success                                :dbg_toast   :Started processing articles
menu_item  :library  :Pocket Images > Status   :cmd_output  :1000:tail -n 15 /mnt/onboard/.adds/pocket/log
If there's nothing new to process it returns almost instantly and the 40-article limit seems to do a decent job of preventing it from slowing down the device for too long at any one time.
CasualBookworm is offline   Reply With Quote