View Single Post
Old 06-20-2012, 07:46 PM   #7
NotTaken
Connoisseur
NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.NotTaken is fluent in JavaScript as well as Klingon.
 
Posts: 65
Karma: 4640
Join Date: Aug 2011
Device: kindle
I've come up with a cross platform solution:

windows instructions:

extract the contents of the attached zip to C:\ (or change path in recipe)

use this recipe:

Code:
from multiprocessing import Process, Queue
from calibre.ebooks.BeautifulSoup import *
import subprocess
import tempfile
import os

class MarketingSensoriale(BasicNewsRecipe):

    title                 = u'Marketing sensoriale'
    description           = 'Marketing Sensoriale, il Blog'
    category              = 'Blog'
    oldest_article        = 7
    max_articles_per_feed = 200
    no_stylesheets        = True
    encoding              = 'utf8'
    use_embedded_content  = False
    language              = 'it'
    remove_empty_feeds    = True
    recursions = 0
    delay = 0.0001
    auto_cleanup = False

    remove_tags_after    = [dict(name='div', attrs={'class':['article-footer']})]
    
    def get_article_url(self, article):
        return article.get('feedburner_origlink',  None)


    def preprocess_raw_html(self, raw, url):

        temp_handle, temp_path = tempfile.mkstemp()

        try:
            # nix style path
            #subprocess.check_call(["calibre-debug", "-e","/home/will/Downloads/spynner_calibre/grabber.py",url,temp_path], shell=False)
            # windows style path
            subprocess.check_call(["calibre-debug", "-e","C:\spynner_calibre\grabber.py",url,temp_path], shell=False)
        except:
            print 'spynner fetch failed'
        
        try:
            f = os.fdopen(temp_handle,'r')
            html = f.read()
        finally:
            f.close()
            try:
                os.remove(temp_path)
            except:
                print 'could not delete temp file:' + temp_path
            
        return html


    feeds          = [(u'Marketing sensoriale', u'http://feeds.feedburner.com/MarketingSensoriale?format=xml')]
I have two questions:

Can someone experienced verify that the subprocess.call in the above recipe is safe from shell injection?

Is it possible to import stuff from a calibre plugin into a script executed by calibre-debug? For example, I can't use
Code:
 import calibre_plugins.webkit_browser.spynner
.

Edit: got the answer to the 2nd question. You can add: from calibre.customize.ui import * , before the plugin import
Attached Files
File Type: zip spynner_calibre.zip (147.6 KB, 134 views)

Last edited by NotTaken; 06-20-2012 at 09:27 PM.
NotTaken is offline   Reply With Quote