Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

View Poll Results: Would an allrecipes.com plugin interest you?
Yes 7 77.78%
No 2 22.22%
Voters: 9. You may not vote on this poll

Reply
 
Thread Tools Search this Thread
Old 12-27-2010, 10:16 PM   #1
brennydoogles
Junior Member
brennydoogles began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2010
Device: Kindle 3
Looking to make a plugin, need some guidance getting started

Hello all, I am looking to write a Calibre Plugin to allow users to input a URL from allrecipes.com, and automatically convert the recipe from the url to .mobi format. I have already written the hard part, which takes the URL as a cli arg and outputs only the relevant bits as a valid HTML doc. All I need to do now is integrate my current code into the plugin. Unfortunately, I have only used Python for CLI programs in the past, so I have no idea how to create the GUI. Anyone have any pointers to help me get started?

As a recap: I want to build a very basic GUI Plugin and need some guidance, the backend is pretty much ready.
brennydoogles is offline   Reply With Quote
Old 12-27-2010, 10:59 PM   #2
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
I've written a bunch of GUI plugins which you can download the source code of in this thread

If you read the first post in the thread it also has some links to some useful help topics etc. I'm happy to give some help if you need it, either send me a PM or post on here for others to also assist. I hadn't written any Python at all before starting to dabble with Calibre and got enough to get by pretty quickly.
kiwidude is offline   Reply With Quote
Old 12-28-2010, 01:32 PM   #3
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by brennydoogles View Post
Hello all, I am looking to write a Calibre Plugin to allow users to input a URL from allrecipes.com, and automatically convert the recipe from the url to .mobi format.
Your other option is to use the Calibre recipe (code) system to grab recipes (food) from allrecipes.com. Although the recipe (code) system of Calibre is not designed for simple user input when a recipe (code) runs, you can load the URL (or a portion thereof) via the username password field, via modifying the recipe to add a list of links, or through a Python coded interface.
Starson17 is offline   Reply With Quote
Old 12-28-2010, 01:39 PM   #4
brennydoogles
Junior Member
brennydoogles began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2010
Device: Kindle 3
Quote:
Originally Posted by kiwidude View Post
I'm happy to give some help if you need it, either send me a PM or post on here for others to also assist.
I would actually love working through it in a thread here if possible. I learn things far more quickly when I have guidance with my first project than if I don't. So here's what I've done so far:

I have written the following methods to input a URL (from allrecipes.com), and return a concise HTML document containing the recipe:
*Note: This script depends on BeautifulSoup.
Code:
import urllib2, sys, string
from BeautifulSoup import BeautifulSoup

def buildHTMLOutput(title, author, ingredientList, directionList):
	output = ""
	titleAuthor = "%s by %s" % (string.capwords(title), string.capwords(author))
	header = "<html><head><title>%s</title></head><body><h2>%s</h2>" % (titleAuthor, titleAuthor)
	ingredients = "<h2>Ingredients:</h2>"
	for ingredient in ingredientList:
		if ingredient != "":
			ingredients += "<li>%s</li>" % (ingredient)
	directions = "<h2>Directions:</h2>"
	for direction in directionList:
		if direction != "":
			directions += "<li>%s</li>" % (direction)
	ingredientSection = "<ul>%s</ul>" % (ingredients)	
	directionSection = "<ol>%s</ol>" % (directions)
	footer = "</body></html>"
	output = "%s\n%s\n%s\n%s" % (header, ingredientSection, directionSection, footer)
	return output

def parseRecipeFromLink(url):
	try:
		ingredientList = []
		directionList = []
		req = urllib2.Request(url)
		response = urllib2.urlopen(req)
		pageText = response.read()
		htmlParser = BeautifulSoup(''.join(pageText))

		ingredientDiv = BeautifulSoup(htmlParser.find('div', "ingredients").prettify())
		ingredients = ingredientDiv.findAll('li')
		for ingredient in ingredients:
			for i in ingredient.findAll(text=True):
				ingredientList.append(i.replace('\n', '').strip())

		directionDiv = BeautifulSoup(htmlParser.find('div', "directions").prettify())	
		directions = directionDiv.findAll('li')
		for step in directions:
			for i in step.findAll(text=True):
				directionList.append(i.replace('\n', '').strip())

		authorDiv = BeautifulSoup(htmlParser.find('div', "author-name").prettify())
		spans = authorDiv.findAll('span')
		for span in spans:
			author =  span.findAll(text=True)[0].replace('\n', '').strip()

		titleArea = BeautifulSoup(htmlParser.find('h1', {"id" : "itemTitle"}).prettify())
		title = titleArea.findAll(text=True)[1].replace('\n', '').strip()
		output = buildHTMLOutput(title, author, ingredientList, directionList)
		soup = BeautifulSoup(output)
		return soup.prettify()
	except Exception, detail: 
		print "Error: ", detail

if __name__ == "__main__":
	if (len(sys.argv) == 1):
		print "No URL was specified"
		quit = 1	
	else:
		if "http://allrecipes.com/" in sys.argv[1]:
			url = sys.argv[1]
		else:
			print "The URL must be from allrecipes.com"
			quit = 1
	if quit != 1:
		print parseRecipeFromLink(url)
So basically, I just need to write the plugin to:
  1. Accept a URL as input from the user
  2. Pass the URL to my script and retrieve the HTML output (possibly into a temp file?)
  3. Convert the HTML into mobi output
  4. Add the recipe to the Library

So first thing's first: Have I missed any crucial steps or is there anything I need to consider before I proceed?
brennydoogles is offline   Reply With Quote
Old 12-28-2010, 03:17 PM   #5
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Quote:
Originally Posted by brennydoogles View Post
So first thing's first: Have I missed any crucial steps or is there anything I need to consider before I proceed?
I would think the first thing would be to consider Starson17's suggestion of whether a "recipe" approach is more suitable than a UI plugin. I know absolutely nothing about recipes having never used them which is why I did not mention the option myself. However if there is indeed a way to incorporate a way of the URL being supplied by the user into that which Starson17 indicates some options for then that sounds like a much better approach given it does most of what you need?
kiwidude is offline   Reply With Quote
Old 12-29-2010, 08:58 PM   #6
brennydoogles
Junior Member
brennydoogles began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2010
Device: Kindle 3
Quote:
Originally Posted by Starson17 View Post
Your other option is to use the Calibre recipe (code) system to grab recipes (food) from allrecipes.com. Although the recipe (code) system of Calibre is not designed for simple user input when a recipe (code) runs, you can load the URL (or a portion thereof) via the username password field, via modifying the recipe to add a list of links, or through a Python coded interface.
I'm not sure about the recipe idea... I thought that recipes were designed for aggregating news items like RSS feeds. Is that assumption incorrect?


Quote:
Originally Posted by kiwidude View Post
I would think the first thing would be to consider Starson17's suggestion of whether a "recipe" approach is more suitable than a UI plugin. I know absolutely nothing about recipes having never used them which is why I did not mention the option myself. However if there is indeed a way to incorporate a way of the URL being supplied by the user into that which Starson17 indicates some options for then that sounds like a much better approach given it does most of what you need?
It seems like the only input I really need is a URL. Even if the plugin simply pops up a dialog box asking for it and then processes from there, I should be able to do the rest automatically. Is there a simple way to get user input?
brennydoogles is offline   Reply With Quote
Old 12-30-2010, 07:54 AM   #7
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by brennydoogles View Post
I'm not sure about the recipe idea... I thought that recipes were designed for aggregating news items like RSS feeds. Is that assumption incorrect?
It can parse an RSS feed or a web page to get links. See the parse_index method.


Quote:
It seems like the only input I really need is a URL. Even if the plugin simply pops up a dialog box asking for it and then processes from there, I should be able to do the rest automatically. Is there a simple way to get user input?
As I posted, the simplest is to edit the recipe directly to add the URL. Second easiest is to pretent the URL is a username or password, tell the recipe it needs authentication, then pass in the input through one of the user/pass boxes that appears. Finally, you can write an interface. You can do anything you can do in Python. There's a thread on user input.
Starson17 is offline   Reply With Quote
Old 12-30-2010, 10:58 AM   #8
brennydoogles
Junior Member
brennydoogles began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2010
Device: Kindle 3
Quote:
Originally Posted by Starson17 View Post
You can do anything you can do in Python. There's a thread on user input.
As someone who has little experience with Python, I have never done any type of GUI in normal Python (I have used IronPython, C#, or Java for GUI stuff). I am not exactly sure how to get the input from the user, but a link to the thread you were talking about would be very helpful (I was unable to find it on the forum) Thanks for all of the help!
brennydoogles is offline   Reply With Quote
Old 12-30-2010, 12:31 PM   #9
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by brennydoogles View Post
As someone who has little experience with Python, I have never done any type of GUI in normal Python (I have used IronPython, C#, or Java for GUI stuff). I am not exactly sure how to get the input from the user, but a link to the thread you were talking about would be very helpful (I was unable to find it on the forum) Thanks for all of the help!
http://www.mobileread.com/forums/sho...ghlight=dialog
Starson17 is offline   Reply With Quote
Reply

Tags
help needed, plugin, plugin development

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
New Plugin Type Idea: Library Plugin cgranade Plugins 3 09-15-2010 12:11 PM
Ebook advice and guidance Radioteacher Workshop 2 01-17-2010 04:43 PM
New to Linux, need some guidance! KindleMan Sony Reader Dev Corner 6 03-10-2008 11:53 AM
Guidance needed Tom Storer Which one should I buy? 7 12-03-2007 02:09 AM


All times are GMT -4. The time now is 10:16 PM.


MobileRead.com is a privately owned, operated and funded community.