Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 03-17-2012, 02:21 PM   #1
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Curious how Calibre "gets" new books

Can one access Calibre's API to search?

Or can anyone offer any information on the subject of getting book information from Amazon? I've looked into Amazon's API and searching for a book returns the "Digital List Price" and never the "Kindle Price".

When asking Amazon about it, they say that they do not provide this in their API (there is a forum post dated Dec. 6, 2010 that they stick to). However, Calibre correctly gets the Kindle price of a book, therefore they have found a way around this.

Is it just a matter of parsing the page? or is their some other API stuff that exists?

Thanks for listening/reading!
Dionysus is offline   Reply With Quote
Old 03-17-2012, 03:03 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,817
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
web scraping. i.e. parsing the page
kovidgoyal is offline   Reply With Quote
Old 03-17-2012, 08:03 PM   #3
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
I don't understand?? I was not asking about DRM removal... I'm talking about searching Amazon. You manage to get the correct prices, I was just curious how since Amazon's API does not support the return via API of "Kindle Price".
Dionysus is offline   Reply With Quote
Old 03-17-2012, 08:03 PM   #4
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Or are you telling me that you parse the page?
Dionysus is offline   Reply With Quote
Old 03-17-2012, 09:16 PM   #5
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,220
Karma: 1333994
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Quote:
Originally Posted by Dionysus View Post
Or are you telling me that you parse the page?
That is exactly what Kovid is saying - the plugins for calibre parse the same sort of Amazon web pages that you would see in a web browser, but using xpath queries to extract the relevant data from the html. Hence why when Amazon changes their web page layout you see people complaining about their metadata downloads not working...
kiwidude is offline   Reply With Quote
Old 03-18-2012, 03:04 AM   #6
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Thank you. Then I know what to do I was wondering if there was some special API bits that I wasn't privy too.

I appreciate your comments!
Dionysus is offline   Reply With Quote
Old 03-22-2012, 04:25 PM   #7
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
It's funny, but when I parse the Amazon search page, I get all the results back but "Pricing information not available". The same page when viewed in a browser returns pricing, but when viewed via a file_get_contents does not!
Dionysus is offline   Reply With Quote
Old 03-23-2012, 08:37 AM   #8
PaulPosition
Enthusiast
PaulPosition began at the beginning.
 
PaulPosition's Avatar
 
Posts: 31
Karma: 16
Join Date: Feb 2012
Location: Montréal
Device: Sony PRS-T1
Dionysus - Pricing information may be available only based on information stored in a cookie (like country on a "real user browser"), it may only be available when you've followed some internal link (with maybe a session number attached, or a 'hidden' GET or POST field), or the website may check for user-agents and serve only recognized names such as Explorer or Chrome...

Most Calibre plugins code is available. If I'm not mistaken it's Python and not PHP, but having a look at one or two of the metadata-grabbing ones could be worth it for you.
PaulPosition is offline   Reply With Quote
Old 03-23-2012, 02:30 PM   #9
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Thanks Paul (and to all the rest of you) for your answer. I appreciate it. And hello from a fellow Montrealer
Dionysus is offline   Reply With Quote
Old 03-26-2012, 04:29 AM   #10
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Well Paul, you were right. I added this code :

ini_set('user_agent', $_SERVER['HTTP_USER_AGENT']);

And after that prices were displayed. However, they are not the right prices so I am betting you're also right about the country not being passed. But I'm still trying to figure out how I could do that.

Thanks for your help and if you know the answer to this, I'd appreciate it.
Dionysus is offline   Reply With Quote
Old 03-26-2012, 05:50 AM   #11
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,220
Karma: 1333994
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Why don't you just look at calibre's amazon store plugin(s) source code? Since that as you already know has the necessary logic to retrieve prices.
kiwidude is offline   Reply With Quote
Old 03-26-2012, 02:44 PM   #12
Dionysus
Junior Member
Dionysus began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Mar 2012
Device: iPhone
Actually, I would if I knew where I should look. I'm on a mac and I don't know where the plugins are. I'll do some searching online to find out where.

Thanks for reminding me.
Dionysus is offline   Reply With Quote
Old 03-26-2012, 02:59 PM   #13
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,220
Karma: 1333994
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Just download the calibre source, and all the store plugins (including the 6 amazon ones) are within:
src\calibre\gui2\store\stores
kiwidude is offline   Reply With Quote
Reply

Tags
amazon, api, calibre, get books

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PB903: Calibre cannot see preloaded books, and only sends to "books" paola PocketBook 4 02-28-2011 03:37 AM
PB903: Calibre cannot see preloaded books, and only sends to "books" paola Devices 2 02-27-2011 03:28 PM
Yep. It's official. Sony Reader has "ruined" books for me. A final "review." WilliamG Sony Reader 48 01-14-2011 03:49 AM
How to remove "Fully read" books from "Last Open" list? pjeanetta PocketBook 4 12-08-2010 10:30 AM
iLiad Nice little "make people curious" message! Aenea iRex Developer's Corner 49 01-30-2008 03:50 AM


All times are GMT -4. The time now is 02:46 PM.


MobileRead.com is a privately owned, operated and funded community.