Luzme.com: News, Comments and Requests for Help

[rwillmer]

Thanks to all of you who send me messages saying that this is a useful site, that's really good to hear.

But let me ask those of you who have found problems with the site - please tell me about them! Don't keep it to yourselves :-)

Quick summary of my requests:

1) If you find problems with my site, please tell me (via this forum, my website's forum, my website's contact page, or via private message - I don't care, just let me know if there's a problem!)

2) If you find it frustrating that you have to wait for up-to-date prices from Fictionwise, BookOnBoard or the Sony Store from my website, please ask them to give me a datafeed.

OK, now the longer version....

I find it very frustrating when I browse websites and find someone saying "I looked for Book X on ebookprice.info and the site didn't work for me" or "the website crashed" but they either haven't told me separately that there was a problem; or don't even give enough information in their post to help me work out the problem myself (for example, someone yesterday in this forum who didn't even give the name of the book they were struggling to find).

If you tell me about a problem, I will try to fix it.

Sometimes it takes a long time to work out how to do that - for example, making it clear that a book with one title from one vendor is actually the same book as another one with a different title from a different vendor - but if I know about the problem, it will help me work out how to do it better. (That naming problem is quick to fix for a single book but there's nearly 400 thousand books in my database, and I can't check them one by one.)

Sometimes, it's really quick! For example, someone yesterday posted a message on my website's forum which let me realise that the ebooks.com code has stopped working - basically, they'd changed the format on the website, so the wrong info was being picked up by my scripts. Problem solved within 30 mins, and ebooks.com is now working again (thank you!)

For those of you interested in the technical stuff going behind the scenes: this is a really complicated problem. Some booksellers supply a datafeed so I can guarantee that when you hit the website, if the book is there in my database, the data is the most up-to-date. But even when the booksellers have the same book in their datafeed, there's no guarantee that they'll call it by the same name or with the same author.

For example, "Flight Of The Nighthawks : Book One Of The Darkwar Saga" by "Ray Feist", is this the same book as "Flight of the Nighthawks" by "Raymond E. Feist"? To the naked eye, it's clear it is. But to a computer script, it's not.

Others - for example, Fictionwise, BooksOnBoard (two of the most popular with the best prices!) - don't even provide me a datafeed.

And some have a datafeed available, but the prices in it aren't accurate, they don't match the prices showing on the booksellers websites.

So for those sites, I have to screen-scrape (in a responsible manner!) - but I can't deluge their sites with hundreds of requests a second to get the best response for you, my website's user. For one thing, that's not ethical; for another, if their sys admins are on the ball, they'd just cut me off completely.

If you want to help with that, please contact their websites and ask them to make a datafeed available to me - then I could give you the up-to-date prices for them as soon as possible.

Until then, I'm afraid I will have to queue requests to their website so that I only don't overwhelm their websites with your requests. Which means, sometimes you have to wait.

OK, rant over. Just please let me know if you find a problem :-)

Rachel

[goldilocks]

Hi Rachel, I don't think the problem I had at ebookprice.info was the sites' fault. It found the book I was searching for then I closed the webpage. When I tried to go back in it wouldn't work. I got out of Internet Explorer and everything worked fine when I got back in. In fact the site found more stores selling the book I was looking at the second time around. I did notice it was missing the Kindle version (but I don't have a Kindle). The book I searched for was "Drawn in Blood" by Andrea Kane. I sure get a lot of responses to the search now!

for a very useful site. I will be sure to use it in the future. If I have any problems with your site I will let you know.

[rwillmer]

Originally Posted by goldilocks

The book I searched for was "Drawn in Blood" by Andrea Kane.

Ah! that was *your* search!

No, you had in fact hit a bug in the site there - every time you attempted to go to the book page for "Drawn in Blood", the site was falling over.

An obscure bug - it was caused by one of the booksellers having an utterly invalid character in the middle of the name of one of the other books written by this author. So I've added in some extra error handling to cope with similar cases in the future, glad it's working for you now.

Do please let me know if you find problems in the future - it really helps!

thanks
Rachel

[JSWolf]

I'm finding the site is missing eBooks that I know are out there.

Searching for Green Rider by Kristen Britain misses a lot of them (for example).

[rwillmer]

Originally Posted by JSWolf

I'm finding the site is missing eBooks that I know are out there.

Searching for Green Rider by Kristen Britain misses a lot of them (for example).

Yes, you're right - something strange is going on there. I can see, for example, that there is a Kindle for that book which isn't being found.

Thanks, I'll investigate.

Rachel

[goldilocks]

Thanks Rachel. Great job!

[JSWolf]

Originally Posted by rwillmer

Yes, you're right - something strange is going on there. I can see, for example, that there is a Kindle for that book which isn't being found.

Thanks, I'll investigate.

Rachel

Thank you. And I really do appreciate the effort.

[kennyc]

I'm just going to say thanks! It looks great.

I'll let you know if I run into any issues.

[rwillmer]

Originally Posted by rwillmer

Yes, you're right - something strange is going on there. I can see, for example, that there is a Kindle for that book which isn't being found.

Thanks, I'll investigate.

Rachel

Aha! Found the problem...

Basically I switched ISP earlier this week and this meant that the website database wasn't getting fed the changes from my master database because the new IP address was being blocked by the website's firewall.

Config fixed now, it will take a few hours for the replication to get back up-to-date.

Rachel

[Terisa de morgan]

Originally Posted by rwillmer

2) If you find it frustrating that you have to wait for up-to-date prices from Fictionwise, BookOnBoard or the Sony Store from my website, please ask them to give me a datafeed.

Ok, so I think this is my problem. I live in Spain, I browse the books usually when I get up in the morning (early morning) the day they're published (I think it's much earlier in USA), so I think, if you can't do it automatically, they won't be yet.

[benham]

Same basic situation as JS. I searched for The Dark Horse by Craig Johnson and it only showed Powells even though Sony Estore has it for about $6 less.

[Elfwreck]

Grumbles about ebookpriceinfo (and these are *my* grumbles; some will be totally irrelevant to making the site functional for lots of people):

1) Incredibly dialup unfriendly. Can use it at work; basically useless at home. No point in having a price comparison website that takes longer to load than going to half a dozen ebook stores separately.

2) Site in general unfriendly. Hard to tell what it's for; hard to tell what the best format is for searching for books (title only? Title: lastname, firstname? Something else?)

3) Can't search for "1984;" title is too short. Orwell's book is not the only ebook with a name under 6 characters long, just the one I noticed recently.

4) Tends to turn up 1 result per site, even if the sites have multiple versions. (Amazon is prone to having multiple ebook versions, esp. of public domain novels.)

5) Books often listed several times with slightly different name formats. Sometimes Title - Author; sometimes Title - Lastname, Firstname; sometimes Title (Author); sometimes Title by Author, and so on. Each is listed separately.

I know this is because the ebook stores list them that way. Doesn't matter; it still makes it hard for me to compare prices. Understanding the technical issues (and I do, and am very sympathetic) doesn't make the site more useful.

6) Would like the site to indicate which ebook stores it searches; this would let me know that if it didn't turn up a book from ebooks.com, there's something wrong with the scrape code, but if it doesn't turn up results from coolerbooks (or whatever), it's because that's not on your list--and I'd know I should mention it.

7) Doesn't seem to pull any data from ebook publishers that aren't general ebook stores: Samhain press, Double Dragon, and so on. Not sure if this is something that there's any intention to fix; generally, those books are only available in one place or a small handful of ebook stores, and all at the same price.

8) Would like books to be listed by *filetype* not just *program.* I know some publishers don't make it clear whether their ADE books are PDF or ePub, but mentioning the possibilities would be nice. And it would help newbie ebook searchers know whether or not a particular book is usable on their device.

[JSWolf]

There is still a problem with Green Rider. Diesel eBooks has two different versions listed and only the more expensive one is showing up.

http://www.diesel-ebooks.com/cgi-bin/category.cgi?category=search&template=search&templ atehead=&templatebody=&templatefoot=&limitcategory =&type=store&searchtype=efts&query=^efts.sql&q1=gr een+rider&index=default

[rwillmer]

Thanks for taking the time for such a detailed grumblelist. This is really useful - I plan to take some time in October to sort out some of the major issues, so the more I know about what the problems are, the best use I can make of that time.

Originally Posted by Elfwreck

Grumbles about ebookpriceinfo (and these are *my* grumbles; some will be totally irrelevant to making the site functional for lots of people):

1) Incredibly dialup unfriendly. Can use it at work; basically useless at home. No point in having a price comparison website that takes longer to load than going to half a dozen ebook stores separately.

I'll sort this out. I'm rather spoiled with a fairly high speed connection at home, so have forgotten what it's like to use dialup. What speed link do you have?

2) Site in general unfriendly. Hard to tell what it's for; hard to tell what the best format is for searching for books (title only? Title: lastname, firstname? Something else?)

You have a point - I don't give any clue, do I? Can do something about that.

3) Can't search for "1984;" title is too short. Orwell's book is not the only ebook with a name under 6 characters long, just the one I noticed recently.

I've just removed that restriction. It was in place to stop searches like "Bible" from killing the database completely. But I've found a better way to do that.

4) Tends to turn up 1 result per site, even if the sites have multiple versions. (Amazon is prone to having multiple ebook versions, esp. of public domain novels.)

Am thinking about this.

5) Books often listed several times with slightly different name formats. Sometimes Title - Author; sometimes Title - Lastname, Firstname; sometimes Title (Author); sometimes Title by Author, and so on. Each is listed separately.

I know this is because the ebook stores list them that way. Doesn't matter; it still makes it hard for me to compare prices. Understanding the technical issues (and I do, and am very sympathetic) doesn't make the site more useful.

This is the biggest problem I have, and it drives me crazy as a user of the site too. I have an idea about how to fix it, or at least make it better; but it's going to take a while to do it.

6) Would like the site to indicate which ebook stores it searches; this would let me know that if it didn't turn up a book from ebooks.com, there's something wrong with the scrape code, but if it doesn't turn up results from coolerbooks (or whatever), it's because that's not on your list--and I'd know I should mention it.

The homepage lists all the stores searched, do you mean you'd like to see on each book page a list of the shops that this particular book was not found at?

7) Doesn't seem to pull any data from ebook publishers that aren't general ebook stores: Samhain press, Double Dragon, and so on. Not sure if this is something that there's any intention to fix; generally, those books are only available in one place or a small handful of ebook stores, and all at the same price.

If I can get a datafeed from a site, I'll use it. The smaller ebook stores don't appear to have such a thing available, and the few I've contacted directly haven't replied to my emails. If you can contact any stores you'd like to see in my site, and ask them to contact me to arrange a data feed, I'd be pleased to include them.

But hand-crafting the screen scraping for large numbers of small stores, plus the associated queueing resources for handling the scraping is a big overhead if they only have a few books.

8) Would like books to be listed by *filetype* not just *program.* I know some publishers don't make it clear whether their ADE books are PDF or ePub, but mentioning the possibilities would be nice. And it would help newbie ebook searchers know whether or not a particular book is usable on their device.

I've thought about doing this but it ends up making for a very complicated user interface. "This book might be this or it might be that". It's not just ADE, there's the Multiformat ones from Fictionwise too.

I don't want my site to be responsible for saying whether a particular book from a particular vendor will work on a particular device; there's too much scope for getting it wrong.

So I think this will stay in the "nice idea but not viable" pile. Although I always reserve the right to change my mind :-)

Thanks again for the list!
Rachel

[rwillmer]

Originally Posted by JSWolf

There is still a problem with Green Rider. Diesel eBooks has two different versions listed and only the more expensive one is showing up.

http://www.diesel-ebooks.com/cgi-bin/category.cgi?category=search&template=search&templ atehead=&templatebody=&templatefoot=&limitcategory =&type=store&searchtype=efts&query=^efts.sql&q1=gr een+rider&index=default

Not sure what's going on here, but it shouldn't be doing that. I'll investigate.
Thanks
Rachel