Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 03-23-2010, 11:18 PM   #1
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Automatic convert webpage to epub/mobi and auto send mail?

I have one favorite webblog which I would like to automatically download and convert it to epub.

http://uncle-rods.blogspot.com/

This is how I do it manually:

1. goto website web2fb2.com (give excellent result!)
2. paste the above website address, fillin title and author
3. press a "do it" button to start converting it to epub
4. download the output epub file to my local drive
5. either copy the epub file to my Nook or add it to Calibre before sending it to my Nook.

Is there a way to automate step 1-4 by using any builtin Calibre commands? If so, I'll create a batch file containing those command with proper parameters and schedule it to run everyday by using Windows Task Scheduler function.

Thanks

Last edited by bthoven; 11-12-2010 at 04:28 AM.
bthoven is offline   Reply With Quote
Old 03-23-2010, 11:35 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,121
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
web2disk is what you're looking for.
kovidgoyal is offline   Reply With Quote
Old 03-24-2010, 12:05 AM   #3
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Thanks Kovid..could you elaborate more? It seems this web2disk is just downloading the web content into my local disk. How about a command to convert html to epub?
bthoven is offline   Reply With Quote
Old 03-24-2010, 12:06 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,121
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
ebook-convert
kovidgoyal is offline   Reply With Quote
Old 03-24-2010, 12:20 AM   #5
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Thanks Kovid.

I tried web2disk with the above website and it creates the following files:

index.xhtml (79kb but it can't be opened with webbrowser; it seems to have unreadable content)
\links\ (no content in this folder)
\images\ (no content in this folder)
\stylesheet\ (no content in this folder)

Any reason why?
bthoven is offline   Reply With Quote
Old 03-24-2010, 01:30 PM   #6
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by bthoven View Post
Thanks Kovid.

I tried web2disk with the above website and it creates the following files:

index.xhtml (79kb but it can't be opened with webbrowser; it seems to have unreadable content)
\links\ (no content in this folder)
\images\ (no content in this folder)
\stylesheet\ (no content in this folder)

Any reason why?
Lots of possible reasons - could be cookies, scripts, referer (sic), useragent, etc. They probably don't want robots to grab content from this site.
Starson17 is offline   Reply With Quote
Old 03-24-2010, 02:48 PM   #7
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 5,631
Karma: 857074
Join Date: Jan 2010
Location: France
Device: Many android devices
I just tried fetching it using wget on windows 7. I got the correct index.html and was able to convert it.

Get wget at http://gnuwin32.sourceforge.net/packages/wget.htm

After installing it, and assuming you took the defaults, in a command box, run
Code:
"C:\Program Files\GnuWin32\bin\wget" -P c:\unclerod http://uncle-rods.blogspot.com/
cd c:\unclerod
ebook-convert index.html unclerod.epub
Change c:\unclerod to the directory you want the index.html to appear.
chaley is offline   Reply With Quote
Old 03-25-2010, 01:26 AM   #8
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Hi..thanks everyone

It was my mistake, Google Chrome can't display index.xhtml; but it displays fine in IE.

When I create a batch file having both web2disk and ebook-convert, the ebook-convert won't work! But if the batch file has ebook-convert command alone, it works fine. I'm not sure whether because ebook-convert start running when index.xhtml has still not been created. How can I set the delay time between web2disk and ebook-convert command?

Another question, how can I put the Title and Author into the output epub file automatically?
bthoven is offline   Reply With Quote
Old 03-25-2010, 03:21 AM   #9
Terisa de morgan
Wizard
Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.
 
Terisa de morgan's Avatar
 
Posts: 2,285
Karma: 1982066
Join Date: Jun 2009
Location: Madrid, Spain
Device: Kobo Mini, iPhone, iPad, Kobo Aura, Galaxy Tab3 8
Quote:
Originally Posted by bthoven View Post
Another question, how can I put the Title and Author into the output epub file automatically?
You can do it directly when you convert it with ebook-convert, with options --authors and --title. It's all described at calibre user-manual (very good manual, BTW )
Terisa de morgan is offline   Reply With Quote
Old 03-25-2010, 04:40 AM   #10
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Quote:
Originally Posted by Terisa de morgan View Post
You can do it directly when you convert it with ebook-convert, with options --authors and --title. It's all described at calibre user-manual (very good manual, BTW )
Thanks a lot Terisa.

I can now add author and cover into the epub when doing the ebook-convert.

Here is my post how I did it:

http://www.mobileread.com/forums/sho...043#post844043

Great software!

Last edited by bthoven; 03-25-2010 at 06:53 AM.
bthoven is offline   Reply With Quote
Old 03-25-2010, 10:04 AM   #11
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by chaley View Post
I just tried fetching it using wget
BTW, nothing to do with Calibre, but wget is a fantastic tool for automatically grabbing content from the web. I've been using it for years for my wife's computerized embroidery hobby to grab files from sites that provide them free, but only if you come back once every 3 days, or only if you hunt through 20 pages to find the page with the free file.
Starson17 is offline   Reply With Quote
Old 03-26-2010, 04:41 AM   #12
bthoven
Evangelist
bthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enoughbthoven will become famous soon enough
 
bthoven's Avatar
 
Posts: 452
Karma: 544
Join Date: Aug 2009
Location: Bangkok, Thailand
Device: Kindle'r 3, iPhone 3Gs, iPad 2, Galaxy Tab Wifi
Hi,

I create a batch file containing the following command lines:


web2disk http://uncle-rods.blogspot.com -r 0
ping -n 60 127.0.0.1 >NUL
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover

"http://skywatch.brainiac.com/astroland/rodnow.jpg"

The problem is after web2disk command was completed, the batch file just quit itself and never execute the next command lines.

I also tried this in another batch file:

ebook-convert index.xhtml unclerod.epub
ping -n 60 127.0.0.1 >NUL (to delay 60 seconds)

Same problem, ping command was never executed.

Any idea what is the cause of problem?
bthoven is offline   Reply With Quote
Old 03-26-2010, 05:34 AM   #13
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,070
Karma: 777825
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
I seem to remember it coming up elsewhere that the calibre tools do not play well with batch files. The symptom being what you state - that subsequent commands do not get executed because the command processor dies.

I seem to remeber that the best answer that anyone came up with was to that when you wanted to execute a new command you did it in a new instance of the command processor (using the CMD /C option).

Another solution was to use alternative command processors.
itimpi is offline   Reply With Quote
Old 03-26-2010, 06:36 AM   #14
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 5,631
Karma: 857074
Join Date: Jan 2010
Location: France
Device: Many android devices
Hmmm....

On windows 7, I created a batch file containing
Code:
web2disk http://uncle-rods.blogspot.com  -r 0
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"
Running that batch file from a command box, I get
Code:
C:\Users\charles\Desktop\foo>run.bat

C:\Users\charles\Desktop\foo>web2disk http://uncle-rods.blogspot.com  -r 0
.

C:\Users\charles\Desktop\foo>ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover
"http://skywatch.brainiac.com/astroland/rodnow.jpg"
Downloading cover from u'http://skywatch.brainiac.com/astroland/rodnow.jpg'
1% Converting input to HTML...
InputFormatPlugin: HTML Input running
on C:\Users\charles\Desktop\foo\index.xhtml
Language not specified
Creator not specified
Building file list...
IgnoreFile(u'Could not read from file: c:\\"http:\\uncle-rods.blogspot.com\\" with error: (22, "invalid mode (\'rb\') or filename"
)',)
Initial parse failed:
Parsing file 'index.xhtml' as HTML
Forcing index.xhtml into XHTML namespace
Stripping comments and meta tags from index.xhtml
Merging multiple <head> and <body> sections
[lots of random warnings deleted]
Creating EPUB Output...
67% Creating EPUB Output
        Looking for large trees in index.xhtml...
        No large trees found
        Split into 6 parts
EPUB output written to C:\Users\charles\Desktop\foo\unclerod.epub
Output saved to   C:\Users\charles\Desktop\foo\unclerod.epub
C:\Users\charles\Desktop\foo>
As it works for me, you must be having some environment-specific problem. Some questions:

1. What version of windows are you running?
2. If run the commands one at a time in a foreground command box, does it work?
3. If the answer to #2 is yes, does running your batch file in the same foreground command box (not with the task scheduler) work? Be sure that your working directory is the same as in #2.

My suspicion is that the problem comes from the current directory being something other than what you expect, especially if you are using a shortcut to the batch file or if the batch file lives in a normally-readonly folder. One thing you can try is to add as the first line of the batch file a cd to the folder where the output files are expected to end up.
chaley is offline   Reply With Quote
Old 03-26-2010, 07:12 AM   #15
slm
Useless
slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.slm ought to be getting tired of karma fortunes by now.
 
Posts: 230
Karma: 481862
Join Date: Feb 2003
Device: Kindle1, Kindle 2, Jetbook, EZ Reader EZ pocket pro, ipad, archos 5 ch
on another thread, the problem of running several lines of batch code for Calibre was discussed. There someone who seemed to know said that the batch file doesn't wait for the first line call to finish but that you can do multiple lines by making each line begin with a "call" statement.

I have no idea what I'm talking about.
slm is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How much is your webpage worth? ProDigit Lounge 33 07-03-2010 03:28 PM
Classic Automatic Webblog conversion to epub by Calibre bthoven Barnes & Noble NOOK 2 03-25-2010 06:18 AM
Want automatic convert on upload eolake Calibre 2 02-02-2010 01:29 AM


All times are GMT -4. The time now is 09:19 AM.


MobileRead.com is a privately owned, operated and funded community.