Automatic convert webpage to epub? - Page 2

DoctorOhh · 03-26-2010, 08:31 AM

Quote:

Originally Posted by bthoven

web2disk http://uncle-rods.blogspot.com -r 0
ping -n 60 127.0.0.1 >NUL
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover

"http://skywatch.brainiac.com/astroland/rodnow.jpg"

I use Windows XP and got it to run consistently from a bat file with the following.

Code:

start web2disk http://uncle-rods.blogspot.com  -r 0 |more
start ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"

bthoven · 03-26-2010, 09:15 AM

Quote:

Originally Posted by chaley

Hmmm....

On windows 7, I created a batch file containing

Code:

web2disk http://uncle-rods.blogspot.com  -r 0
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"

Running that batch file from a command box, I get

Code:

C:\Users\charles\Desktop\foo>run.bat

C:\Users\charles\Desktop\foo>web2disk http://uncle-rods.blogspot.com  -r 0
.

C:\Users\charles\Desktop\foo>ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover
"http://skywatch.brainiac.com/astroland/rodnow.jpg"
Downloading cover from u'http://skywatch.brainiac.com/astroland/rodnow.jpg'
1% Converting input to HTML...
InputFormatPlugin: HTML Input running
on C:\Users\charles\Desktop\foo\index.xhtml
Language not specified
Creator not specified
Building file list...
IgnoreFile(u'Could not read from file: c:\\"http:\\uncle-rods.blogspot.com\\" with error: (22, "invalid mode (\'rb\') or filename"
)',)
Initial parse failed:
Parsing file 'index.xhtml' as HTML
Forcing index.xhtml into XHTML namespace
Stripping comments and meta tags from index.xhtml
Merging multiple <head> and <body> sections
[lots of random warnings deleted]
Creating EPUB Output...
67% Creating EPUB Output
        Looking for large trees in index.xhtml...
        No large trees found
        Split into 6 parts
EPUB output written to C:\Users\charles\Desktop\foo\unclerod.epub
Output saved to   C:\Users\charles\Desktop\foo\unclerod.epub
C:\Users\charles\Desktop\foo>

As it works for me, you must be having some environment-specific problem.

Some questions:

1. What version of windows are you running? (Windows XP SP3)
2. If run the commands one at a time in a foreground command box, does it work? (Yes)
3. If the answer to #2 is yes, does running your batch file in the same foreground command box (not with the task scheduler) work? Be sure that your working directory is the same as in #2.
(Yes, I run both commands in foreground. If I split each command into separate batch files, and run each batch file one at a time in sequence, it works)

My suspicion is that the problem comes from the current directory being something other than what you expect, especially if you are using a shortcut to the batch file or if the batch file lives in a normally-readonly folder. One thing you can try is to add as the first line of the batch file a cd to the folder where the output files are expected to end up.

Please see my answers inserted in your original quote.

bthoven · 03-26-2010, 09:29 AM

Quote:

Originally Posted by dwanthny

I use Windows XP and got it to run consistently from a bat file with the following.

Code:

start web2disk http://uncle-rods.blogspot.com  -r 0 |more
start ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"

Hey...thanks a lot. Using your commands work every time!..great!

chaley · 03-26-2010, 11:19 AM

Quote:

Originally Posted by dwanthny

Code:

start web2disk http://uncle-rods.blogspot.com  -r 0 |more
start ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"

Interesting. It seems that the 'start' command creates a new process then executes the specified command in it, which would isolate a command's strange behavior. My guess is that Windows 7 automatically starts subprocesses, which is why I don't have the problem.

There is one experiment I would like to try, but I don't have access to an XP machine. My assumption is that the '|more' is there to ensure that the next 'start' does not execute until the first one finishes. I imagine that it works because the 'more' would execute under the control of the original cmd processor. What I wonder is whether using 'start /wait web2disk ...' without piping the output through 'more' would do the same thing. If someone with XP has the time and inclination, could you try it and post the results?

Starson17 · 03-26-2010, 01:28 PM

Quote:

Originally Posted by chaley

Interesting. It seems that the 'start' command creates a new process then executes the specified command in it, which would isolate a command's strange behavior.

Yes, the new process isolates the problem. It took me half a day to find the "start" workaround. There's a bug ticket on this, that describes another workaround from Kovid using PowerShell.

Quote:

My guess is that Windows 7 automatically starts subprocesses, which is why I don't have the problem.

That's interesting, because Vista does have the problem.

Quote:

There is one experiment I would like to try, but I don't have access to an XP machine. My assumption is that the '|more' is there to ensure that the next 'start' does not execute until the first one finishes.

Using "more" is a nice trick that I hadn't seen. Simultaneous execution was a problem with using "start." I worked around it with a less subtle hack (loop waiting for output from the command above).

Quote:

What I wonder is whether using 'start /wait web2disk ...' without piping the output through 'more' would do the same thing. If someone with XP has the time and inclination, could you try it and post the results?

I'd be interested in the results, too. I may test on Vista when I get a chance. Anyone who knows that this works may want to hunt up the bug report and post the "more" workaround, with the others, to keep them all in a single spot.

bthoven · 03-26-2010, 02:47 PM

Quote:

Originally Posted by Starson17

Yes, the new process isolates the problem. It took me half a day to find the "start" workaround. There's a bug ticket on this, that describes another workaround from Kovid using PowerShell.

That's interesting, because Vista does have the problem.

Using "more" is a nice trick that I hadn't seen. Simultaneous execution was a problem with using "start." I worked around it with a less subtle hack (loop waiting for output from the command above).

I'd be interested in the results, too. I may test on Vista when I get a chance. Anyone who knows that this works may want to hunt up the bug report and post the "more" workaround, with the others, to keep them all in a single spot.

Hi..on Winxp, the following commands also worked:

start /wait web2disk http://uncle-rods.blogspot.com -r 0
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg" --level1-toc //h:h2 --no-chapters-in-toc

Starson17 · 03-26-2010, 04:06 PM

Quote:

Originally Posted by bthoven

Hi..on Winxp, the following commands also worked:

start /wait web2disk http://uncle-rods.blogspot.com -r 0
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg" --level1-toc //h:h2 --no-chapters-in-toc

I'm not sure, but I think there's some confusion in this thread, some of which I may have contributed to. I did a lot of testing when looking for a workaround, and it's the Calibre programs (calibredb is what I tested) that have bat file problems, and there are problems only when another command follows them in the bat file.

IOW, I'd expect

web2disk ....
ebook-convert ....

to work in a bat file, but not:

ebook-convert ....
web2disk ....

The problem is that the bat file dies after the Calibre executable is done.

Does W7 run bat files to the end when the Calibre executable is followed by another executable or only when it is preceded by a non-Calibre executable? The latter always works for me regardless of the use of "start."

chaley · 03-26-2010, 07:50 PM

Quote:

Originally Posted by Starson17

I did a lot of testing when looking for a workaround, and it's the Calibre programs (calibredb is what I tested) that have bat file problems, and there are problems only when another command follows them in the bat file.

One source of the confusion could be that web2disk is in fact a calibre program. The source is in "src\calibre\web\fetch\simple.py"

Quote:

IOW, I'd expect

web2disk ....
ebook-convert ....

to work in a bat file, but not:

ebook-convert ....
web2disk ....

The problem is that the bat file dies after the Calibre executable is done.

Does W7 run bat files to the end when the Calibre executable is followed by another executable or only when it is preceded by a non-Calibre executable? The latter always works for me regardless of the use of "start."

For me running W7, both permutations work. I also tried both permutations at once, as in:

Code:

del index.xhtml
web2disk http://uncle-rods.blogspot.com  -r 0
ebook-convert index.xhtml unclerod1.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"
del index.xhtml
web2disk http://uncle-rods.blogspot.com  -r 0
ebook-convert index.xhtml unclerod2.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg"

and it worked. The output in the cmd box is correct, and I get both unclerod1.epub and unclerod2.epub. It seems that W7 cmd processing is in fact different from both XP and Vista.

Starson17 · 03-26-2010, 09:28 PM

Quote:

Originally Posted by chaley

One source of the confusion could be that web2disk is in fact a calibre program.

(I did look it up to see if it was a Calibre program, but I must have found another program.)

DoctorOhh · 03-26-2010, 10:12 PM

Quote:

Originally Posted by bthoven

Hi..on Winxp, the following commands also worked:

start /wait web2disk http://uncle-rods.blogspot.com -r 0
ebook-convert index.xhtml unclerod.epub --authors "Rod Mollise" --author-sort "Mollise, Rod" --cover "http://skywatch.brainiac.com/astroland/rodnow.jpg" --level1-toc //h:h2 --no-chapters-in-toc

For future reference this is the better way to do it.

kovidgoyal · 03-27-2010, 12:33 AM

sorry I should have chimed in earlier, yes web2disk is calibre program and on xp, vista you have to use start in batch files if you are executing multiple calibre programs, I have absolutely no idea why.

thingumybob · 07-08-2010, 08:03 PM

Came across this online conversion option -- seems to fit the need to convert a blog, as well as -- it seems -- most other online content

www.zinepal.com

bthoven · 10-27-2010, 05:29 AM

Sorry to dig this thread up.

I want to further enhance the batch file to include sending the ePUB output file to my yahoo mail. Would appreciate all your suggestion.

Nexutix · 10-27-2010, 05:38 AM

Hey ! Simple! Use Instapaper!

bthoven · 10-27-2010, 05:55 AM

Quote:

Originally Posted by patilsaurabhr

Hey ! Simple! Use Instapaper!

Thanks. How do I automate uploading this epub file to instapaper?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How much is your webpage worth?	ProDigit	Lounge	33	07-03-2010 04:28 PM
Classic Automatic Webblog conversion to epub by Calibre	bthoven	Barnes & Noble NOOK	2	03-25-2010 07:18 AM
Want automatic convert on upload	eolake	Calibre	2	02-02-2010 02:29 AM

03-27-2010, 12:33 AM	#26
kovidgoyal creator of calibre Posts: 45,609 Karma: 28549044 Join Date: Oct 2006 Location: Mumbai, India Device: Various	sorry I should have chimed in earlier, yes web2disk is calibre program and on xp, vista you have to use start in batch files if you are executing multiple calibre programs, I have absolutely no idea why.

07-08-2010, 08:03 PM	#27
thingumybob Junior? JUNIOR?! OK :-) Posts: 10 Karma: 42 Join Date: Jun 2010 Device: Sony PRS 600	Came across this online conversion option -- seems to fit the need to convert a blog, as well as -- it seems -- most other online content www.zinepal.com

10-27-2010, 05:29 AM	#28
bthoven Evangelist Posts: 475 Karma: 590 Join Date: Aug 2009 Location: Bangkok, Thailand Device: Kindle Paperwhite	Sorry to dig this thread up. I want to further enhance the batch file to include sending the ePUB output file to my yahoo mail. Would appreciate all your suggestion.

10-27-2010, 05:38 AM	#29
Nexutix Reading and reading Posts: 582 Karma: 8250144 Join Date: Oct 2010 Device: Infibeam Pi, iPod Touch 4G, iPad Air 2, iPad mini 2, Oneplus One	Hey ! Simple! Use Instapaper!