[Conversion Input] Microsoft Doc Input Plugin

[rebl]

If you only have one or a few files, it's easy to convert DOC->DOCX (or DOC->ODT using OppenOffice), but if you want to apply calibre's settings and do it in bulk, this plugin would be very hand. Unfortunatelly on my Win7 x64 system with Office 2010 installed I can't use the plugin in bulk because I can't change the setting of wordconv path.
Aditeddly I still haven't got the time to install fileformatconverters (the MS File Formats Compatiblity Pack or what's its name) but even if I do it it will install in "Program Files (x86)" because it's a 64 bit system, and the path the plugin is using contains just "program files". Unless we can really edit, change and SAVE a different path, I don't think it would work.

[BetterRed]

@rebl - you probably don't need this PI if you can convert your .doc files to .docx

Using these tools ==>> Bulk Convert DOC to DOCX I converted about 27,000 .doc files last year to .docx in a few hours. I did it shortly after Kovid implemented built in DOCX Input module.

I then used calibre to convert the .docx files to EPUB. By and large the resulting EPUBS were far better than any I ever got from .RTF, Filtered HTML or .DOC via this PI - by better I mean a less 'crud' and far easier to read HTML.

BR

[rebl]

@BetterRed - thank you for the link, I've also read that page before writing here, but I was hesitant do go that route because it needed something extra to install and also some of the users have reported problems (with files size and others if I recall correctly).
For the moment I gave up because I still have tons of books to sort and rename outside of Calibre before adding.
I guess I'll cross that bridge when I get to it. Maybe meanwhile the new versions of calibre and plugin will work.

[davidfor]

Originally Posted by rebl

But a problem still persists- the usefulness of the select location / browse for file dialogue.
I've manally copied the wordconv.exe to a different path, just to test, and the behavior is the same as noticed yesterday with the older version of the plugin: the dialogue allows me to select a different location, it correctly lists the location in the textbox (and also allows me to manually edit/input the path) but after applying it actually does not save the change and the default path is still there.
That behaviour is in the Preferences / input / doc dialogue and not in the Settings of the Conversion dialogue. The two dialogues are very similar, but one is in the Preferences and should set the .. well.. preference forever, while the other one is in the convert dialoge and I suppose is a temporary setting valid for the active conversion.

I've had another look at this. For some reason, in the preferences dialog, the "DOC Input" is listed twice as the first and last items. If I try to change path using the first one, I see what you describe. But, if I use the second one, it works correctly. I don't know if this is something others are seeing, or just something weird on my system.

I also found an issue with the file selection in calibre v2. There was an error when you cancelled the file selection. And possibly when selecting the file. This version will work with both v1.48 and v2.

[davidfor]

Let's try this again.

While fixing another plugin, I stole some code from the changes I made here. I discovered these changes didn't work with calibre v1. I fixed it there and copied the changes back to DOC Input. The problem was that selecting the conversion file didn't update the field properly. This is fixed and working in both versions of calibre.

If someone could test this and tell me if it is OK, I can get the official version replaced.

[rebl]

Thanks a million, I'm going to try it this afternoon and report back.

L.E. On my 64 bit win7 system I can't change the path in the preferences/input.
I've even installed fileformatconverters, and now I have two wordconv.exe files (they have same size).
Since this is a 64 bit sytem, the location for 32-bit programs is Program Files (x86) and the location of wordconv.exe is:
C:\Program Files (x86)\Microsoft Office\Office12\wordconv.exe
But the plugin keeps changing it to C:\Program Files\Microsoft Office\Office12\wordconv.exe
Does it matter I'm running calibre64 but the wordconv is 32bit?

[BetterRed]

Originally Posted by rebl

Does it matter I'm running calibre64 but the wordconv is 32bit?

@rebl - No

Try copying C:\Program Files (x86)\Microsoft Office\Office12\wordconv.exe to C:\Program Files\Microsoft Office\Office12\wordconv.exe

The two Program Files directories are a mere convention.

BR

[rebl]

BetterRed, thank you again for your efforts. I was very optimistic, since the conversion worked with single files using the previous calibre and plugin versions and only the wrong path prevented the bulk operation to work. So I've copied the wordconv.exe in the required directory (how didn't I think of this trick?) but the result was the error below.
Then I tested again with only one file and it's the same.

calibre, version 2.0.0 (win32, isfrozen: True)
Conversion Error: Failed: Convert book 1 of 1 (Gxxxx)

Convert book 1 of 1 (Gxxxx)
Resolved conversion options
calibre version: 2.0.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0.0,
'book_producer': None,
'change_justification': u'original',
'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|pr ologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']",
'chapter_mark': u'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'docx_no_cover': False,
'dont_compress': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': u'',
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000000002013550>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0.0,
'linearize_tables': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': u'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindleOutput object at 0x0000000002013B00>,
'page_breaks_before': u"//*[name()='h1' or name()='h2']",
'personal_doc': u'[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': u'C:\\Users\\rebl\\AppData\\Local\\Temp\\calibre_d dmeos\\almttb.opf',
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': u'',
'search_replace': '[]',
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': None,
'sr1_search': None,
'sr2_replace': None,
'sr2_search': None,
'sr3_replace': None,
'sr3_search': None,
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2,
'wordconv_exe_path': u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe'}
InputFormatPlugin: DOC Input running
on C:\Users\rebl\AppData\Local\Temp\calibre_ddmeos\cb kde7.doc
Convert doc C:\Users\rebl\AppData\Local\Temp\calibre_ddmeos\cb kde7.doc to docx via c:\Program Files\Microsoft Office\Office12\Wordconv.exe
Temp directory C:\Users\rebl\AppData\Local\Temp\calibre_ddmeos\lk zihndoc_input temp output fileC:\Users\rebl\AppData\Local\Temp\calibre_ddmeo s\lkzihndoc_input\cbkde7.doc.docx
Python function terminated unexpectedly
Command '[u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe', '-oice', '-nme', u'C:\\Users\\rebl\\AppData\\Local\\Temp\\calibre_d dmeos\\cbkde7.doc', u'C:\\Users\\rebl\\AppData\\Local\\Temp\\calibre_d dmeos\\lkzihndoc_input\\cbkde7.doc.docx']' returned non-zero exit status -1 (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1041, in run
File "site-packages\calibre\customize\conversion.py", line 241, in __call__
File "calibre_plugins.doc_input.__init__", line 56, in convert
File "subprocess.py", line 540, in check_call
subprocess.CalledProcessError: Command '[u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe', '-oice', '-nme', u'C:\\Users\\rebl\\AppData\\Local\\Temp\\calibre_d dmeos\\cbkde7.doc', u'C:\\Users\\rebl\\AppData\\Local\\Temp\\calibre_d dmeos\\lkzihndoc_input\\cbkde7.doc.docx']' returned non-zero exit status -1


Update:
I've tried performing the conversion for a single file, this time modifying the path to any of the two locations of wordconv.exe in Program Files x86:
C:\Program Files (x86)\Microsoft Office\Office14\wordconv.exe
and
C:\Program Files (x86)\Microsoft Office\Office12\wordconv.exe
Both of them worked.

Maybe wordconv.exe requires some kind of registration like when registering controls or dlls? Anyway it doesn't seem to like being run from a different location that the one it was installed to.
So I guess the solution would be to be able indeed to specify a different path in Preferences/Input. Currently, after browsing to the new location, the path appears in the filed but after applying changes it is not saved.

[BetterRed]

Originally Posted by rebl

BetterRed, thank you again for your efforts. I was very optimistic, since the conversion worked with single files using the previous calibre and plugin versions and only the wrong path prevented the bulk operation to work. So I've copied the wordconv.exe in the required directory (how didn't I think of this trick?) but the result was the error below.
Then I tested again with only one file and it's the same.

...

Update:
I've tried performing the conversion for a single file, this time modifying the path to any of the two locations of wordconv.exe in Program Files x86:
C:\Program Files (x86)\Microsoft Office\Office14\wordconv.exe
and
C:\Program Files (x86)\Microsoft Office\Office12\wordconv.exe
Both of them worked.

Maybe wordconv.exe requires some kind of registration like when registering controls or dlls? Anyway it doesn't seem to like being run from a different location that the one it was installed to.
So I guess the solution would be to be able indeed to specify a different path in Preferences/Input. Currently, after browsing to the new location, the path appears in the filed but after applying changes it is not saved.

@rebl - It would appear the PI is trying to use
'wordconv_exe_path': u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe'

You could right about wordconv.exe being 'locked' into the location into which it was installed, Office is a cantankerous thing. On my system it appears in the Default value of ...Wow6432Node\Classes\CLSID\{BC7ADC2B-CC8C-48d2-A820-1BC605B0D3C7}\LocalServer32 as c:\\Program Files (86)\\Microsoft Office\\Office12\\Wordconv.exe.

Aaaah, I just installed DOC Input. I think I see the problem, the PI doesn't appear to save the location of wordconv.exe. I would expect it to create a file named DOC Input.json in the %CALIBRE_CONFIG_DIRECTORY%\plugins in which it would persist the path to wordconv.exe - but I don't see any such thing.

So every time it executes it reverts back to its default location ==>> c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe

I suspect the PI was written and tested on a 32 bit (maybe XP) environment.

Maybe David can tweak the PI to get it to save the location of wordconv.exe in its own json file.

Office would assume that any program wanting to invoke wordconv.exe would get its location from the Windows Registry, via its 'well known' CLSID - Ψ²*∞

BR

[davidfor]

Originally Posted by BetterRed

@rebl - It would appear the PI is trying to use
'wordconv_exe_path': u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe'

You could right about wordconv.exe being 'locked' into the location into which it was installed, Office is a cantankerous thing. On my system it appears in the Default value of ...Wow6432Node\Classes\CLSID\{BC7ADC2B-CC8C-48d2-A820-1BC605B0D3C7}\LocalServer32 as c:\\Program Files (86)\\Microsoft Office\\Office12\\Wordconv.exe.

Aaaah, I just installed DOC Input. I think I see the problem, the PI doesn't appear to save the location of wordconv.exe. I would expect it to create a file named DOC Input.json in the %CALIBRE_CONFIG_DIRECTORY%\plugins in which it would persist the path to wordconv.exe - but I don't see any such thing.

So every time it executes it reverts back to its default location ==>> c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe

I suspect the PI was written and tested on a 32 bit (maybe XP) environment.

Maybe David can tweak the PI to get it to save the location of wordconv.exe in its own json file.

Office would assume that any program wanting to invoke wordconv.exe would get its location from the Windows Registry, via its 'well known' CLSID - Ψ²*∞

BR

Are you using the last version I posted? This fixed a problem I found with saving the file location. Plus, are you changing the setting during the conversion or from the preferences? If you do it during the conversion, it is for that conversion only and overrides what is in the preferences if that gets changed.

I will admit to cheating in my testing. I don't have the conversion program installed. I simply created a file with the correct name in a few places. But, both versions of calibre save the location with no problems.

[BetterRed]

Originally Posted by davidfor

Are you using the last version I posted? This fixed a problem I found with saving the file location. Plus, are you changing the setting during the conversion or from the preferences? If you do it during the conversion, it is for that conversion only and overrides what is in the preferences if that gets changed.

I will admit to cheating in my testing. I don't have the conversion program installed. I simply created a file with the correct name in a few places. But, both versions of calibre save the location with no problems.

David,

The version I installed was the one in post #20

I just double checked, I changed it in Preferences to "...Program Files (86)\Microsoft Office\..." I then went to another book and it was showing "...Program Files\Microsoft Office\... Just to be sure I clicked Restore Defaults, nothing changed. So I went to Preferences and it was the same there - "...Program Files\Microsoft Office\..."

Where is the default setting saved ? As far as I see when I change the default nothing is written in the %CALIBRE_CONFIG_DATA%\calibre\plugins, the %CALIBRE_CONFIG_DATA%\calibre directories, nor in the relevant metadata.db.

If I change the path on an individual book then then metadata.db is updated and when I go back to that book the correct path is there.

Bottom line - the path sticks if its changed at the book level, it doesn't stick if the default is changed

BR

[BetterRed]

@David I did not pay enough attention to this post

Originally Posted by davidfor

I've had another look at this. For some reason, in the preferences dialog, the "DOC Input" is listed twice as the first and last items. If I try to change path using the first one, I see what you describe. But, if I use the second one, it works correctly. I don't know if this is something others are seeing, or just something weird on my system.

I see it twice, I didn't notice the one at the bottom of the list (it was out of sight because of window size), but when I change it there it seems to stick.

It would be good to get rid of that extra item in Preferences, especially as the spurious item is the one at the top of the list.

And where are the PI's preferences written - should there not be a DOCX Input.json file in the plugins folder?

@David - I'm going to install the version from post#1 into portable 1.48, I'll let you know what happens. I have a 'gut' feeling this might be an old problem.

@rebl - see first screen shot, when I changed the one at the bottom, the path to wordconv.exe seemed to persist.

Added - @David - I see DOC Input twice with original PI on 1.48 - same problems too. See 2nd attachment.

I did install this PI for a brief while, IIRC the actual conversions were not much better than RTF so I put it aside. And shortly thereafter Kovid released his DOCX which has given me sterling silver service ever since.

BR

[davidfor]

Originally Posted by BetterRed

@David I did not pay enough attention to this post



I see it twice, I didn't notice the one at the bottom of the list (it was out of sight because of window size), but when I change it there it seems to stick.

It would be good to get rid of that extra item in Preferences, especially as the spurious item is the one at the top of the list.

And where are the PI's preferences written - should there not be a DOCX Input.json file in the plugins folder?

David, I'm going to install the version from post#1 into portable 1.48, I'll let you know what happens. I have a 'gut' feeling this might be an old problem.

@rebl - see screen shot, when I changed the one at the bottom, it seems to persist.

BR

It's good to hear that the duplicate entry problem isn't just me. When I noticed it the other day, I couldn't see why. I'll have another look.

[BetterRed]

@rebl & davidfor - I found where the path to wordconv.exe is written, its in :

%calibre_config_directory%\conversion\doc_input.py and it contains

{
	u'docx_no_cover' : False,
	u'wordconv_exe_path' : u'C:/Program Files (x86)/Microsoft Office/Office12/Wordconv.exe',
}

I was watching the plugins directory Found it when I searched the config directory for the path string with Ransack. I'm a bit OCD re knowing where my data is kept.

Phew - all that for a measly 5 bytes - '(x86)'

BR

[davidfor]

Yes, I should have looked there.

But, more importantly, I have worked out what is causing the double entry in conversion options. There was a fix to how the configuration widgets for the input profiles were loaded made at the end of June. The fix was actually made by igi. But, he hasn't updated the plugin. He also doesn't seem to have been around since then.

Attached is the fixed version. I have done some testing with both v2.0 and v1.48. It won't work with a version from before the 30th of June. It would be possible to fix this, but I would need to hear a lot of screams and justification before I did.

This version is quite ready for release. There are some debug statements and some commented out code that should be removed first. If you can test it and there are no problems, I will do the cleanup.