[GUI Plugin] Marvin XD (closed)

[hymie]

Originally Posted by Philantrop

You treated the symptom but didn't cure the disease. The next update will break things again.

I am aware of that. But at least i can use it now.

[kovidgoyal]

You should change your system encoding from ISO-8859-1 to UTF-8. While the details of doing that will differ from distro to distro, usually, launching calibre like this:

LANG=en_US.UTF-8 calibre

will do the trick. (If you use some other language than english, change the en_US part accordingly)

[At_Libitum]

Originally Posted by GRiker

I will take a look at the issue you posted, thanks for the detailed description.

G

problem solved. much obliged.

[hymie]

Originally Posted by kovidgoyal

launching calibre like this:

LANG=en_US.UTF-8 calibre

will do the trick. (If you use some other language than english, change the en_US part accordingly)

Indeed. (The next question is, can I change it system-wide, or does it break something else...)

[kovidgoyal]

There's always a chance that changing it could break something -- because of linux's bone headed design of storing filenames as opaque binary blobs with no defined encoding.

Basically, if some application created files with non-ascii characters in their names using ISO-8859-1 then those filenames will become garbage if you switch your system to UTF-8. To make the migration safely you have to first search your system for files with non-ascii chars in their names, change their names to be utf-8 encoded and then make the switch. IIRC there are some blogs that outline how to do that process.

[ilany]

Help please...

I have a fully updated system both Calibre 1.4, Marvin 2.0, iOS reader application 1.1.2 & Marvin XD 1.0.6. My OS is Windows 7 64bit. Nothing out of the ordinary, really.
I have followed all the plugin installation procedures as advertised and to the letter. When my iPad is connected and I click the Marvin XD icon it initiates a scan of my libraries and then time and again gets stuck and delivers an error message at 23% in the process: 'identifying 156 books in Marvin Library'.

The error message:

calibre, version 1.4.0
ERROR: Unhandled exception: <b>IOError</b>:[Errno 22] invalid mode ('wb') or filename: u'L:\\Users\\IlanY\\AppData\\Local\\Temp\\calibre_ ber5up\\s2tt80_ios_local_db\\urn:uuid:934d610a-e1de-4ced-9d94-f9e367b438e4.epub'

calibre 1.4 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.marvin_manager.action", line 352, in main_menu_button_clicked
File "calibre_plugins.marvin_manager.action", line 665, in show_installed_books
File "calibre_plugins.marvin_manager.book_status", line 1191, in initialize
File "calibre_plugins.marvin_manager.book_status", line 3323, in _generate_booklist
File "calibre_plugins.marvin_manager.book_status", line 4158, in _get_installed_books
File "calibre_plugins.marvin_manager.book_status", line 4652, in _scan_marvin_books
File "calibre_plugins.marvin_manager.book_status", line 3179, in _fetch_marvin_content_hash
IOError: [Errno 22] invalid mode ('wb') or filename: u'L:\\Users\\IlanY\\AppData\\Local\\Temp\\calibre_ ber5up\\s2tt80_ios_local_db\\urn:uuid:934d610a-e1de-4ced-9d94-f9e367b438e4.epub'

I went over my library checking all uuid's and couldn't find the one in the message, I was thinking maybe the data on one of my books is corrupted and creates some conflict. Please help me, what should I do?

[GRiker]

Originally Posted by ilany

Help please...

Spoiler:

I have a fully updated system both Calibre 1.4, Marvin 2.0, iOS reader application 1.1.2 & Marvin XD 1.0.6. My OS is Windows 7 64bit. Nothing out of the ordinary, really.
I have followed all the plugin installation procedures as advertised and to the letter. When my iPad is connected and I click the Marvin XD icon it initiates a scan of my libraries and then time and again gets stuck and delivers an error message at 23% in the process: 'identifying 156 books in Marvin Library'.

The error message:

calibre, version 1.4.0
ERROR: Unhandled exception: <b>IOError</b>:[Errno 22] invalid mode ('wb') or filename: u'L:\\Users\\IlanY\\AppData\\Local\\Temp\\calibre_ ber5up\\s2tt80_ios_local_db\\urn:uuid:934d610a-e1de-4ced-9d94-f9e367b438e4.epub'

calibre 1.4 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.marvin_manager.action", line 352, in main_menu_button_clicked
File "calibre_plugins.marvin_manager.action", line 665, in show_installed_books
File "calibre_plugins.marvin_manager.book_status", line 1191, in initialize
File "calibre_plugins.marvin_manager.book_status", line 3323, in _generate_booklist
File "calibre_plugins.marvin_manager.book_status", line 4158, in _get_installed_books
File "calibre_plugins.marvin_manager.book_status", line 4652, in _scan_marvin_books
File "calibre_plugins.marvin_manager.book_status", line 3179, in _fetch_marvin_content_hash
IOError: [Errno 22] invalid mode ('wb') or filename: u'L:\\Users\\IlanY\\AppData\\Local\\Temp\\calibre_ ber5up\\s2tt80_ios_local_db\\urn:uuid:934d610a-e1de-4ced-9d94-f9e367b438e4.epub'

I went over my library checking all uuid's and couldn't find the one in the message, I was thinking maybe the data on one of my books is corrupted and creates some conflict. Please help me, what should I do?

I've updated Marvin XD to version 1.0.7 which adds error handling to recover from this situation. Please update to this latest version.

G

[ilany]

Hi, thank you very much, version 1.0.7 did solve this issue. Now that I have access to the plugin, I tried to match all my records between Calibre and Marvin, i.e. turn everything green. A bunch of books however refuse to go green and remain yellow with the following error:

calibre, version 1.4.0
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 1.4 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.marvin_manager.book_status", line 1026, in dispatch_context_menu_event
File "calibre_plugins.marvin_manager.book_status", line 5650, in _update_metadata
File "calibre_plugins.marvin_manager.book_status", line 5499, in _update_marvin_metadata
IndexError: list index out of range


Would love it if you could tackle and solve this one.

Cheers,

Ilan.

[GRiker]

@ilany, please create a debug log as described in the first post in this thread and I will take a look at it.

G

[ilany]

Spoiler:

calibre 1.4 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
DEBUG: 0.0 iOSReaderApp:initialize(v1.2.0)
DEBUG: 0.0 iOSReaderApp:compile_ui()
DEBUG: 0.0 iOSReaderApp:_init_prefs(prefs created under v1.0.1)
DEBUG: 0.0 debug_libimobiledevice: False
DEBUG: 0.0 debug_plugin: True
DEBUG: 0.0 development_mode: False
DEBUG: 0.0 marvin_edit_collections_cb: True
DEBUG: 0.0 marvin_enabled_collection_fields: [u'Shelf']
DEBUG: 0.0 marvin_protect_rb: False
DEBUG: 0.0 marvin_replace_rb: False
DEBUG: 0.0 marvin_update_rb: True
DEBUG: 0.0 preferred_reader_app: u'Marvin'
DEBUG: 0.1 iOSReaderApp:_get_connected_device_info()
DEBUG: 4.4 mounting 'com.appstafarian.Marvin'
DEBUG: 5.5 iOSReaderApp:_class_reconfigure('Marvin')
DEBUG: 5.5 iOSReaderApp:_load_reader_app_overlays('Marvin')
DEBUG: 5.6 loading 30 overlays
DEBUG: 5.6 adding overlay '_cover_to_thumb()'
DEBUG: 5.6 adding overlay '_create_new_book()'
DEBUG: 5.6 adding overlay '_get_field_items()'
DEBUG: 5.6 adding overlay '_initialize_overlay()'
DEBUG: 5.6 adding overlay '_localize_database_path()'
DEBUG: 5.6 adding overlay '_remove_existing_copy()'
DEBUG: 5.6 adding overlay '_report_upload_results()'
DEBUG: 5.6 adding overlay '_reset_ios_connection()'
DEBUG: 5.6 adding overlay '_schedule_metadata_update()'
DEBUG: 5.6 adding overlay '_stage_command_file()'
DEBUG: 5.6 adding overlay '_update_epub_metadata()'
DEBUG: 5.6 adding overlay '_wait_for_command_completion()'
DEBUG: 5.6 adding overlay '_watchdog_timed_out()'
DEBUG: 5.6 adding overlay '_xform_metadata_via_plugboard()'
DEBUG: 5.6 adding overlay 'add_books_to_metadata()'
DEBUG: 5.6 adding overlay 'books()'
DEBUG: 5.6 adding overlay 'can_handle()'
DEBUG: 5.6 adding overlay 'can_handle_windows()'
DEBUG: 5.6 adding overlay 'delete_books()'
DEBUG: 5.6 adding overlay 'eject()'
DEBUG: 5.6 adding overlay 'get_busy_flag()'
DEBUG: 5.6 adding overlay 'get_file()'
DEBUG: 5.6 adding overlay 'is_usb_connected()'
DEBUG: 5.6 adding overlay 'is_usb_connected_windows()'
DEBUG: 5.6 adding overlay 'post_yank_cleanup()'
DEBUG: 5.6 adding overlay 'prepare_addable_books()'
DEBUG: 5.6 adding overlay 'remove_books_from_metadata()'
DEBUG: 5.6 adding overlay 'set_busy_flag()'
DEBUG: 5.6 adding overlay 'sync_booklists()'
DEBUG: 5.6 adding overlay 'upload_books()'
DEBUG: 5.6 iOSReaderApp:_initialize_overlay(Marvin)
DEBUG: 5.6 existing thumb cache at 'L:\Users\IlanY\AppData\Local\calibre-cache\Marvin\thumbs.zip'
DEBUG: 5.6 ~~~~~~~~~~~~~~~~~~~~~~~~~ switching to Marvin overlay ~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting up...
DEBUG: 6.0 MarvinManagerAction:genesis(v1.0.7)
DEBUG: 6.0 MarvinManagerAction:init_options()
DEBUG: 6.0 MarvinManagerAction:inflate_dialog_resources()
DEBUG: 6.0 MarvinManagerAction:compile_ui()
DEBUG: 6.4 iOSReaderApp:startup()
DEBUG: 6.6 MarvinManagerAction:rebuild_menus()
DEBUG: 6.6 Marvin not connected
Started up in 1.04 seconds with 180 books
DEBUG: 9.6 iOSReaderApp:reset()
DEBUG: 9.6 detected_device = (1452, 4767, 528, None, None, None)
DEBUG: 9.6 report_progress = <bound method DeviceManager.report_progress of <DeviceManager(Thread-5, started daemon 6024)>>
DEBUG: 9.6 iOSReaderApppen()
DEBUG: 9.6 vid:05ac pid:129f
DEBUG: 9.7 MarvinManagerActionn_device_connection_changed(i OS reader applications)
DEBUG: 9.7 mounting com.appstafarian.Marvin
DEBUG: 10.8 MarvinManagerActionn_device_connection_changed(i Pad2,1 running iOS 7.0)
DEBUG: 10.8 self.has_password: False
DEBUG: 10.8 MarvinManagerAction:rebuild_menus()
DEBUG: 10.8 Marvin connected
DEBUG: 10.8 iOSReaderApp:set_progress_reporter()
Job: 1 Get device information started
DEBUG: 10.8 iOSReaderApp:get_device_information()
DEBUG: 10.8 DeviceClass : iPad
DEBUG: 10.8 DeviceColor : black
DEBUG: 10.8 DeviceName : Ilan Yakar’s iPad
DEBUG: 10.8 FSBlockSize : 8192
DEBUG: 10.8 FSFreeBytes : 16278970368
DEBUG: 10.8 FSTotalBytes : 62189592576
DEBUG: 10.8 FirmwareVersion : iBoot-1940.1.75
DEBUG: 10.8 HardwareModel : K93AP
DEBUG: 10.8 ModelNumber : MC916
DEBUG: 10.8 PasswordProtected : False
DEBUG: 10.8 ProductType : iPad2,1
DEBUG: 10.8 ProductVersion : 7.0
DEBUG: 10.8 SerialNumber : DN6G56VZDFJ0
DEBUG: 10.8 TimeIntervalSince1970: 1380020956.0
DEBUG: 10.8 TimeZone : Asia/Jerusalem
DEBUG: 10.8 TimeZoneOffsetFromUTC: 10800.0
DEBUG: 10.8 UniqueDeviceID : 8cbc1f5fb6086b9af90a2c927c50bda58b85fc50
DEBUG: 10.8 iOSReaderApp:free_space()
DeviceJob: 1 Get device information done, calling callback
Job: 1 Get device information finished
No details available.
DeviceJob: 1 Get device information callback returned
DEBUG: 10.9 iOSReaderApp:set_progress_reporter()
Job: 2 Get list of books on device started
DEBUG: 10.9 iOSReaderApp:books()
DEBUG: 10.9 iOSReaderApp:_localize_database_path()
DEBUG: 10.9 updating local_db from u'/Library/mainDb.sqlite'
Job: 1 Get device information finished
No details available.
DEBUG: 13.6 MarvinManagerAction:main_menu_button_clicked()
DEBUG: 13.6 MarvinManagerAction:show_installed_books()
DEBUG: 13.6 MarvinManagerAction:launch_library_scanner(updatin g library index for virtual library u'')
DEBUG: 13.6 MarvinManagerAction:start_library_indexing()
DEBUG: 13.6 MarvinManagerAction:_busy_operation_setup()
DEBUG: 13.7 MarvinManagerAction:library_index_complete()
DEBUG: 13.7 MarvinManagerAction:_busy_operation_teardown()
DEBUG: 13.7 BookStatusDialog:initialize()
DEBUG: 13.7 BookStatusDialog:_generate_booklist()
DEBUG: 13.8 BookStatusDialog:_scan_library_books(170)
DEBUG: 13.9 BookStatusDialog:_get_installed_books()
DEBUG: 13.9 BookStatusDialog:_busy_operation_setup(Waiting for driver to finish initialization…)
DeviceJob: 2 Get list of books on device done, calling callback
DEBUG: 14.0 iOSReaderApp:settings(format_map for 'Marvin': [u'epub'])
DEBUG: 14.1 iOSReaderApp:settings(format_map for 'Marvin': [u'epub'])
DeviceJob: 2 Get list of books on device callback returned
DEBUG: 14.1 BookStatusDialog:_busy_operation_teardown()
DEBUG: 14.1 BookStatusDialog:_scan_marvin_books(171)
DEBUG: 14.1 BookStatusDialog:_localize_hash_cache()
DEBUG: 14.2 remote hash cache: version 1
DEBUG: 14.2 BookStatusDialog:_purge_cached_orphans()
Job: 2 Get list of books on device finished
No details available.
DEBUG: 14.4 BookStatusDialog:_update_remote_hash_cache()
DEBUG: 15.0 BookStatusDialog:_purge_cover_hash_orphans()
DEBUG: 15.0 BookStatusDialog:_generate_marvin_hash_map()
DEBUG: 15.0 BookStatusDialog:_find_fuzzy_matches()
DEBUG: 15.0 BookStatusDialog:_busy_operation_setup(Preparing Marvin library view…)
DEBUG: 15.0 BookStatusDialog:_construct_table_data()
DEBUG: 15.2 BookStatusDialog:_construct_table_view()
DEBUG: 15.6 BookStatusDialog:_update_refresh_button()
DEBUG: 15.6 BookStatusDialog:_busy_operation_teardown()
DEBUG: 15.6 MarvinManagerAction:show_installed_books(BookStatu s initialized)
DEBUG: 125.3 BookStatusDialog:dispatch_context_menu_event(u'exp ort_metadata' row: 143)
DEBUG: 125.3 BookStatusDialog:_update_metadata(export_metadata)
DEBUG: 125.5 BookStatusDialog:_update_marvin_metadata(Shame and Necessity)
DEBUG: 125.5 mismatches:
{u'pubdate': {u'Marvin': None, u'calibre': datetime.datetime(101, 1, 1, 0, 0)}}
DEBUG: 125.5 BookStatusDialog:_build_metadata_update()
DEBUG: 125.5 'Shame and Necessity': cover is up to date
DEBUG: 125.5 BookStatusDialog:_issue_command()
DEBUG: 125.5 BookStatusDialog:_stage_command_file(update_metada ta)
DEBUG: 125.5 BookStatusDialog:_wait_for_command_completion()
DEBUG: 125.5 14:11:13.908000: waiting for '/Library/calibre/status.xml'
DEBUG: 127.9 14:11:16.271000: monitoring progress of update_metadata
DEBUG: 127.9 14:11:16.280000: -1 50%
DEBUG: 128.0 14:11:16.387000: 0 100%
DEBUG: 128.1 14:11:16.493000: 'update_metadata' complete
DEBUG: 128.1 BookStatusDialog:_localize_marvin_database(startin g)
DEBUG: 128.1 BookStatusDialog:_busy_operation_setup(Updating local database)
DEBUG: 128.1 mainDb: 2,769,920 bytes
DEBUG: 128.4 BookStatusDialog:_busy_operation_teardown()
DEBUG: 128.4 BookStatusDialog:_localize_marvin_database(finishe d)
DEBUG: 128.4 BookStatusDialog:_clear_selected_rows()
DEBUG: 128.5 BookStatusDialog:_flash_affected_rows([143])
DEBUG: 129.8 BookStatusDialog:_flasher_complete()
DEBUG: 132.1 BookStatusDialog:dispatch_button_click()
DEBUG: 132.1 BookStatusDialog:accept()
DEBUG: 132.1 BookStatusDialog:_save_column_widths()

Clarification: those specific books are usually painted green but upon re-scan on next use they are again reverted to yellow. When I compare metadata what usually happens in these cases is a mismatch in publication dates: the field in Marvin is 'unknown' whereas the field in calibre is either a logical date or something nonsensical: 1st January 101...

Ilan.

[GRiker]

The debug log you posted doesn't reflect the error you initially reported. If possible, please run in debug mode while recreating the steps that led to the initial error.

Also, as described in the first post, please create a GitHub issue with your debug log. That makes it easier to communicate with you about the problem.

G

[GRiker]

Originally Posted by ilany

Clarification: those specific books are usually painted green but upon re-scan on next use they are again reverted to yellow. When I compare metadata what usually happens in these cases is a mismatch in publication dates: the field in Marvin is 'unknown' whereas the field in calibre is either a logical date or something nonsensical: 1st January 101...

Ilan.

@ilany, I've updated Marvin XD to version 1.0.8 which should resolve this particular problem. If you are able to recreate the crash you observed earlier and post a GitHub issue with the debug log, that would be helpful.

G

[PO40600]

Hello,

I've just discovered Marvin, and the 2 plug-ins, iOS Reader, and Marvin XD.

I'm using an iPad Retina, and ihad no problem for the connexion.

I've transfered around 600eBooks in Marvin. All of them are Sci-Fi. In calibre, I used 19 different tags according to the category of the book (Fantasy, Sci-FI, THriller, HIstorical, etc, ...

I thought that the plug-ins would create collections according to the tags, but nothing happened in this way.

the category that have been created are only based on the series .

I would like to know if it's possible to automatically create collections according to tags during importation, or if it must be done after the transfer to Marvin. If such, it might be a very long process, because, while exploring Marvin XD, I've selected all of my books and create a collection assignment, to a new colelction named S-F. And it took nearly one hour.

A strange thing happened : Marvin XD has now finished to update Marvin Library. The S-F collection has been created in the iPad, and Marvin say that it contains 603 books, which is correct. But in Marvin XD main window, for each of these 603 books, in tne row names Collections, there is a kind of " i ", written in blue, bold. I thought it should be the name of the collection, but no ...

If you can explain me, and answer to my main question about automatic creation of collection, it would be kind of you.


Thank you.

[GRiker]

Originally Posted by PO40600

Hello,

...But in Marvin XD main window, for each of these 603 books, in tne row names Collections, there is a kind of " i ", written in blue, bold. I thought it should be the name of the collection, but no ...

Double-click the blue, bold "i" to see the collection assignments for a book.

You may also find it helpful to review Marvin XD's help file.

G

[rjh4242]

This is brilliant!!! Thanks so much. It's what I've been looking for for years in terms of a way to integrate a great book reader on my iPad with calibre.

I have a question - I'm not sure if i've done something wrong or if this is a bug or a "feature", so i'm posting here to start with. Happy to provide more details if necessary.

Here's what I've done so far:

I installed both the iOs reader plugin and the Marvin XD plug in in calibre, and set them up according to the instructions on page 1 of this thread.

I installed Marvin on my iPad and imported the included ebooks.

I connected the iPad to my laptop, started calibre, started Marvin on the iPad and clicked on the cloud and calibre. Hey presto! The device was detected.

I copied my books from my calibre library into Marvin by selecting them in calibre and then clicking "Send to device". Sure enough, they showed up in Marvin. yeay!!!

I then started Marvin XD, and it detected all my books. However, everything i had copied from calibre to Marvin showed up in Yellow, not Green. I don't understand this, because initially there was nothing in my Marvin library except the included books, so the metadata of all the other books should be identical between calibre and Marvin.

I then copied the included books from Marvin into calibre using Marvin XD. They turned green, as I expected.

I disconnected my iPad using the disconnect button in Marvin, and then reconnected again. This time all the books turned yellow, INCLUDING the ones I had copied from Marvin that were green when originally copied.

When I checked the metadata in the books that are yellow, the only thing highlighted as different was the cover. Is there some sort of conversion of the cover which means it does't stay the same between Marvin and calibre? Did I do something wrong in the transfer? Thanks for any clarification you can give.

Details of setup:
Windows 7 x64 on an ASUS ux32vd laptop with 10Gb Ram
calibre x64 1.7.0 clean installed before starting this process
ios Reader applications plugin version 1.2.1
Marvin XD plugin v 1.0.8.
iTunes 11.1.1.11
iPad 3 wi-fi only, 64Gb (with 28Gb free) running ios7
Marvin 2.2.5 on iPad