Metadata transfer *from* device to calibre?

[lamerc]

I’ve looked as best as I can but I can’t quite find anything to address my situation.

I’m running Calibre as my library on my PC. I regularly download, read, and edit metadata on ebooks on my iPad.

I want to be able to send/add/save the books I’ve added/changed metadata to while on my iPad to my Calibre library.

The basic problem: How can I get custom/added metadata that I entered while on a device into my Calbre library?

(FWIW, I don’t need xfer of annotations or bookmarks or even “read” status or dates. Title and author come through just fine with the ebook file. I just need series, tags, and description metadata.)

Ideas I’ve tried to work out:

• Specific ereader?— Due to a serious crash* I’ve lost all the books and metadata I had in Marvin, so I’m hunting around and perfectly open to starting back up from any ereader at the moment if that can help. But, best as I can tell, they all make a point of not including the metadate in the epub (which would presumably help me), but in a reader-specific database I’d have no way of importing into calibre with the new books.

• Calibre Companion?— Something in the Calibre Companion app sounded like I could do a reverse transfer (book & metadata from device to calibre) with it, but it talks about only updating “read” status. It sounds like—if I knew what I was doing—I might be able to access the calibre library from the device and thus enter the new metadata directly into calibre from the device (I’d be the only one using it, so no conflict of multiple users or anything), but I suspect my uncertainty comes from trying to read into the descriptions what I want it to do, not what it actually can do.

• Some sort of epub editor/recompiler app that I can run them through in order to embed the metadata while on the iPad before transferring to calibre?

They don’t have to be real-time transfers, I can transfer books/metadata to the library every so often by plugging in the device. I can merge/add to a database if something compatible can be found. I’m easy.

So: Is there any way I can edit metadata fields on an ipad and transfer them to my PC Calibre library? Or is it an inherently flawed concept I won’t be able to work around?

I download and read a lot of short works (fanfic) on the iPad and sort it all by tag fields, series fields, etc., which do not come embedded in the downloaded epub and which I have to add by hand. And, having inadvertently been given a clean slate, I don’t want to end up yet again with a separate “PC ebook library (Calabre)” and “iPad library (whatever ereader I use).

TIA for any help!
(And congratulations on getting this far—economy of words is not my forte!)

Dianne

—-
*as in “physically flung from roof of my car to crash to the ground somewhere on the 134 freeway”—and in attempting to recover from this disaster finding out that my auto-backup arrangement settings had been messed up almost a year previous & and all my Marvin books & metadata were gone. (In 35 years of driving I’ve never accidentally left *anything* on the roof of my car—so, of course the one time I do, it’s my entire purse—wallet and iPad mini included—and it manages to stay up there for 3/4 of a mile until I’m on the freeway at full speed. )

[BetterRed]

If you can export the metadata from your IOS reader app to a CSV file you can probably import it into calibre via the Import List plugin, see Index of plugins.

BR

[ilovejedd]

Curious, what method do you use to download your fanfics?

For the past 2-4 years, I've been getting most of mine through AO3 because the built-in epub download is so convenient. I save fanfics directly to Dropbox for immediate reading. All the metadata editing though, I leave for later via Calibre. It took a few months of tweaking but I have a pretty decent Calibre workflow for AO3 now (mostly lots of saved searches and bulk search and replace).

Granted, I'd sometimes have several months worth of unorganized fanfics when I'm too lazy to fix the metadata in Calibre. Mind, I'm even lazier fixing metadata on Marvin anyway so no big deal. It's a major pain having to enter all that info manually.


Marvin
For what it's worth, Marvin does allow backup of the library and database via General settings -> Advanced -> Backup and restore

Alas, you'll need to figure out how to convert from Marvin's SQLite database to something importable by Calibre. Another thing, it backs up the entire library every single time (although given what happened to your iPad, that might actually be a good thing).

There used to be better integration between Calibre and Marvin (Classic). Unfortunately, I believe Apple killed the method that Marvin used post iOS 8.


FanFicReader plugin
Alternately, as long as the story url is located in the EPUB somewhere, you might be able to use the FanFicReader plugin to scrape metadata from the fanfic website (assuming it hasn't been deleted from the site yet).


Calibre Companion
Thus far, I believe Calibre Companion only has the ability to update read status. With Calibre content server recently having added metadata editing support though, mayhaps @chaley could chime in regarding feasibility of adding similar functionality in CC.

[BetterRed]

Originally Posted by ilovejedd

Calibre Companion
Thus far, I believe Calibre Companion only has the ability to update read status.

IIRC CC can also update a Date Read column

BR

[Adoby]

If you have the PC in the same network as the reader, and the device is powerful enough, you can run calibre remotely and edit metadata in calibre from the ereader that way. Most likely not comfortable, but possible. Using a laptop as reader would make it more comfortable to edit metadata, but perhaps not reading.

[ilovejedd]

Originally Posted by BetterRed

IIRC CC can also update a Date Read column

BR

I know. I consider that part of read status.

[ilovejedd]

Originally Posted by Adoby

If you have the PC in the same network as the reader, and the device is powerful enough, you can run calibre remotely and edit metadata in calibre from the ereader that way. Most likely not comfortable, but possible. Using a laptop as reader would make it more comfortable to edit metadata, but perhaps not reading.

How would this be possible on an iPad? Last I checked, Calibre doesn't run on iOS.

One can edit metadata via web browser using the Calibre content server but it's a pain in the neck.

An alternative is to remote desktop into the computer running Calibre. Caveat, it's hard to operate without mouse and keyboard.

[Adoby]

I don't mean that you run calibre ON the iPad. Just that you control it FROM the iPad. I assume that there are VNC or RDP software for the iPad? (Spoiler: I just checked. Yes there are.) So you can read a book on your iPad and switch app to instead remotely control calibre on your PC from the iPad. And edit metadata as you like. I assume that you can even attach a keyboard and a mouse to the iPad and control calibre fully from the iPad.

Or you might even use the calibre reader app on your PC remotely from your iPad.

I wouldn't do it. But I don't feel any need to transfer metadata from my reader device to calibre.

Or skip the iPad and use some other device. Perhaps even one that can run calibre. Perhaps even a laptop or a hybrid laptop/tablet. That may or may not be an alternative depending on how important it is to be able to edit the metadata in calibre on/from the reader device compared to how important it is to keep using an iPad.

[theducks]

Originally Posted by ilovejedd

I know. I consider that part of read status.

Me too.

[ilovejedd]

Originally Posted by Adoby

I don't mean that you run calibre ON the iPad. Just that you control it FROM the iPad. I assume that there are VNC or RDP software for the iPad? (Spoiler: I just checked. Yes there are.) So you can read a book on your iPad and switch app to instead remotely control calibre on your PC from the iPad. And edit metadata as you like. I assume that you can even attach a keyboard and a mouse to the iPad and control calibre fully from the iPad.

Or you might even use the calibre reader app on your PC remotely from your iPad.

I wouldn't do it. But I don't feel any need to transfer metadata from my reader device to calibre.

Or skip the iPad and use some other device. Perhaps even one that can run calibre. Perhaps even a laptop or a hybrid laptop/tablet. That may or may not be an alternative depending on how important it is to be able to edit the metadata in calibre on/from the reader device compared to how important it is to keep using an iPad.

Yes, there are RDP and VNC apps available (note, I did mention remote desktop on my previous post). I've been using those since iPad 2 (iOS 4?). I'm sure they've been available for even longer for iPhone but remote desktop is difficult on a tiny screen so I never bothered prior to getting an iPad.

Mouse, only Citrix X1 and only for select apps. One of the frequent complaints I've seen on MacRumors is lack of built-in mouse support on iPad (e.g. for bluetooth mouse).

Mind, when I start needing mouse and keyboard to accomplish tasks or for efficiency, that's when I ditch the iPad and switch to desktop/laptop.


I don't believe any ebook reader app on Android or iOS facilitate easy transfer of metadata from Android/iOS to Calibre. On iOS, it's also more difficult to extract books from apps (pretty much the only way to do so is if the app has built-in export function). Hence, I always save to Dropbox instead of directly importing to the reading app.


@lamerc/Dianne
Here are my settings/workflow. Maybe it might give you some ideas on how you want to tackle your own setup:

Marvin
Link to Dropbox: Linked
Dropbox search path: Custom (/fanfic)

Dropbox
  /fanfic
    /ao3-tmp -> my intake folder
    /ao3 -> copy of my Calibre library

1. Download epub on iOS device.
2. More... -> Save to Dropbox -> /fanfic/ao3-tmp (iOS/Dropbox usually keeps a history of recently used folders). Once on Dropbox, the epub is accessible on any of my iOS devices and computers.
3. Add books from ao3-tmp to Calibre library then delete. This process can be automated with Automatic adding but I would lose access to "unprocessed" books unless I either 1.) move my live Calibre library to Dropbox (probably bad idea) or 2.) automate step 6 below (probably doable with RealTimeSync).
4. Add and cleanup metadata in Calibre.
5. Run Polish books to embed updated metadata into epub.
6. FreeFileSync (one-way mirror) from Calibre library to Dropbox\fanfic\ao3
7. Delete old copies with missing metadata from Marvin and batch re-download from Dropbox to Marvin (I sort by Date, newest to make this easier).

[lamerc]

Originally Posted by ilovejedd

Curious, what method do you use to download your fanfics?

For the past 2-4 years, I've been getting most of mine through AO3 because the built-in epub download is so convenient.

Bingo. Same here.

Originally Posted by ilovejedd

I save fanfics directly to Dropbox for immediate reading. All the metadata editing though, I leave for later via Calibre.

The problem is that I download them on the ipad to the reader, read them there, and then start a new one. If I don't put in the metadata (Fandom, rating, pairing, stars, notes) pretty much as soon as I finish, it will never get done without re-reading them all again later on my computer. :-/)

Originally Posted by ilovejedd

There used to be better integration between Calibre and Marvin (Classic). Unfortunately, I believe Apple killed the method that Marvin used post iOS 8.

See, that's what I thought... then I went to play with it... and realized Marvin's been redone, and people are saying it the old tricks don't work anymore.

Originally Posted by ilovejedd

With Calibre content server recently having added metadata editing support though, mayhaps @chaley could chime in regarding feasibility of adding similar functionality in CC.

I could set up a content server version... (I'm *pretty* sure. I know enough to know that I'm gonna have to look up a few things to do it, but if that would work...

[lamerc]

Originally Posted by Adoby

I don't mean that you run calibre ON the iPad. Just that you control it FROM the iPad. I assume that there are VNC or RDP software for the iPad? (Spoiler: I just checked. Yes there are.) So you can read a book on your iPad and switch app to instead remotely control calibre on your PC from the iPad. And edit metadata as you like. I assume that you can even attach a keyboard and a mouse to the iPad and control calibre fully from the iPad.

...

Or skip the iPad and use some other device. Perhaps even one that can run calibre. Perhaps even a laptop or a hybrid laptop/tablet. That may or may not be an alternative depending on how important it is to be able to edit the metadata in calibre on/from the reader device compared to how important it is to keep using an iPad.

Yeah, iPad mini fits in my purse & is available to me for reading/downloading/etc. 100% of the time I needed, vs. the PC at about 30% of the times. So, yeah, I need an iPad solution.

I hadn't thought to remote into the PC, that has promise...

[lamerc]

Originally Posted by ilovejedd

Yes, there are RDP and VNC apps available (note, I did mention remote desktop on my previous post). I've been using those since iPad 2 (iOS 4?).

Whereas I feel kind of silly now not having thought of it, given I'm a PC person stuck using iOS apps.

Originally Posted by ilovejedd

I don't believe any ebook reader app on Android or iOS facilitate easy transfer of metadata from Android/iOS to Calibre. On iOS, it's also more difficult to extract books from apps (pretty much the only way to do so is if the app has built-in export function). Hence, I always save to Dropbox instead of directly importing to the reading app.

*sigh* Yeah, that was my original quest.

You do have a good point about the book export/Dropbox thing. Gotta start doing that.

Originally Posted by ilovejedd

@lamerc/Dianne
Here are my settings/workflow. Maybe it might give you some ideas on how you want to tackle your own setup:

I'm going to mess around with these and see. I really appreciate both your and @Adoby's help with this. It's been an ongoing "mess around with it when I get a bit of time" project forever. This gives me new avenues to pursue.

[ilovejedd]

Originally Posted by lamerc

The problem is that I dponwload them on the ipad, read them there and then start a new one. If I don't put in the metadata (Fandom, rating, pairing, stars, notes) pretty much as soon as I finish, it will never get done without re-reading them all again later on my computer. :-/)

Lol, I do the same except I gave up tagging in Marvin because I keep forgetting to tag before moving on to the next fic.

By the time I imported my AO3 fics to Calibre (collected over 2-4 years), I had close to 4K. Extracting fandom, ships, tags, summary, etc is actually quite easy. It's AO3 tag wrangling/cleanup that's a hassle.

Alas, semi-automated metadata extraction doesn't really help with personal ratings/stars or notes.

Like I mentioned previously, Marvin does have an option to backup and restore the full library. It appears all book info including collections, tags, notes, etc is stored in a single SQLite database. I've yet to figure out an automated method to import the contents of said database to Calibre, though.

[DNSB]

@ilovejedd: Not sure if this is going of much help but one friend of mine used to use the Firefox SQLite Manager plugin to export data into a csv format, massage that data in Excel and then import the massaged data using the Import List plugin for calibre. When Firefox went to their Quantum releases, that broke SQLite Manager and he never bothered to go to the effort of reworking his setup to use another program to export csv. SQLiteStudio does have the capability of exporting to csv (freebie with Linux, OSX and Windows support).