View Single Post
Old 06-21-2022, 03:13 PM   #14
shayaknyc
Enthusiast
shayaknyc began at the beginning.
 
Posts: 42
Karma: 10
Join Date: Sep 2012
Device: Kobo Aura
@davidfor - apologies, my ADD brain sometimes operates on a million cylinders and it all makes sense in MY head, but sometimes gets discombobulated on it's way out through my fingers.

Baseline premise: local calibre library contains no copyrighted materials (personal intellectual property, open-source materials, or expired copyrights)

What inspired my question was Calibre's in-built content server, as well as the calibre-web project. I like the idea of being able to provide people access to my library, and search its vast database of metadata to identify an asset they'd like to download and read. Calibre's GREAT for this - not just for all the default columns, but for the added columns that work specifically for my use-case, such as DOI (Digital Object Identifier - for journal articles), LOC (Library of Congress call numbers), DDC (Dewey Decimal Classification), DDC Genre (Genre mapped to the DDC number), etc. I also include other metrics with each entry such as the Flesh-Kincaid reading quotient, word counts, etc.

Unrelated to Calibre, I've been reading a lot about IPFS, and it was first on my radar back in the early days of scihub/zlibrary. Within that context, I immediately thought about how I could leverage IPFS to syndicate my curated (copyright-free) calibre library for wider distribution. Keep in mind, I'm not expecting to change anything in the process of adding books and similar assets to calibre - books will still be saved to the "Calibre Library" folder on my local computer.

With this in mind, I couldn't figure a clear and easy way for a file to be added to the IPFS (Inter-Planetary File System) and make it easily discoverable and accessible to someone coming through a federated IPFS client/portal. To further complicate matters, each file gets its own unique CID when added to the IPFS. If there are any updates to that file, the hash will change, and subsequently, change the CID of that asset. To combat this, IPFS has introduced IPNS (which I don't fully understand) - but it effectively points to the same asset as it changes over time.

To bring these together, there are a number of independent projects of individuals making online web search portals for assets that are uploaded to the IPFS. The search leaves....a lot to be desired, and I think it's largely limited by the metadata that's available for each CID/IPNS.

Calibre, by contrast, is very effective at searching the metadata of the assets in its database. Of course, if someone is connected directly to my self-hosted content-server or calibre-web, easy-peasy, they just download directly from my local server, however, there may be cases where a remote user may either have internet connectivity issues, or maybe bandwidth between my local server and the remote user isn't sufficient.

By being able to cross-link the IPNS of a given asset in my library, a remote user would potentially be able to download that book/journal article/calibre entry through the IPFS, which could potentially be hosted by several peers, thereby providing a higher-level of integrity of those assets and their ability to get downloaded safely and securely and guaranteed to be exactly the same file (by way of its hash).

Honestly, now that I've put it down to paper in a more organized way, I may have overcomplicated the whole thing, but it's something that's been brewing in my brain and thought it might be interesting to explore. Likely everyone's gut here could be correct, that it's an unworthwhile endeavor, at least given my specific use-case here.
shayaknyc is offline   Reply With Quote