Override 'calibre' as converted feed's author in modified recipe

[sk365]

Hi,

I am desperately trying to find out how to modify a recipe such that the I can set an arbitrary author of the feed instead of "calibre". Eg. I would like have appear 'FAZ' as the author of a converted news feed. Now when I take the pre-made FAZ rececpie below and insert

authors = 'FAZ'
conversion_options = { 'authors' : 'FAZ' }

the result is that calibre still figures as the epub author. Why?

Please help

Best wishes
Stefan



__license__ = 'GPL v3'
__copyright__ = '2008-2011, Kovid Goyal <kovid at kovidgoyal.net>, Darko Miletic <darko at gmail.com>'
'''
Profile to download FAZ.NET
'''

from calibre.web.feeds.news import BasicNewsRecipe

class FazNet(BasicNewsRecipe):
title = 'FAZ.NET_1'
authors = 'FAZ'
__author__ = 'Kovid Goyal, Darko Miletic'
description = 'Frankfurter Allgemeine Zeitung'
publisher = 'Frankfurter Allgemeine Zeitung GmbH'
category = 'news, politics, Germany'
use_embedded_content = False
language = 'de'

max_articles_per_feed = 30
no_stylesheets = True
encoding = 'utf-8'
remove_javascript = True
keep_only_tags = [{'class':'FAZArtikelEinleitung'},
{'id':'ArtikelTabContent_0'}]

feeds = [
('FAZ.NET Aktuell', 'http://www.faz.net/aktuell/?rssview=1'),
('Politik', 'http://www.faz.net/aktuell/politik/?rssview=1'),
('Wirtschaft', 'http://www.faz.net/aktuell/wirtschaft/?rssview=1'),
('Feuilleton', 'http://www.faz.net/aktuell/feuilleton/?rssview=1'),
#('Sport', 'http://www.faz.net/aktuell/sport/?rssview=1'),
('Gesellschaft', 'http://www.faz.net/aktuell/gesellschaft/?rssview=1'),
('Finanzen', 'http://www.faz.net/aktuell/finanzen/?rssview=1'),
('Technik & Motor', 'http://www.faz.net/aktuell/technik-motor/?rssview=1'),
('Wissen', 'http://www.faz.net/aktuell/wissen/?rssview=1'),
#('Reise', 'http://www.faz.net/aktuell/reise/?rssview=1'),
#('Beruf & Chance', 'http://www.faz.net/aktuell/beruf-chance/?rssview=1'),
#('Rhein-Main', 'http://www.faz.net/aktuell/rhein-main/?rssview=1') # AGe add 2012-07-13
]

conversion_options = { 'authors' : 'FAZ' }

[kovidgoyal]

You cannot.

[nickredding]

Assuming you have Python installed on your machine, you can use the attached script to set the author field (EXTH 100) to whatever you like. Note that the author value should be enclosed in quotes if it contains embedded spaces. For example

set_author.py nyt.mobi "Sunday, December 22"

will set the author to today's date in the file nyt.mobi

There's not much in the way of error checking so if you give it a non-existent file it will just crash (harmlessly, of course).

UPDATE: I just noticed you are using EPUB. This only works for MOBI files. Sorry, I don't use EPUB myself so I don't have any script utilities to do this.

In case anyone is wondering why this might be useful for MOBI, when you read news files using the Kindle for iPad app there is no way to distinguish files with the same name but different publication dates. I use the author field to store the publication date since having "calibre" in all of the files as the author isn't very useful, but since the Kindle for iPad app displays the author and can sort by author, it allows me to see the publication date in the list view.

[siebert]

Originally Posted by kovidgoyal

You cannot.

Of course you can! You just need some monkey patch.

Running the following demo recipe with "ebook-convert authorpublisher.recipe authorpublisher.mobi" will create a dummy Mobipocket file which has the author and the publisher set to the values defined in the recipe:

'''
Demo recipe to demonstrate how to set the author and publisher in an ebook to values defined in the recipe.
'''

__license__   = 'GPL v3'
__copyright__ = '2014, Steffen Siebert <calibre at steffensiebert.de>'
__version__   = '1.0'

import os
import re
import string

import calibre.web.feeds.news

from calibre import fit_image
from calibre.constants import preferred_encoding
from calibre.ebooks.metadata.book.base import Metadata
from calibre.utils.magick import Image
from calibre.web.feeds import templates
from calibre.web.feeds.recipes import BasicNewsRecipe

from lxml.html.builder import HTML, HEAD, DIV, BODY, HR

author = "Recipe author"
publisher = "Recipe publisher"

''' 
Replaces calibre.web.feeds.news.MetaInformation via monkey patching.
Creates a LockedMetadata instead of a Metadata object and sets the author and publisher defined in this receipe. 
'''
def LockedMetaInformation(title, authors=(_('Unknown'),)):
    mi = None
    if hasattr(title, 'title') and hasattr(title, 'authors'):
        title.authors = (author, )
        mi = title
        title = mi.title
        authors = mi.authors
    authors = (author, )
    metadata = LockedMetadata(title, authors, other=mi)
    metadata.__setattr__("publisher", publisher)
    return metadata

'''
Extends calibre.ebooks.metadata.book.base.Metadata.
'''
class LockedMetadata(Metadata):
    '''
    Set fields author, authors and publishers only once.
    '''
    def __setattr__(self, field, val, extra=None):
        if field in ["author", "publisher"] and self.has_key(field):
            return
        elif field == "authors" and self.authors[0] != _('Unknown'):
            return
        Metadata.__setattr__(self, field, val, extra)

class AuthorPublisherRecipe(BasicNewsRecipe):
    title = u'Author Publisher Recipe'

    def __init__(self, options, log, progress_reporter):
        BasicNewsRecipe.__init__(self, options, log, progress_reporter)
        # Monkey patch calibre.web.feeds.news to use LockedMetaInformation.
        # This prevents calibre to set author and publisher to 'calibre'.
        calibre.web.feeds.news.MetaInformation = LockedMetaInformation

    def parse_index(self):
        feed = []
        articles = []
        articles.append({'title': 'demo'})
        feed.append(('feed', articles))
        return feed
        return [('demofeed', [{'title': 'demo'}, ])]

If you run the recipe from within the Calibre GUI, the author in the calibre database will still be set to 'calibre'.

A comment in the Calibre code responsible for this "feature" claims that this is needed for the auto delete process to identify the ebooks to delete, but this is a very lame excuse for messing with the author field instead of introducing a dedicated flag in the database.

Anyway, if you also want to see the actual author in the calibre database, you can install the following plugin by saving it as a file named "__init__.py" in an otherwise empty directory and run "calibre-customize -b ." in that directory:

import os

from calibre.customize import FileTypePlugin
from calibre.ebooks.metadata.meta import get_metadata, string_to_authors

class FixRecipeAuthorPlugin(FileTypePlugin):
    name                = 'Fix Recipe Author Plugin'
    description         = 'Replace "calibre" with the true authors in calibre database.'
    supported_platforms = ['windows', 'osx', 'linux']
    author              = 'Steffen Siebert <calibre at steffensiebert.de>'
    version             = (1, 0, 0)
    file_types          = set(['epub', 'mobi'])
    on_postimport       = True
    minimum_calibre_version = (0, 7, 53)

    def postimport(self, book_id, book_format, db):
        if db.field_for("authors", book_id) != ('calibre', ):
            # The author isn't set to 'calibre', nothing to do.
            return

        # Open file and extract metadata to get the real authors.
        path = db.format_abspath(book_id, book_format)
        file = open(path, 'r+b')
        mi = get_metadata(file, book_format)
        file.close()

        if not mi.authors:
            # No authors found in metadata, don't change anything.
            return

        # Extract authors from metadata and write them to database.
        authors = []
        for a in mi.authors:
            authors += string_to_authors(a)
        db.set_field("authors", {book_id: authors})

Have fun hacking Calibre!

[PeterT]

I've done that in the past via use of plugboards to ensure the fields are over written when sent to my device.

[sup]

Originally Posted by siebert

Of course you can! You just need some monkey patch.

Running the following demo recipe with "ebook-convert authorpublisher.recipe authorpublisher.mobi" will create a dummy Mobipocket file which has the author and the publisher set to the values defined in the recipe:

'''
Demo recipe to demonstrate how to set the author and publisher in an ebook to values defined in the recipe.
'''

__license__   = 'GPL v3'
__copyright__ = '2014, Steffen Siebert <calibre at steffensiebert.de>'
__version__   = '1.0'

import os
import re
import string

import calibre.web.feeds.news

from calibre import fit_image
from calibre.constants import preferred_encoding
from calibre.ebooks.metadata.book.base import Metadata
from calibre.utils.magick import Image
from calibre.web.feeds import templates
from calibre.web.feeds.recipes import BasicNewsRecipe

from lxml.html.builder import HTML, HEAD, DIV, BODY, HR

author = "Recipe author"
publisher = "Recipe publisher"

''' 
Replaces calibre.web.feeds.news.MetaInformation via monkey patching.
Creates a LockedMetadata instead of a Metadata object and sets the author and publisher defined in this receipe. 
'''
def LockedMetaInformation(title, authors=(_('Unknown'),)):
    mi = None
    if hasattr(title, 'title') and hasattr(title, 'authors'):
        title.authors = (author, )
        mi = title
        title = mi.title
        authors = mi.authors
    authors = (author, )
    metadata = LockedMetadata(title, authors, other=mi)
    metadata.__setattr__("publisher", publisher)
    return metadata

'''
Extends calibre.ebooks.metadata.book.base.Metadata.
'''
class LockedMetadata(Metadata):
    '''
    Set fields author, authors and publishers only once.
    '''
    def __setattr__(self, field, val, extra=None):
        if field in ["author", "publisher"] and self.has_key(field):
            return
        elif field == "authors" and self.authors[0] != _('Unknown'):
            return
        Metadata.__setattr__(self, field, val, extra)

class AuthorPublisherRecipe(BasicNewsRecipe):
    title = u'Author Publisher Recipe'

    def __init__(self, options, log, progress_reporter):
        BasicNewsRecipe.__init__(self, options, log, progress_reporter)
        # Monkey patch calibre.web.feeds.news to use LockedMetaInformation.
        # This prevents calibre to set author and publisher to 'calibre'.
        calibre.web.feeds.news.MetaInformation = LockedMetaInformation

    def parse_index(self):
        feed = []
        articles = []
        articles.append({'title': 'demo'})
        feed.append(('feed', articles))
        return feed
        return [('demofeed', [{'title': 'demo'}, ])]

If you run the recipe from within the Calibre GUI, the author in the calibre database will still be set to 'calibre'.

A comment in the Calibre code responsible for this "feature" claims that this is needed for the auto delete process to identify the ebooks to delete, but this is a very lame excuse for messing with the author field instead of introducing a dedicated flag in the database.

Anyway, if you also want to see the actual author in the calibre database, you can install the following plugin by saving it as a file named "__init__.py" in an otherwise empty directory and run "calibre-customize -b ." in that directory:

import os

from calibre.customize import FileTypePlugin
from calibre.ebooks.metadata.meta import get_metadata, string_to_authors

class FixRecipeAuthorPlugin(FileTypePlugin):
    name                = 'Fix Recipe Author Plugin'
    description         = 'Replace "calibre" with the true authors in calibre database.'
    supported_platforms = ['windows', 'osx', 'linux']
    author              = 'Steffen Siebert <calibre at steffensiebert.de>'
    version             = (1, 0, 0)
    file_types          = set(['epub', 'mobi'])
    on_postimport       = True
    minimum_calibre_version = (0, 7, 53)

    def postimport(self, book_id, book_format, db):
        if db.field_for("authors", book_id) != ('calibre', ):
            # The author isn't set to 'calibre', nothing to do.
            return

        # Open file and extract metadata to get the real authors.
        path = db.format_abspath(book_id, book_format)
        file = open(path, 'r+b')
        mi = get_metadata(file, book_format)
        file.close()

        if not mi.authors:
            # No authors found in metadata, don't change anything.
            return

        # Extract authors from metadata and write them to database.
        authors = []
        for a in mi.authors:
            authors += string_to_authors(a)
        db.set_field("authors", {book_id: authors})

Have fun hacking Calibre!

Thanks, it works as intended. I just wish I could do this with two lines in a recipe instead of twenty:-). (unless that already happened)

[sup]

Any idea why this works when I run the recipe with ebook-convert but not when I run it manually from calibre via "Schedule News Download/Download Now"?

EDIT: Ah, I see, it is explained (and solved) in the comment by Siebert.

EDIT2: Hm, it does not actually seem to work. Plugin got installed but the author field is still set to "calibre" even though I rerun the recipe.

Kovid, any hope of making changes as outlined by the above post by siebert) that would make this work?

[kovidgoyal]

No, the author is set to calibre for a reason. I have no interest whatsoever in enabling people to change that.

[sup]

Originally Posted by kovidgoyal

No, the author is set to calibre for a reason. I have no interest whatsoever in enabling people to change that.

So there are more reasons but this: "A comment in the Calibre code responsible for this "feature" claims that this is needed for the auto delete process to identify the ebooks to delete, but this is a very lame excuse for messing with the author field instead of introducing a dedicated flag in the database." ? Because that really sounds weird.

[jtmart]

Has there been any evolution on this matter in the past decade or so?

Contributors such as @kbeckman had offered back in the day to work on the issue (I can recall his conversation with @kovidgoyal https://bugs.launchpad.net/calibre/+bug/1320600) and like some people in this forum I believe it would be a nice feature to be able to automatically set a recipe's title and author when fetching news without using @sup's plugin (I have not tested it).

I use a custom RSS feed (https://zapier.com/blog/make-rss-fee...-pocket-items/) in Calibre and I love every bit of it!

[sup]

It is not my code but Siebert's, if I recall correctly. I still do not understands the reason why this is not possible, my question about it eight years ago went unanswered :-).

[jtmart]

Originally Posted by sup

It is not my code but Siebert's, if I recall correctly. I still do not understands the reason why this is not possible, my question about it eight years ago went unanswered :-).

Right, I am sure @kovidgoyal has his reasons, and hope one day this kind of customization of RSS recipes will become possible...