News management: delete older than xxx on cli

sws · 10-02-2012, 10:17 AM

Hi,

I am running a calibre server for news only on a hosted vserver. That's why I manage everything by using the cli. I can load news via ebook-convert, I can update meta data via ebook-meta and I can add news to calibre via calibredb. I created some simple scripts to do the job daily, weekly, every fortnight and monthly by cronjobs.

But I didn't find an option to automatically delete news in the same way as it is possible in the GUI: "delete news older than xxx". So here is my question:

Can I delete news epubs form the calibre db by script in the way described above, and if so, how?

Greets,
Sebastian

kovidgoyal · 10-02-2012, 10:24 AM

calibredb remove

sws · 10-02-2012, 10:27 AM

Well, yes.
But how do I write a script that reads the IDs of the epubs I want to delete?

kovidgoyal · 10-02-2012, 02:25 PM

calibredb list --fields timestamp,authors

sws · 10-02-2012, 04:07 PM

Thank you Kovid for your quick answers. I will give it another try.

kovidgoyal · 10-03-2012, 03:02 AM

Just for completeness:

Code:

calibredb list --search 'date:<60daysago and authors:=calibre and tags:News'

will find all news downloads whose timestamp is older than 60 days

kovidgoyal · 10-03-2012, 03:08 AM

And

Code:

calibredb list --search 'date:<1daysago and tags:news and authors:=calibre' | cut -f1 -d" " | tail -n+2

will give you the list of ids.

sws · 10-04-2012, 06:19 AM

Hello Kovid,

thanks again for this quick responses. I'm really impressed.

I tried to execute your code but generated the following error:

Quote:

Traceback (most recent call last):
File "site.py", line 58, in main
File "site-packages/calibre/library/cli.py", line 1361, in main
File "site-packages/calibre/library/cli.py", line 198, in command_list
File "site-packages/calibre/library/cli.py", line 64, in do_list
File "site-packages/calibre/library/caches.py", line 833, in search
File "site-packages/calibre/library/caches.py", line 851, in search_getting_ids
File "site-packages/calibre/utils/search_query_parser.py", line 207, in parse
File "site-packages/calibre/utils/search_query_parser.py", line 218, in _parse
File "site-packages/calibre/utils/search_query_parser.py", line 226, in evaluate
File "site-packages/calibre/utils/search_query_parser.py", line 232, in evaluate_and
File "site-packages/calibre/utils/search_query_parser.py", line 226, in evaluate
File "site-packages/calibre/utils/search_query_parser.py", line 265, in evaluate_token
File "site-packages/calibre/utils/search_query_parser.py", line 269, in _get_matches
File "site-packages/calibre/library/caches.py", line 698, in get_matches
File "site-packages/calibre/library/caches.py", line 413, in get_dates_matches
ParseException: Date conversion error (at char 8), (line:1, col:9)

So there is something going on with date conversion. Do you have a hint how to get rid of this error message?

OS is Debian Squeeze,
Python is: Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2

Greets,
Sebastian

kovidgoyal · 10-04-2012, 07:12 AM

You need to use the correct localized version of daysago for whatever language you have set on your system.

sws · 10-04-2012, 08:35 AM

Quote:

Originally Posted by kovidgoyal

You need to use the correct localized version of daysago for whatever language you have set on your system.

Sorry to stress your patience further, but I don'T speak python. How do I use the correct localized version of daysago?

$ locale

gives me a complete german utf8 environment.

Quote:

LANG=de_DE.utf8
LANGUAGE=de_DE.utf8
LC_CTYPE="de_DE.utf8"
LC_NUMERIC="de_DE.utf8"
LC_TIME="de_DE.utf8"
LC_COLLATE="de_DE.utf8"
LC_MONETARY="de_DE.utf8"
LC_MESSAGES="de_DE.utf8"
LC_PAPER="de_DE.utf8"
LC_NAME="de_DE.utf8"
LC_ADDRESS="de_DE.utf8"
LC_TELEPHONE="de_DE.utf8"
LC_MEASUREMENT="de_DE.utf8"
LC_IDENTIFICATION="de_DE.utf8"
LC_ALL=de_DE.utf8

Where should I look for the german version of "daysago"?

PeterT · 10-04-2012, 08:44 AM

When I look in the calibre source tree, there is a file src\calibre\translations\de.po (http://bazaar.launchpad.net/~kovid/c...slations/de.po)

Within it I see

Code:

msgid "daysago"
msgstr "vor Tagen"

so I presume you want to use

Code:

calibredb list --search 'date:<1vor tagen and tags:news and authors:=calibre' | cut -f1 -d" " | tail -n+2

frostschutz · 10-04-2012, 08:48 AM

That would be kind of strange, and in that case I'd just run this command with LANG=C

Edit: Which doesn't work either because it's a language setting in Calibre.

kovidgoyal · 10-04-2012, 09:01 AM

CALIBRE_OVERRIDE_LANG=en calibredb ...

frostschutz · 10-04-2012, 09:06 AM

Ah, nice.

Reading into the source code, it also accepts _daysago instead of daysago regardless of language.

EDIT: I take it back, _daysago only works when it's not English? I don't get it.
EDIT2: ah, it's because _daysago ends with daysago ... horror

Search filters should stay untranslated I think, it's not like you replace "and" with "und" either. And it's not easy to guess what the translation could be.

sws · 10-04-2012, 09:16 AM

Sorry, but error persists:

Quote:

Traceback (most recent call last):
File "site.py", line 58, in main
File "site-packages/calibre/library/cli.py", line 1361, in main
File "site-packages/calibre/library/cli.py", line 198, in command_list
File "site-packages/calibre/library/cli.py", line 64, in do_list
File "site-packages/calibre/library/caches.py", line 833, in search
File "site-packages/calibre/library/caches.py", line 851, in search_getting_ids
File "site-packages/calibre/utils/search_query_parser.py", line 207, in parse
File "site-packages/calibre/utils/search_query_parser.py", line 218, in _parse
File "site-packages/calibre/utils/search_query_parser.py", line 226, in evaluate
File "site-packages/calibre/utils/search_query_parser.py", line 232, in evaluate_and
File "site-packages/calibre/utils/search_query_parser.py", line 226, in evaluate
File "site-packages/calibre/utils/search_query_parser.py", line 265, in evaluate_token
File "site-packages/calibre/utils/search_query_parser.py", line 269, in _get_matches
File "site-packages/calibre/library/caches.py", line 698, in get_matches
File "site-packages/calibre/library/caches.py", line 413, in get_dates_matches
ParseException: Date conversion error (at char 4), (line:1, col:5)

I even tried to execute this comand on the python interpreter command line including

Quote:

import locale
locale.setlocale(locale.LC_ALL, '')

import os
os.system("calibredb list --search 'date:<1daysago and tags:news and authors:=calibre' | cut -f1 -d" " | tail -n+2")

Didn't help, same error.

10-02-2012, 10:17 AM	#1
sws Enthusiast Posts: 40 Karma: 10 Join Date: Oct 2012 Device: Pocket Book Touch HD3	News management: delete older than xxx on cli Hi, I am running a calibre server for news only on a hosted vserver. That's why I manage everything by using the cli. I can load news via ebook-convert, I can update meta data via ebook-meta and I can add news to calibre via calibredb. I created some simple scripts to do the job daily, weekly, every fortnight and monthly by cronjobs. But I didn't find an option to automatically delete news in the same way as it is possible in the GUI: "delete news older than xxx". So here is my question: Can I delete news epubs form the calibre db by script in the way described above, and if so, how? Greets, Sebastian

10-03-2012, 03:02 AM	#6
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Just for completeness: Code: calibredb list --search 'date:<60daysago and authors:=calibre and tags:News' will find all news downloads whose timestamp is older than 60 days Last edited by kovidgoyal; 10-03-2012 at 03:08 AM.

10-03-2012, 03:08 AM	#7
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	And Code: calibredb list --search 'date:<1daysago and tags:news and authors:=calibre' \| cut -f1 -d" " \| tail -n+2 will give you the list of ids.

10-04-2012, 08:44 AM	#11
PeterT Grand Sorcerer Posts: 13,381 Karma: 78877538 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	When I look in the calibre source tree, there is a file src\calibre\translations\de.po (http://bazaar.launchpad.net/~kovid/c...slations/de.po) Within it I see Code: msgid "daysago" msgstr "vor Tagen" so I presume you want to use Code: calibredb list --search 'date:<1vor tagen and tags:news and authors:=calibre' \| cut -f1 -d" " \| tail -n+2

10-04-2012, 08:48 AM	#12
frostschutz Linux User Posts: 2,282 Karma: 6123806 Join Date: Sep 2010 Location: Heidelberg, Germany Device: none	That would be kind of strange, and in that case I'd just run this command with LANG=C Edit: Which doesn't work either because it's a language setting in Calibre. Last edited by frostschutz; 10-04-2012 at 08:59 AM.

10-02-2012, 10:24 AM	#2
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	calibredb remove

10-02-2012, 10:27 AM	#3
sws Enthusiast Posts: 40 Karma: 10 Join Date: Oct 2012 Device: Pocket Book Touch HD3	Well, yes. But how do I write a script that reads the IDs of the epubs I want to delete?

10-02-2012, 02:25 PM	#4
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	calibredb list --fields timestamp,authors

10-02-2012, 04:07 PM	#5
sws Enthusiast Posts: 40 Karma: 10 Join Date: Oct 2012 Device: Pocket Book Touch HD3	Thank you Kovid for your quick answers. I will give it another try.

10-04-2012, 07:12 AM	#9
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You need to use the correct localized version of daysago for whatever language you have set on your system.

10-04-2012, 09:01 AM	#13
kovidgoyal creator of calibre Posts: 45,257 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	CALIBRE_OVERRIDE_LANG=en calibredb ...

10-04-2012, 09:06 AM	#14
frostschutz Linux User Posts: 2,282 Karma: 6123806 Join Date: Sep 2010 Location: Heidelberg, Germany Device: none	Ah, nice. Reading into the source code, it also accepts _daysago instead of daysago regardless of language. EDIT: I take it back, _daysago only works when it's not English? I don't get it. EDIT2: ah, it's because _daysago ends with daysago ... horror Search filters should stay untranslated I think, it's not like you replace "and" with "und" either. And it's not easy to guess what the translation could be. Last edited by frostschutz; 10-04-2012 at 09:08 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Is there a way to delete old news from the device when downloading new news?	mwheinz	Library Management	2	01-16-2012 12:15 PM
Start all scheduled news downloads using CLI	Zillion	Calibre	2	07-19-2011 11:21 AM
Delete news older than	Stingo	Calibre	2	12-25-2010 05:13 AM
folder management & delete source file after conversion	Taantric	Calibre	1	12-16-2008 01:10 AM

Advert

Advert