Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 08-20-2018, 07:29 AM   #1
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
Calibre "save to disk", "last_modified" field format

Hi,

I would like to have the "last modified" modification date for generated file when i "save to disk" instead of the file "last modified date".

I can see 2 solutions:
- change the "last modified date" in generated file after it has been written
- allow to alter the "last_modified" format in "save to disk" options, so I could have for instance "yyyyMMddhhmm", (it is "MMM, yyyy" today). In that case I could make a simple script to change the file modification date from this format information.

Any idea?

Regards,

bodi.

Last edited by bodiccea; 08-20-2018 at 07:36 AM.
bodiccea is offline   Reply With Quote
Old 08-23-2018, 06:54 PM   #2
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,467
Karma: 10584684
Join Date: Mar 2012
Location: Sydney Australia
Device: none
bump

@bodiccea - I'm not sure what you're asking, but I am wondering if you're confusing calibre's last_modified/Modified column and the file system's Modified Date.

The first is the date/time that a book's entry in the metadata database was last modified, it cannot be changed. The second is the date/time the contents of a file were last modified, it can be changed via your file browser (File Explorer, Finder etc).

BR
BetterRed is offline   Reply With Quote
Advert
Old 08-24-2018, 06:46 AM   #3
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
Better explanation of my need...

Hi, and thanks for your answer.

I know of course the difference of times between the calibre database one, and the exported file one.

I present my books are on a web site, and I would like people to know what has changed (i.e. file modification date should be calibre modification date).

I would like to be able to include both calibre "modification date" date in the save to disk (in the Preferences/import/export/template).
For instance, if I could add "calibre creation date" in format yyyyMMddhhmm, the exported file could be (on linux file system) file system

$ ls -l --time-style=long-iso
-rw-rw-r-- 1 br br 1119273 2018-05-19 15:56 200402232331 - Gallo,Max - Nice.epub

Here 200402032331 would be the modification date in calibre library, and would become part of the created file name (bold).

With a 2 lines script I could change the exported file modification date and rename the file, so that the file would become:

$ ls -l --time-style=long-iso
-rw-rw-r-- 1 br br 1119273 2004-02-23 23:31 Gallo,Max - Nice.epub


You will notice (blue color), that the file modification date is now what I would like to see, so that people can find easily what is new).

Is it more clear ?

Thanks.

PS. Have a look on:
https://ebooks.raoult.com/eng/?C=M;O=A

These books were added in Calibre at different dates for 5-6 years, and it is impossible for people to know if there are some new ones...
bodiccea is offline   Reply With Quote
Old 08-24-2018, 07:46 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,722
Karma: 10215826
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I dont recall if {last_modified} works in save to disk templates or not, it probably does, but if not you can definitely sue either {timestamp} or {pubdate} for the date and ublished columns respectively.

https://manual.calibre-ebook.com/template_lang.html
kovidgoyal is offline   Reply With Quote
Old 08-24-2018, 03:59 PM   #5
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
date metadata

I found a temporary solution.

I added a custom column {#brtime} of type date and built from other values.
Its template is "{timestamp:format_date(yyyyMMddhhmm)}". The issue is {timestamp} is book change, not metadata. For me changing metadata is changing book too.

My "Saving books to disk" template is (one subdir per book language).
{languages}/{#brtime}-{author_sort} - {series:|(| } {series_index:0>2s|#|) - }{title}

Is looks like for instance:
-rw-rw-r-- 1 br br 390974 Aug 24 20:11 201508150000-du Maurier,Daphné - Rebecca.epub

In calibre, Modified column is 20180224021402 (metadata change).
The date in red is book modification time {timestamp} of Calibre (strangely hhmm are 0000 for all of them).
I made a script which gets this date, renames correctly, and changes its "modification time".

Better than a situation where all books were dated Aug 2018.

The script (bash):
----------------------------------------------------------------------
#! /bin/bash
#
# finds all epubs in first level subdirs, which should have the format:
# 201402150000-Arnaud,G.J. - (La Compagnie des Glaces #08) - La Compagnie de la Banquise.epub
#
# the first date is inclusion of file in calibre.
# We will save it, suppress it from filename, and change modification time of book with this time.
#
# br, Jul 24, 2018

# first we find files corresponding to the pattern [0-9]{12}-.*\.epub in direct subdirectories,
# in case we added a single book, we do not want to do something stupid

find . -maxdepth 2 -regextype sed -type f -regex '\.\/[a-z-]*/*[0-9]\{12\}-.*\.epub$' -print | \
while read fullname
do
dirname=$(dirname "$fullname")
basename=$(basename "$fullname")
date=$(echo "$basename" | cut -c 1-12)
file=$(echo "$basename" | cut -c 14-)
newname="$dirname/$file"
mv "$fullname" "$newname"
touch -t $date "$newname"
echo processed "$fullname"
done

exit 0
----------------------------------------------------------------------

Next step, as I regular backup, I think I can find correct {last_modified} information. If the CLI Calibre permits to get/set this field, I believe I could restore everything to a normal status. I could also find information in some Calibre dates... In that case, I will change everything and choose the latest of meta-data and book last change.

Last edited by bodiccea; 08-24-2018 at 04:10 PM. Reason: typos
bodiccea is offline   Reply With Quote
Advert
Old 08-25-2018, 04:31 AM   #6
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
Forgot to show you the result after the script run...

-rw-rw-r-- 1 br br 1119387 2014-02-15 00:00 fra/Gallo,Max - Nice.epub


The file modofication date has been changed to something more correct, even if not perfect as I said in my previous mail
bodiccea is offline   Reply With Quote
Old 08-25-2018, 09:22 AM   #7
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,467
Karma: 10584684
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@bodiccea - I wondered if this might be of any use ==>> https://www.mobileread.com/forums/sh...47&postcount=7

BR
BetterRed is offline   Reply With Quote
Old 08-25-2018, 01:16 PM   #8
chaley
CC Android & calibre dev
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 8,795
Karma: 2692874
Join Date: Jan 2010
Location: France
Device: Many android devices
Quote:
Originally Posted by bodiccea View Post
I added a custom column {#brtime} of type date and built from other values.
Its template is "{timestamp:format_date(yyyyMMddhhmm)}". The issue is {timestamp} is book change, not metadata. For me changing metadata is changing book too.

My "Saving books to disk" template is (one subdir per book language).
{languages}/{#brtime}-{author_sort} - {series:|(| } {series_index:0>2s|#|) - }{title}
Why create a custom column? You can do the same thing in the save-to-disk template
Code:
{languages}/{timestamp:format_date(yyyyMMddhhmm)}-{author_sort} - {series:|(| } {series_index:0>2s|#|) - }{title}
You can also use the last_modified date, which is the last time the metadata for the book changed.
Code:
{languages}/{last_modified:format_date(yyyyMMddhhmm)}-{author_sort} - {series:|(| } {series_index:0>2s|#|) - }{title}
chaley is offline   Reply With Quote
Old 08-25-2018, 09:09 PM   #9
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
@BetterRed,

I don't think it is practical for 3,000 books with a GUI, even as I don't know the last modification date.

For now, I will try, if possible (all by CLI interface, not GUI), so it can be scriptable:
1) try to get the last_modified date from a correct backup (1 have 7 days/4 weeks/12 months/3 years backups). I believe a few months old one will be fine.
2) try to inject these dates in new Calibre DB for each book (making a list of the few ones which were not found for some reason, such as author/title change). I will have to manually change these errors.
3) replace #brtime with MAX({last_modified}, {timestamp}). The reason I created this new brtime field was to end-up with this: The filesystem date will show the real last change, metadata or book.

I believe I can write a function for point 3, never wrote one in Calibre.

I need to study the CLI interface first, not sure how powerful/easy it is...

Another maybe simpler solution: As I have file system backups. I know which books were added recently (look in ebooks.raoult.com, french section).
I save them somewhere else, delete the library and import a backup, where {last_modified} fields are correct, and add the new books.

In that case, just the point (3) will be the difficult task:
I will need a new plugin, but python is an easy language, just not sure how it integrates in columns.

br.
bodiccea is offline   Reply With Quote
Old 08-25-2018, 09:20 PM   #10
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
@Chaley,

The issue is the {last_modification} field is wrong (a mistake from me somewhere, don't know what,when, and why). So I need first to restore them.

I created a new column to be able later to eventually choose the MAX({last_modified}, {timestamp}) for that column, as indicated in my answer to BetterRed.
And this date will be shown in file system as modification date for books (my last bash script above does this).

br.
bodiccea is offline   Reply With Quote
Old 08-25-2018, 11:21 PM   #11
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,467
Karma: 10584684
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@bodiccea - a books last_modified column will be updated whenever you change any column that affects a book, but what's not obvious is that a modification in one book, say the spelling of a tag, can ripple through many books.

I found that the rippling changes were almost invariably caused by corrections to my errors, so I gave up on trying to make any use of the last_modified column. Instead I maintain a custom column - '#last-change'. This is pretty easy for me because I do all my metadata editing in the book list. So, I just tab into the cell and press '=', to set it to 'now'. If it's an error correction I don't update '#last_change'.

And, if I change the content of a book I can update '#last_change', IIRC changing the content of a book with an editor does not update 'last-modified'. The vast majority of my books are not commercial publications, so I do a lot of content editing.

BR

Last edited by BetterRed; 08-26-2018 at 12:13 AM.
BetterRed is offline   Reply With Quote
Old 08-25-2018, 11:33 PM   #12
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 11,467
Karma: 10584684
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by bodiccea View Post
And this date will be shown in file system as modification date for books (my last bash script above does this).

br.
This is where you confuse me: is the 'modification date' referred to above, the file systems last modified date, or the date you put into the save to disk file name, i.e. calibre's '[metadata] last_modified'?

BR
BetterRed is offline   Reply With Quote
Old 08-26-2018, 11:55 AM   #13
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
Modification date on file system

@BetterRed,

I want the file system modification date to be the MAX({last_modified}, {timestamp}).
Or, to be more clear: I must set my 3000 books modification date on file system automatically from calibre data.
So any GUI (finder or other) is not useable: I have to extract the dates from calibre automatically, and only a script (already done, look in a previous message) can do it.

So the date seen on web site will almost be correct.
I did read somewhere that some changes on boot itself (for instance adding a chapter, this kind of things) may change the {timestamp} field, so taking the MAX wil be the secure way.
If this {timestamp} ("date" column in GUI) always stays the same, and is the insertion book date in library, the MAX() will maybe not be necessary). This field is not really well documented.

br.
bodiccea is offline   Reply With Quote
Old 08-26-2018, 12:31 PM   #14
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 22,354
Karma: 22265700
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: K4NT, Galaxy Tab A, Kobo Aura2
Quote:
Originally Posted by bodiccea View Post
@BetterRed,

I want the file system modification date to be the MAX({last_modified}, {timestamp}).
Or, to be more clear: I must set my 3000 books modification date on file system automatically from calibre data.
So any GUI (finder or other) is not useable: I have to extract the dates from calibre automatically, and only a script (already done, look in a previous message) can do it.

So the date seen on web site will almost be correct.
I did read somewhere that some changes on boot itself (for instance adding a chapter, this kind of things) may change the {timestamp} field, so taking the MAX wil be the secure way.
If this {timestamp} ("date" column in GUI) always stays the same, and is the insertion book date in library, the MAX() will maybe not be necessary). This field is not really well documented.

br.
{last_modified} has no clue of what the actual book files Modified date is.
It refers SOLELY to DB actions. It will record the time you ADD (a book)
eg. using Open_with to Edit the EPUB does not change the {last_modified} value. ( I keep a custom column for the last edited date which is manually set) changing that, also bumps the last_modified timestamp
So if you are more interested in EDITED Dates, create your own column.
theducks is offline   Reply With Quote
Old 08-26-2018, 02:38 PM   #15
bodiccea
Member
bodiccea began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Aug 2018
Device: samsung tablet 8", kitkai
@theducks,

I know last_modified is about metadata change only.
I am less clear about the "date" column. As said, if you make some changes on the book itself (I did read it somewhere it could, if you add a chapter or maybe other changes), this date could be changed too. But again, I am not sure if it is true or not. So using the MAX will be the "secure" way just in case of.

In any case, my main issue (initial question, 1st email of this thread) was to have the file system file modified date the same as the Calibre last_modified date. With or without the MAX function. This is done with my script, which runs once a day a 2:00 AM (it takes seconds to run), the library (changes only) itself is copied to server at 2:30 AM daily.
Don't forget my issue was that that my "last_modified" dates were all lost a few days ago, probably a mistake from me. This is way I changed temporarily the file system date to the "book creation date", which is better, but not good enough, as I think I can recover last_modified from a backup.
bodiccea is offline   Reply With Quote
Reply

Tags
calibre, export date format

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
"Add a book" template like "Save to disk"? vr8ce Library Management 10 06-09-2017 09:16 AM
Title format in "save book to disk"? Phssthpok Library Management 5 11-26-2014 06:18 AM
Request: separate "Preferred output format" from "Save to Disk" format Katsunami Calibre 2 08-24-2014 12:23 PM
"Save single format to disk" trouble vbdasc Calibre 2 12-01-2010 03:30 AM
Propagate a new "Save to Disk" Folder Structure using Calibre? Trickery Calibre 3 09-11-2010 10:27 PM


All times are GMT -4. The time now is 12:03 AM.


MobileRead.com is a privately owned, operated and funded community.