Custom column: "Updated date", when adding new "versions" of the same file?

[enriquep]

I am trying to create a custom column that allows me to sort my library by date, but taking into account "updated files" - With this I mean new versions of the same eBooks.

I'll try to exlain myself. My intention is to have a way to to track files that have been replaced by new "versions", or files with the same name, same author, same format..., to be able to review them after adding them, as often they have improved metadata etc. That will enable me to "put them together" by sorting them by this "updated date" column.

The current problem is that when these new "versions" are added to the library they are just merged but the "date" and the "modified" fields do not change, so these eBooks are difficult to review after adding new files - you have to manually search for each one of them.

Any ideas on how to do this, or any alternative ideas on how to do what I intend to do?

Thanks in advance, sorry for my limited command of English and greetings from sunny Spain.

[theducks]

Originally Posted by enriquep

I am trying to create a custom column that allows me to sort my library by date, but taking into account "updated files" - With this I mean new versions of the same eBooks.

I'll try to exlain myself. My intention is to have a way to to track files that have been replaced by new "versions", or files with the same name, same author, same format..., to be able to review them after adding them, as often they have improved metadata etc. That will enable me to "put them together" by sorting them by this "updated date" column.

The current problem is that when these new "versions" are added to the library they are just merged but the "date" and the "modified" fields do not change, so these eBooks are difficult to review after adding new files - you have to manually search for each one of them.

Any ideas on how to do this, or any alternative ideas on how to do what I intend to do?

Thanks in advance, sorry for my limited command of English and greetings from sunny Spain.

Modified changes with every change done from Calibre (and metadata affecting plugins).
If you make a change via the side or back doors, then, no the modified does not get bumped.

Side door: Open With PI, Tweak Epub
Backdoor: Open Folder, direct edits

I keep a column "Edited Date just for cases (requires a Manual click of the Today button in the Metadata Editor).

[enriquep]

Originally Posted by theducks

Modified changes with every change done from Calibre (and metadata affecting plugins).
If you make a change via the side or back doors, then, no the modified does not get bumped.

Thanks for your help.

The problem is that that is not quite so: when adding a new "version" of the same eBook the modified date does NOT change (as it doesn't, for example, when reloading an existing eBook file), and therefore I can not (semi)automatically follow up those updated files.

Well, since this "last time file added" field doesn't seem to be easily accessible I'll guess I'll have to go on living without this version tracking!

[theducks]

Originally Posted by enriquep

Thanks for your help.

The problem is that that is not quite so: when adding a new "version" of the same eBook the modified date does NOT change (as it doesn't, for example, when reloading an existing eBook file), and therefore I can not (semi)automatically follow up those updated files.

Well, I'll guess I'll have to go on living without this version tracking!

How are you updating those versions, because the minute you click OK after dropping a new version into the metadata editor, the Modified date (time stamp) gets bumped.
Just passing through using the Previous/next buttons bumps it (no other change performed).

BTW
Modified is a default column that is not shown by default (right click on a column and set it to show). It updates on the Departure from the current Library record, not while editing (which may get canceled).

Sorting the column, brings the most current change to the (trailing/leading) edge of the date grouping.

[enriquep]

Originally Posted by theducks

How are you updating those versions, because the minute you click OK after dropping a new version into the metadata editor, the Modified date (time stamp) gets bumped.

(This is probably irrelevant, but in all cases I am talking of ePub files).

Calibre is my default app for ePub files, so I double click on xxxxx.epub, and it is just added and automerged; I have "Overwrite existing" selected by default, so the warning popup appears. I click OK - and that's it. End of the story.

If the file I have just added is a new "version" of an existing file and I now sort my library by "Modified">"Descending" the file I have just added will not be at the top of the list. I've just checked (again) a few seconds ago.

I don't know if the Preserve the date when using the "Copy to library" action to copy books between libraries setting (Also in the "Adding books" tab) can have anything to do with this, but I've experimented and have not observed any changes.

And, yes, we're talking of the same "Modified" Column, the one that does not appear by default.

The whole story, in short, is that very often I add improved versions of existing books - mainly with minor spelling / editing errors: The "new" versions are usually very similar to the "old" ones. In 99% of cases with no changes in title / author spelling and in 95% of cases with no changes in cover or metadata. When they DO have minor changes in title or author there is no problem, because they show up as "new" books (and it is my problem to detect them as duplicates), but when they don't they just seem to replace the existing version (and in fact they do: I do number versions within the book and check occassionally, the process works OK) BUT, for some reason, Calibre seems to ignore that the file has been updated. The "new" file can exceptionally have the same name as the old one by mistake, but even when it doesn't (usually they are appended with version code, vx.x) there's no change in "updated" column, no change anywhere.

Since I don't edit all the new versions (we are a group of people who, as a hobby, adapt Spanish classics to a strict ePub standard) I like to review all the newly versioned files - for example, to see if the cover has been changed (not common, but can happen) and, if so, to make it Calibre's default. But now (and since there is no way to "automatically" know what files have been replaced) I have to stop adding files every time the warning popup appears, manually search for the file, check, and then continue adding files.

[theducks]

Originally Posted by enriquep

These is probably irrelevant, but in all cases I am talkink of ePub files. Calibre is my default app for them, so I just double click on xxxxx.epub, and it is just added and automerged; I have "Overwrite existing" selected by default, so the warning popup appears - and that's it. End of the story.

Thanks for the clear-up reply. (I don't use your work flow sequence, so you may have found something I have not seen.)

Your work flow could be considered 'front door' or 'side door' (because the O/S called for Calibre to process the file) usage of Calibre: the developer crew will have to call this one

[enriquep]

Thanks for your time, theducks.

Let's see if someone with my workflow has experienced my "problem" or can think of a workaround

[chaley]

The "modified" date is the date the metadata was modified. Adding or modifying a format does not change the metadata, so it does not change the date.

You can get the date that the formats are modified using the formats_modtimes template function. This list is sorted so that the latest change is the first item in the list (accidentally today, guaranteed with the next release). As such, a composite custom column (column made from other columns) containing the following template will show you the date and time of the most recently modified format.

{:'re(list_item(formats_modtimes('iso'), 0, ','), '^.*?:', '')'}

This template

• gets the list of formats and their modification times formatted using the 'iso' date format that includes both the date and the time. (formats_modtimes)
• selects the first (the most recently modified) format (list_item)
• Strips off the format name (re)

Sorting the library by this column should give you what you are looking for.

[enriquep]

Sorting the library by this column should give you what you are looking for.

You are right: It does give me exactly what I was looking for, so many thanks. My problems are now solved.

Now.. if only... and while you're here... I am really interested in being able to sort the files by the order in which new formats were added, but not so much in all the details of the date, hour, minute, second, milisecond and nanosecond when that happenned in GMT - that I'm not saying that it is not VERY interesting, only that in my particular case...

In other words: How can I modify that

{:'re(list_item(formats_modtimes('iso'), 0, ','), '^.*?:', '')'}

that works as a dream (thank you again) to make "the result" as short as possible? - I am lost with all that coding, but a very simple dd mm yy expression will do for me (or something similar).

Thanks again and greetings from sunny Madrid,

[chaley]

You need at least yyyy mm dd to make sorting work. The column is sorted by the string you are seeing, not the real date underneath it, so the order of the date components is important.
Try

{:'re(list_item(formats_modtimes('yyyy-MM-dd'), 0, ','), '^.*?:', '')'}

Take out or replace the - characters if you don't like them.

And you are welcome.

[enriquep]

Originally Posted by chaley

You need at least yyyy mm dd to make sorting work. The column is sorted by the string you are seeing, not the real date underneath it, so the order of the date components is important.
Try

{:'re(list_item(formats_modtimes('yyyy-MM-dd'), 0, ','), '^.*?:', '')'}

Take out or replace the - characters if you don't like them.

That is perfect!! - Trying to keep compatibility with the Spanish standard "Date" format I have used this slight modification, and (... to my surprise, I must say) it has worked:

{:'re(list_item(formats_modtimes('dd/MM/yy'), 0, ','), '^.*?:', '')'}

So I take this as absolutely finished.


Thanks again.

[chaley]

Does the column sort correctly? My guess is no, although it might if you chose "Date sort" when you created the custom column. If you chose text, or if the 'meaning' of the number cannot be determined, then the sort will be wrong. For example, when sorting ascending, 10 Oct 2011 (10/10/11) will come before 12 June 2011 (12/06/11) because 10 is less than 12. The fact that October is larger than June will be ignored.

[enriquep]

Originally Posted by chaley

Does the column sort correctly? My guess is no, although it might if you chose "Date sort" when you created the custom column. If you chose text, or if the 'meaning' of the number cannot be determined, then the sort will be wrong. For example, when sorting ascending, 10 Oct 2011 (10/10/11) will come before 12 June 2011 (12/06/11) because 10 is less than 12. The fact that October is larger than June will be ignored.

Yes, It does sort correctly as far as I see. I did choose "Date sort". The column appears perfectly sorted in descending ordered all the way from today 28/10/2011 to 1/10/2011 (October 1 2011), where it jumps correctly to 30/09/11 (September 30 2011)... etc.

Now the problem I have, and am trying to solve, is that if I add a new file now it does appear somewhere within today's files, mixed with the files that I added, say, a few hours ago - in an apparently random way but not necessarily at the top of the list... My fault: because I didn't include any timestamp (to be honest: I removed it) and because my files come in "working peaks", sometimes quite a few in the same day. So I've thought of to appending "hour:minutes" to the date tag; that will surely be enough, as I'm certain that I will not add more than one file within the same minute!!.

[chaley]

Sorry, the only way to get the time into the string is to use the complicated 'iso' format. The problem you are having is the reason I chose iso to begin with.

[enriquep]

Originally Posted by chaley

Sorry, the only way to get the time into the string is to use the complicated 'iso' format. The problem you are having is the reason I chose iso to begin with.

There ain't no such thing as a free lunch.

I'll stop trying all sort of stupid code combinations and take your word: That's why you're a Wizard, after all