Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 09-11-2019, 07:03 PM   #1
Cool Javelin
Member
Cool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a Texan
 
Cool Javelin's Avatar
 
Posts: 23
Karma: 18222
Join Date: Aug 2012
Device: Kindle (early)
Adding books regular expressions a real PITA

The expressions method for adding books is a real great idea. It allows for flexibility in setting stuff up like author name, title, series, etc...

But entering the expressions in the "nuts and bolts" format is a real PITA.

Each time I want to add books, the filename downloaded may have the author name first, or not. The title first, or not, it may include a series name, or not.

Perhaps I want to get the metadata from within the file and that metadata is incorrect where it has the author name where the title belongs or other mistakes.

Using a complex character structure like...

^((?P<author>([^\-_0-9]+)(?=\s*-\s*)(?!\s*-\s*[0-9.]+)|\b))(\s*-\s*)?(\[?(?P<series>[^0-9\-]+) (- )?(?P<series_index>[0-9.]+)\]?\s*-\s*)?(?P<title>.+)

while detaild in the extreme, it is very difficult for us mere humans to remember and having to refer to the tutorial each time I want to change the expression turns a ten minute job into a few hour task.

Is there a plan in the works to streamline the task, or a plugin to set this expression for me?

I'd like to use a much higher level method to specify the expression.

Like having a series of options to choose from (say on the left) and build a list (say on the right) with add or remove buttons, then "move up" and "move down" arrows to rearrange the list.

Once the list is created have further options to refine the list items. For example: options for selecting hyphens or commas or other...

Also, having constants like the author name be allowed as that info may not be in the filename at all.

Once the list is built, have a "go" button to create the expression for me then I can hit the "add books" button.

Thanks, Mark.
Cool Javelin is offline   Reply With Quote
Old 09-11-2019, 07:33 PM   #2
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 23,532
Karma: 27600000
Join Date: Aug 2009
Location: The Central Coast of California
Device: K4NT, Galaxy Tab A, Kobo Aura2
That method is for WHEN the book does not have proper metadata and YOU chose to use the file name.
most publishers do it right and embed the proper way. It is only Series (and series_index) that is non-standard..
FWIW I use the Quality check Plugin's FIX section to flip Author names (flip Ln,Fn to Fn Ln or the other way)

Bulk or Single Metadata edit has a button to interchange Title and Author
theducks is online now   Reply With Quote
Advert
Old 09-12-2019, 12:10 AM   #3
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 14,384
Karma: 12542140
Join Date: Mar 2012
Location: Sydney Australia
Device: none
If it were to be done, it would probably have to be done in a plugin, and it would need to be a File Type plugin (they run as books are added). IIRC such plugins can't be interactive when books are added, I only know of two File Type plugins - Alf's DRM gadget and the GetFileName plugin.

Almost all plugins have been developed by the people who need them, I've sometimes seen people offering to pay for a plugin - not sure if they ever found anyone.

I make any adjustments to the file names using file rename utilities before I add files to calibre - Xplorer²'s rename function and/or Bulk Rename Utility. I don't try to get them perfect. I add to an Intake library where I make any metadata improvements/adjustments before moving the books to the target library.

BR

Quote:
Originally Posted by Cool Javelin View Post
The expressions method for adding books is a real great idea. It allows for flexibility in setting stuff up like author name, title, series, etc...

But entering the expressions in the "nuts and bolts" format is a real PITA.

Each time I want to add books, the filename downloaded may have the author name first, or not. The title first, or not, it may include a series name, or not.

Perhaps I want to get the metadata from within the file and that metadata is incorrect where it has the author name where the title belongs or other mistakes.

Using a complex character structure like...

^((?P<author>([^\-_0-9]+)(?=\s*-\s*)(?!\s*-\s*[0-9.]+)|\b))(\s*-\s*)?(\[?(?P<series>[^0-9\-]+) (- )?(?P<series_index>[0-9.]+)\]?\s*-\s*)?(?P<title>.+)

while detaild in the extreme, it is very difficult for us mere humans to remember and having to refer to the tutorial each time I want to change the expression turns a ten minute job into a few hour task.

Is there a plan in the works to streamline the task, or a plugin to set this expression for me?

I'd like to use a much higher level method to specify the expression.

Like having a series of options to choose from (say on the left) and build a list (say on the right) with add or remove buttons, then "move up" and "move down" arrows to rearrange the list.

Once the list is created have further options to refine the list items. For example: options for selecting hyphens or commas or other...

Also, having constants like the author name be allowed as that info may not be in the filename at all.

Once the list is built, have a "go" button to create the expression for me then I can hit the "add books" button.

Thanks, Mark.
BetterRed is offline   Reply With Quote
Old 09-12-2019, 03:35 AM   #4
kacir
Wizard
kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.
 
kacir's Avatar
 
Posts: 3,356
Karma: 5555555
Join Date: May 2006
Device: PocketBook 360, before it was Sony Reader, cassiopeia A-20
Quote:
Originally Posted by Cool Javelin View Post
The expressions method for adding books is a real great idea. It allows for flexibility in setting stuff up like author name, title, series, etc...

But entering the expressions in the "nuts and bolts" format is a real PITA.
There is a plugin that remembers numerous Regular Expressions and lets you choose the one to use. Quick Preferences.

https://www.mobileread.com/forums/sh...d.php?t=118776

You still have to supply your expressions, but you only do this once, or when you come across a bunch of books that you need to import that has different naming scheme. Usually you can get away with a complicated RE for parsing
Author Name - optional Series Name [number in a bracket or naked] - Book Title and another for Book Title - Author Firstname Lastname. When I used to import a bunch of books with Lastname, Firstname format, I used Edit books in batch dialog and used regular expressions to swap author names post import.

Send me a PM if you want my [small] collection of import REs. I do not have it available at this computer. It has been a long time since I needed to import a bunch of files that did not have decent metadata included in epub or mobi or other ebook format.

Nowadays I usually just let Calibre parse metadata. You usually need import patterns for recognizing file names when it comes to fan-fiction or other types of documents or perhaps a very old archive of *.txt, *.html or *.doc files.

Last edited by kacir; 09-12-2019 at 03:47 AM.
kacir is offline   Reply With Quote
Old 09-12-2019, 11:12 AM   #5
Gary_M_Mugford
Groupie
Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.Gary_M_Mugford has a complete set of Star Wars action figures.
 
Gary_M_Mugford's Avatar
 
Posts: 173
Karma: 299
Join Date: Jul 2010
Location: Brampton ON
Device: Kobo, Kindle3
Cool Javelin,

It wouldn't be that hard to write an AutoHotkey macro that would input the impressive regex you've created AND bring up a notepad with the text of your tutorial in a separate window so that you can customize from your notes on the spot. Or you can use Quick Preferences for an internal solution.

Kacir,

I intend to PM you for the collection, if I can. I still struggle with Regex and ready-made examples will help me traverse the intake of the many papers, created by me and by students. I'm very appreciative for any that you can provide me. Thank You.
Gary_M_Mugford is offline   Reply With Quote
Advert
Old 09-12-2019, 01:11 PM   #6
Cool Javelin
Member
Cool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a TexanCool Javelin might easily be mistaken for a Texan
 
Cool Javelin's Avatar
 
Posts: 23
Karma: 18222
Join Date: Aug 2012
Device: Kindle (early)
Oh my, people, Thanks for the quick and great responses.

I did think about using a plugin to swap author and title after the fact, It just feels a bit like fixing an issue rather then preventing it in the first place. It is an option I will look into. It does seem the author vs. title swap is the most common issue.

I also have Bulk Rename and use it from time to time. Currently I am downloading and adding a bunch of books in a series one at a time. Bulk Rename works best when doing, well, a bulk of renames. But, I guess I can change my errant ways.

The "Quick Preferences" idea sounds like pretty good option. I thought about saving the preferences in a text file and doing a copy and paste. I may do as I can put a description to remind me what each does.

Thanks, Mark.
Cool Javelin is offline   Reply With Quote
Old 09-12-2019, 05:00 PM   #7
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 14,384
Karma: 12542140
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by Cool Javelin View Post
But, I guess I can change my errant ways.
The only errant way is the way that doesn't work best for you

BR
BetterRed is offline   Reply With Quote
Old 09-15-2019, 12:37 PM   #8
kacir
Wizard
kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.kacir ought to be getting tired of karma fortunes by now.
 
kacir's Avatar
 
Posts: 3,356
Karma: 5555555
Join Date: May 2006
Device: PocketBook 360, before it was Sony Reader, cassiopeia A-20
Quote:
Originally Posted by Cool Javelin View Post
I did think about using a plugin to swap author and title after the fact, It just feels a bit like fixing an issue rather then preventing it in the first place. It is an option I will look into. It does seem the author vs. title swap is the most common issue.
The Quick preferences plugin does have a tickmark for "Swap Names" This might be what you are looking for.
Just make two identical entries and on one of them check the option to "Swap Names"

When dealing with non-standard authors, the format is usually Firstname Lastname, or Lastname, Firstname
Sometimes the coma separates multiple authors.
So, when I imported a number of documents this way, I searched calibre for
authors:";"
then I selected the documents where the semi-colon separates multiple authors and replaced ';' with '&'

Please notice that you can save your favorite search and replace Regular Expressions and this will be one of them.

then I searched for
authors:",[^,]*,"
selected all cases where the commas separates multiple authors and replaced ',' with '&'

Having dealt with author names with multiple comas I searched for
authors:","
selected all cases where the comma separates multiple authors and replaced ',' with '&'

Now I can finally deal with Surname, Name case, so I search for
authors:","
replace
([^,]*), (.*)
with
\2 \1

Calibre remembers search queries, the bulk edit Search and Replace dialog can save your replacements, so this can be done very quickly if you use Calibre to process various weirdly named documents, fanfiction or that 10* year old directory with books in txt format.

You can also write script in your favorite file manager or operating system scripting language (I use Linux shell) and process or filter the filenames before importing into Calibre. There are numerous programs that people use to rename or sort files with photos mp3s. I used Krusader and krename on several occasions. TotalCommander, and its bulk rename of course when processing files on Windows.
kacir is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Help with regular expressions for adding books Vortex Library Management 5 12-11-2014 05:07 PM
Please help with regular expressions when adding books gill_d Library Management 1 10-25-2014 04:25 AM
Could use a bit of help with regular expressions to edit books on conversion Flammy Conversion 3 12-29-2011 10:29 AM
Adding books - need help with regular expressions tweebee Library Management 10 08-05-2011 08:58 PM
Custom Regular Expressions for adding book information bigbot3 Calibre 1 12-25-2010 06:28 PM


All times are GMT -4. The time now is 12:27 PM.


MobileRead.com is a privately owned, operated and funded community.