Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Kobo Reader > Kobo Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 09-19-2019, 11:18 PM   #16
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,018
Karma: 17701112
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, (PW) & PW2; Kobo H2O & Forma
@rtiangha: 2 backslashes.

That commit was simply meant as a reference to me messing up the regex twice today ^^. And to illustrate the fact that nickel could happily purge previously indexed content if you mess it up badly enough .

The content of my earlier post is correct, and is what most users should care about, not me losing my mind in commit logs .

EDIT: The live version of this block of code has been fixed to use less confusing escaping shenanigans .

Last edited by NiLuJe; 09-19-2019 at 11:35 PM.
NiLuJe is offline   Reply With Quote
Old 09-19-2019, 11:54 PM   #17
geek1011
Wizard
geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.
 
Posts: 1,692
Karma: 4214636
Join Date: May 2016
Location: Canada
Device: Kobo Mini, Aura Edition 2 v1, Clara HD
Quote:
Originally Posted by NiLuJe View Post
@rtiangha: 2 backslashes.

That commit was simply meant as a reference to me messing up the regex twice today ^^. And to illustrate the fact that nickel could happily purge previously indexed content if you mess it up badly enough .

The content of my earlier post is correct, and is what most users should care about, not me losing my mind in commit logs .

EDIT: The live version of this block of code has been fixed to use less confusing escaping shenanigans .
There's one trick I sometimes do when multiple layers of escaping are needed. If you replace \. with [.], it essentially does the same thing, but without backslashes.
geek1011 is offline   Reply With Quote
Advert
Old 09-20-2019, 12:42 AM   #18
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,018
Karma: 17701112
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, (PW) & PW2; Kobo H2O & Forma
That works when you happen to not need to escape square brackets, yeah .

Last edited by NiLuJe; 09-20-2019 at 09:45 AM.
NiLuJe is offline   Reply With Quote
Old 09-20-2019, 01:11 AM   #19
dmapr
Evangelist
dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.dmapr holds these truths to be self-evident.
 
Posts: 458
Karma: 126922
Join Date: Sep 2009
Device: Kobo Aura HD, Kobo Aura One
Quote:
Originally Posted by frostschutz View Post
they ignore .add (which nobod uses?) but not .adds or .addons

but they also ignore (not hidden) xadd, madd, ladd, ...

regexp 101, I guess
Quote:
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
Source
dmapr is offline   Reply With Quote
Old 09-20-2019, 04:27 AM   #20
frostschutz
Linux User
frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.
 
frostschutz's Avatar
 
Posts: 2,150
Karma: 5050906
Join Date: Sep 2010
Location: Heidelberg, Germany
Device: H2O, StoryHD, PW4
Quote:
Originally Posted by NiLuJe View Post
It does lead to the subsequent deletion of the content (at least if it happened to be in the db previously). So that'd potentially work for *new* content only .
...

So we're deleting user files without asking now?

What were they trying to fix and what's the point of it all... it's a mystery to me.
frostschutz is offline   Reply With Quote
Advert
Old 09-20-2019, 09:06 AM   #21
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 18,635
Karma: 30757200
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by frostschutz View Post
...

So we're deleting user files without asking now?

What were they trying to fix and what's the point of it all... it's a mystery to me.
No, they are deleting details from the database for files that the import process can no longer find. Which is basically what has always been done. But, there is a new reason for not finding the file. Before, the file had to be deleted, renamed or moved. Now, we also have this filter changed to not be able to search where it is.
davidfor is offline   Reply With Quote
Old 09-20-2019, 09:46 AM   #22
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,018
Karma: 17701112
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, (PW) & PW2; Kobo H2O & Forma
Yup, it basically trips the "oh no, file is gone" check, and it turns out that, besides purging the database, it also purges the actual file (which I assume is done as part of the process of deleting the associated sidecar/thumbnails), something which we had no really easy way to know/trigger before, as @davidfor just detailed the most common ways to trip that check, and they all involved the file being gone in the first place .

TL;DR: One could argue that borking the regex *was* in fact "asking for it" .

Last edited by NiLuJe; 09-20-2019 at 09:59 AM.
NiLuJe is offline   Reply With Quote
Old 09-20-2019, 09:56 AM   #23
frostschutz
Linux User
frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.
 
frostschutz's Avatar
 
Posts: 2,150
Karma: 5050906
Join Date: Sep 2010
Location: Heidelberg, Germany
Device: H2O, StoryHD, PW4
Okay, I guess it makes sense to delete files (generated thumbnails and the like) in addition to database purging. It's just weird if they delete the non-generated stuff too... oh well.
frostschutz is offline   Reply With Quote
Old 09-21-2019, 05:25 AM   #24
baskerville
Addict
baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.
 
baskerville's Avatar
 
Posts: 251
Karma: 304054
Join Date: Aug 2015
Device: Kobo Glo HD, Kobo Aura ONE
Quote:
Originally Posted by NiLuJe View Post
In case (not) ignoring kobo & adobe happens to be meaningless, \\..*? is enough.
Why would .kobo and .adobe need to be included?

Have you found the default value for ExcludeSyncFolders?

I'd rather ignore hidden directories/files at any depth:
Code:
[FeatureSettings]
ExcludeSyncFolders=([^.][^/]*/)*\\..+

Last edited by baskerville; 10-06-2019 at 11:37 AM.
baskerville is offline   Reply With Quote
Old 09-21-2019, 08:23 AM   #25
geek1011
Wizard
geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.geek1011 ought to be getting tired of karma fortunes by now.
 
Posts: 1,692
Karma: 4214636
Join Date: May 2016
Location: Canada
Device: Kobo Mini, Aura Edition 2 v1, Clara HD
Quote:
Originally Posted by baskerville View Post
Why would .kobo and .adobe need to be included?

Have you found the default value for ExcludeSyncFolders?

I'd rather ignore hidden directories/files at any depth:
Code:
([^.][^/]+/)*\\..+
NiLuJe was using a negative lookahead for Kobo and Adobe. Regarding the default value(s), have a look at the attached files here (at 0x5771E2): https://www.mobileread.com/forums/sh....php?p=3892386 .

Last edited by geek1011; 09-21-2019 at 08:25 AM. Reason: Added offset
geek1011 is offline   Reply With Quote
Old 09-21-2019, 10:56 AM   #26
baskerville
Addict
baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.
 
baskerville's Avatar
 
Posts: 251
Karma: 304054
Join Date: Aug 2015
Device: Kobo Glo HD, Kobo Aura ONE
Quote:
Originally Posted by geek1011 View Post
NiLuJe was using a negative lookahead for Kobo and Adobe.
I understood that. .kobo doesn't match the regexp. But why would we want that?

Quote:
Originally Posted by geek1011 View Post
Regarding the default value(s), have a look at the attached files here (at 0x5771E2): https://www.mobileread.com/forums/sh....php?p=3892386.
Thanks, that's impressive! So the default regexp looks like this:
Code:
(.kobo|koboExtStorage)/(articles|images|kepub|screensaver|slideshow|dict|certificates|webcache|webstorage)|.fseventsd|.Spotlight-V100|.Trash|.Trashes|.Trash-\\d+|\\$RECYCLE.BIN
?
baskerville is offline   Reply With Quote
Old 09-21-2019, 11:14 AM   #27
frostschutz
Linux User
frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.frostschutz ought to be getting tired of karma fortunes by now.
 
frostschutz's Avatar
 
Posts: 2,150
Karma: 5050906
Join Date: Sep 2010
Location: Heidelberg, Germany
Device: H2O, StoryHD, PW4
Quote:
Originally Posted by NiLuJe View Post
The thing to remember is that this is sandwiched inside a
Code:
^/mnt/(sd|onboard)/%1$
so, can't use ^ or $, and that QSettings probably wants some stuff escaped, hence \\. for \.


Code:
Sep 21 17:08:42 nickel: (    22.221 @ 0x12dd9c0 / ui.debug) void SyncFileSystemCommand::findFilesToParse(const QString&, QStringList&) skipping "/mnt/onboard/.adobe-digital-editions" , matches "\x00|/\.|\x00"
this breaks out of the sandwich (making sure it doesn't match by requiring binary \0 character) and then should go on to ignore hidden folders at any depth

Last edited by frostschutz; 09-21-2019 at 11:16 AM.
frostschutz is offline   Reply With Quote
Old 09-21-2019, 11:17 AM   #28
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,018
Karma: 17701112
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, (PW) & PW2; Kobo H2O & Forma
@frostschutz: You'll have to walk me through that one... o_O .

EDIT: Thanks! Ooh, that's sneaky xD.

EDIT²: Thankfully, as we found out with the preload hack, blacklisting a toplevel hidden folder is enough to make it entirely non-traversable. But this could still be useful for more esoteric directory structures ...

@baskerville: Yeah, the idea was to let the default patterns do their stuff for adobe/kobo folders. Again, not 100% convinced it makes any difference in practice (at least for my use cases), in which case, (EDIT: a top-level only variant of) what you proposed (i.e., \..*? [-> \\..*? in the config] once you account for the full pattern) works just fine.

Last edited by NiLuJe; 09-21-2019 at 12:48 PM. Reason: Oops, fixed it.
NiLuJe is offline   Reply With Quote
Old 09-21-2019, 12:26 PM   #29
baskerville
Addict
baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.baskerville ought to be getting tired of karma fortunes by now.
 
baskerville's Avatar
 
Posts: 251
Karma: 304054
Join Date: Aug 2015
Device: Kobo Glo HD, Kobo Aura ONE
Quote:
Originally Posted by NiLuJe View Post
Again, not 100% convinced it makes any difference in practice (at least for my use cases), in which case, what you proposed (i.e., \..*? [-> \\..*? in the config] once you account for the full pattern) works just fine.
Thanks. Please note that this isn't the regexp I've suggested: I also want to ignore the hidden directories inside the visible directories.
baskerville is offline   Reply With Quote
Old 09-21-2019, 12:46 PM   #30
NiLuJe
BLAM!
NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.NiLuJe ought to be getting tired of karma fortunes by now.
 
NiLuJe's Avatar
 
Posts: 10,018
Karma: 17701112
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, (PW) & PW2; Kobo H2O & Forma
@baskerville: Yep, I should have made that clearer. Would your example work, given the "sandwich", or would that need some trickery like @frostschutz did?

(I'm not completely terrible at regexps, but I'm no wizard either, so, err, small words, please ;p).

EDIT: If I put my brain in "regexp mode", I think it looks fine for that purpose, but I haven't tested it . (i.e., accepts zero or more "classic" directories in front of an hidden one).

EDIT²: The first group might fail to match for a single-character folder name, though, right?

Last edited by NiLuJe; 09-21-2019 at 12:56 PM.
NiLuJe is offline   Reply With Quote
Reply

Tags
hack, hidden, import, kobo, ldpreload

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to prevent replacement of cover.jpg in kepubs when sending to Kobo e-reader? JeanPierre Kobo Reader 2 01-20-2019 04:57 AM
Kobo should be incapable to prevent the publishing of a stolen epub? bathop General Discussions 36 06-04-2016 05:49 PM
Arc Importing to Kobo Arc Roadkill Kobo Tablets 2 08-04-2013 07:18 AM
Dragging and Dropping Folders to Kobo david_e Kobo Reader 2 10-24-2012 06:20 PM


All times are GMT -4. The time now is 04:58 AM.


MobileRead.com is a privately owned, operated and funded community.