What a regex is

[Worldwalker]

DOS batch files ... heh .... Want to know how crazy I am? When I got my first real programming job, on a PDP-11/34 running RSX-11M, we were at least one major OS release behind the times. Well, okay, the whole bloody machine was behind the times -- full disclosure: this was in 1981, I believe. I'd previously used a VAX 11/780 at a college computer lab, and I missed DCL. The MCR stock CLI sucked. So, out of sheer insanity, I wrote a DCL work-alike ... as a batch file. I even included command aliasing (although I couldn't figure out how to make that persistent, unfortunately). You're really making me feel old. Also making me realize how much I've let my skills slip.

As far as our responses to the troll:

I assume I'm the one you're referring to in "no better than the troll". I make no apologies. I do not like people waltzing into a friendly, peaceful forum and trying to stir up @#$%. I do not like people who insult my friends and especially insult people like Kovid, whom I admire greatly. And yes, we've had several trolls here recently (along with a possible/probable astroturfing incident), and I would not be surprised to find that this was just a recurrence of another who was banned. We don't get nearly as many of them as some forums, of course, but no forum is immune.

I respond to people in accordance with their own attitudes. If someone asks for help, I'll help them to the best of my ability or direct them to someone who can. If someone is bumbling around looking confused, I'll get them sorted out. But if someone comes in here in attack mode, I will damn well defend this forum and its members, and I will make it clear that the intruder's actions and attitude are not acceptable, and their accusations and insinuations are wrong. Most people here are very nice, polite, trusting, and kind; they see deliberate trolling as honest misunderstanding, because that's why they would make a post like that, and get drawn into the troll's trap. I am none of those things, which makes it easier to spot trolls for what they are. There is absolutely no question in my mind that individual was a troll. His first post could be summarized as "That software you all like so much is trash, and I know you're not going to agree with me because you're all really stupid." You may have missed most of his second post, which Kovid heavily edited before locking the thread, but it was nasty.

Should a friendly, welcoming community extend that friendship and welcome to a person who is actively working to disrupt or destroy that community?

I say no.

Defending oneself, one's family, one's home, or one's community does not put one on an equivalent moral level with whoever is attacking them. And we are a community of adults here, not a group of children who need to wait for a teacher to come settle their schoolyard disputes. We have not only the ability but IMO the obligation to speak out in defense of the standards of our community (something which, sadly, many offline communities have abandoned in favor of the "children in a schoolyard" model, where any action not illegal is also unreproachable).

Was I going to convince the troll of anything? Of course not. He was just a ROM-head; you pretty much have to be, to be a troll. I wasn't really writing for him. I was writing for the lurkers, and especially for the people he was attacking. I'm sure Kovid knows we've all got his back, but sometimes a public demonstration is beneficial. And I was writing to define what was out of bounds by our community standards, defining it in the way that it should be done: from within the community, by community members saying "what you just did is wrong", not by some drafting committee of some study board writing up a mealy-mouthed policy statement full of neither sound nor fury, but still signifying nothing.

Anyway, he seems to be gone, and we can return to talking about more important things, like regexes.

P.S. Don't apologize for long posts. There is one thing that everyone in this forum shares: we're all readers.

[kovidgoyal]

Originally Posted by Starson17

From reading very old threads, I believe the earliest versions of calibre stored ebooks as real BLOBs within the calibre database. From what I read, the change to pseudo-BLOBs stored in a computed directory path was made to improve performance.

Performance and also to allow easy differential backups of the calibre library over low bandwidth links.

[Disfrutalavida]

Thanks for all the good info Chaley.
Long posts like yours are very welcome.

I've been out of programming for 15 years, and as everything else things move on.
Glad to see the number field can go much higher. I expected it could, but I have seen some really, really good programmers make simple errors before.

I dislike "black boxes" intensely. I always want to know how things work.

It's been a curse and a blessing my entire life, and it's why I dislike OS X and Windows 7 so much but that's for a different thread.

I don't expect a rewrite of the library storage code. Not worth the hassle.

I don't see how referencing the files by the (xxx) number in the folder would cause data integrity problems, but without knowing more about how the tables are setup I will defer to you.
We may be talking about 2 different things.
There is how you can save files externally, and how Calibre saves files to it's library.
In my rather rigid mindset I would like the basic library file structure to be the same as
the database field structure. It's just easier to visualize, though I am aware it's just as easy to make it any way a programmer wants.

I didn't think about Asian languages, since I never had to deal with them in my work which was all with US companies and government.
Funny since I had a Chinese fiancee' who was Lin Fang in Chinese, but Fang Lin in English.

Glad to see someone else as old as I around here.

Thanks and nice to meet you,

Rich

[Disfrutalavida]

Worldwalker, I wasn't referring to anyone specific, I just noticed the overall tenor of the responses. What is an astroturfing incident?

Sorry, but I really don't spend much times on forums as the weather is too good, my fiancee' too beautiful and life too short.

I got a very different impression of what the "troll" wrote, but as I said, I don't spend much time on forums and was taught to simply ignore people if I don't like what they say unless they are spouting bigotry etc, but I certainly understand and respect your position.

I'll continue to "lurk" here as I find I learn a lot more and ask fewer questions which have already been answered.

I read about paid trolling which was hard to believe. How do people live with themselves? But where is the origin of Astroturfing as a reference for that sort of attack ad why the heck would they be doing it on this forum? What's to gain?

Guess one of the reasons I moved out of the USSA is the sort of &*#& people will do for money and the US seems to be #1 among cultures that says that's OK.
Of course the climate, cost of living, fresh, cheap organic food, lower crime rates, and respect for older people might have had something to do with it also. :-))

Rich

[chaley]

Originally Posted by Disfrutalavida

I dislike "black boxes" intensely. I always want to know how things work.

I agree with you here! I try very hard to look in the boxes. Haven't had good luck with changing their contents, though.

I don't see how referencing the files by the (xxx) number in the folder would cause data integrity problems, but without knowing more about how the tables are setup I will defer to you.

Having spent a fair amount of time inside calibre recently, I think I can infer some basic design decisions. I am not 100% certain, as I wasn't around when Kovid did the design work, but here goes.

Basically, calibre does four things: stores data about publications in a way similar to bibliographic managers like Zotero or Endnote, converts publications between formats, stores those formats for use in various ways, and interfaces with reading devices. We can argue about whether it should do these things, but that is beside the point.

Because of 'things' 2-3, and to some extent 4, calibre wants to have guaranteed access to formats either supplied by the user or that it creates. I don't want to say 'books', because even though a book might exist in several forms, it is still the same book from the metadata standpoint. In order to guarantee access, it insists that the path to a format be 1) computable using book metadata, and 2) unique. BLOBs in a database have these properties, but are not ideal for reasons ranging from incremental backup capability to single errors causing complete loss of data.

There are many schemes that can conform to these two criteria, and yours is one of them. I can imagine a folder containing thousands of files named nnn.format, but that raises the problem of folder size. Kovid probably wanted a scheme that partitions the file space into folders that weren't so large as to overflow some OS's folder capacity or run so slowly that it is unusable. He probably also wanted a structure that a human could make sense of, so that backup/recovery is easier.

One scheme that does not conform is storing references to files contained in random folders. Neither constraint is met: the path is not computable and there is no guarantee of uniqueness. In addition, because the folders are 'owned' by something other than calibre, the problem of data integrity arises. The files can easily be moved, altered, or even deleted.

Clearly Kovid could have made different choices. For example, he could have implemented libraries in the same fashion as MediaMonkey, which supports a DB + path-to-file scheme. However, different choices lead to different problems. For example, I use MediaMonkey, and several times been required to rebuild the library because of mistakes and naming scheme clashes. MM supports a calibre-like scheme where MM renames and copies my music into a 'library', which I have used for some years now because it makes library maintenance so much easier. But all that aside, Kovid made the choices he did, and either we live with them, go somewhere else, or get into the code and change them.

I didn't think about Asian languages, since I never had to deal with them in my work which was all with US companies and government.
Funny since I had a Chinese fiancee' who was Lin Fang in Chinese, but Fang Lin in English.

I ran into this problem while living in Malaysia. We had to deal with Chinese, Indian, and Arabic naming schemes, and no system worked. For example, I have a friend named nik xx binte yy, which means approximately Lady xx daughter of yy. She insists that she does not have a last name. Another friend is of Palestinian origin, and his name mixes systems (his-fn son of fathers-fn ln); his problem is that his father's name is continuously assumed to be his middle name (something he does not have). I had Chinese students who would use their Chinese name in the traditional way (LN FN) but also had western names written FN LN that usually weren't related to Chinese at all. Add to all of this European names like Werner von Braun, where 'von' is both part of and not part of the last name (last name is written von Braun but should be sorted under B), and I pull (what is left of) my hair out.

Glad to see someone else as old as I around here.

Charles

[Worldwalker]

The astroturfing involved first one person, then multiple people, who showed up to promote a company that had been reported by forum members to be taking money and not delivering products. It got messy. As far as what's to gain, sales and/or reputation for the company in question.

That's called "astroturfing" because it's an artificial imitation of "grassroots" activity. You see it a lot in fake "community" groups that just happen to be promoting some organization/company/politician's viewpoint, "informational" websites that are secretly owned by whoever's side they're pushing, bogus blogs that look legitimate but are really selling some product or agenda, etc. One notable example from my own experience was a "grassroots" group that was opposed to a river cleanup, which turned out to have been fabricated by ... wait for it ... the company that dumped the toxic waste in the river in the first place. "We're happy with the river the way it is" turned out to mean "because we don't want to pay to clean up our mess." Online, it goes back at least to the Windows vs. OS/2 days, when people who were later found to be Microsoft employees started spamming up the OS/2 Usenet groups with posts trashing OS/2.

There's also what you might call "reverse astroturfing" where the goal is not to promote your own sponsor but to bring the opposition into disrepute. The classic example is hiring a bunch of ... I suppose the PC term would be "hygiene-challenged individuals of no permanent address" ... to hand out your opponent's campaign literature and be as obnoxious about it as possible. The more modern version involves chain emails and robo-calls.

Both forms show up on forums. Due to their fundamental dishonesty, starting with lying about who they are or represent, they are a Bad Thing if they move into your forum.

Trolls, of course, are people who clearly don't get enough attention at home, and would rather be hated than ignored. Nope, doesn't make sense to me either. They're a chronic problem, and as their goal is to disrupt the forum, and they will keep on trying until they succeed, the only real solution is the ban stick.

And yes, that fellow was a troll. Legitimate users do not show up in a forum, insult everyone in it, and then when they get negative reactions, say that just proves their point that the forum members and their interests/topics are deserving of insults. Trolls, on the other hand, operate exactly that way, and they also try to set forum members against each other by playing on the natural trust of those with less exposure to that sort of behavior.

As you might guess, I spend time in places more attractive to trolls than a forum full of mild-mannered bookworms *pointedly ignores the RIAA flamewar going on a few sections up* and I've developed a certain sensitivity to their patterns of operation. Aside from his choice of targets (politics, video games, and the eternal Mac/Win battle are all more common) he almost was a textbook case.