Disappointed in organization options - Page 3

pilotbob · 03-28-2010, 12:45 AM

Quote:

Originally Posted by kovidgoyal

For this one person sure, but if I got into the business of writing code snippets on demand...

That would not end well

Totally understood.

BOb

kovidgoyal · 03-28-2010, 01:11 AM

Incidentally, @rchiav, I do agree that if you have a lot of tags, the tag browser can be cumbersome, which is why one of the items on my serpentine TODO list is to implement a way of defining a set of "preferred tags" i.e. a set of tags that will be displayed in the tag browser and turned into collections on the device, etc.

chaley · 03-28-2010, 08:15 AM

@rchiav: Thank you for your thought-provoking comments. Being forced to think about something can be painful.

I had a similar set of exchanges when I asked about how to make calibre preserve my preferred author name ordering (last, first) when generating author_sort. To distill the responses harsher than they were, what I was told is that calibre works the way the developers want it to work, and why should they take time to build something of no interest to them? This attitude is eminently reasonable. Why should someone donate time and effort for something s/he won't use? Well, the problem annoyed me enough where I decided to look into fixing it. Being semi-retired (and soon fully retired) I had both the time and the desire to learn something new. (If you are curious about who I am, this is my home page.) Perhaps you are in the same situation, having desire, time and inclination.

There is a nice page in the manual that describes setting up a development environment. Unfortunately, that is the easy part. Calibre is large and complex, is built from large and complex tools, and (naturally) reflects Kovid's style of doing things which often doesn't match mine. It took me some days to be sure that a 5-line change to solve my problem was the right change. However, in the end I got the functionality I wanted, so it was worth it.

Since then I have implemented more functions for my own use: tags with exact matching (released), an added GUI column that indicates whether a book is also on a plugged-in device (not released - someone is implementing it differently), and now saved searches (will be released). My next project will probably be tags-as-columns, assuming that Kovid agrees in principle to release it (I haven't asked him yet).

The point of this screed is that if you can't convince someone to build what you want, 'use the source, Luke'. In my experience, Kovid is very open to changes, and will discuss alternatives if what you want to do doesn't fit into his model or future plans.

@Kovid: what about building a 'job builder' that would permit a GUI user to run groups of 'batch' commands on a selected set of books. What I am imagining is something that provides a subset of that provided by the command line calibredb, ebook-meta and ebook-convert commands. As calibre already has a parser, setting up a batch language wouldn't be too difficult. One would probably also want a way to select a set of books -- to run a search. To use it, the user would save the list of commands under some job name, and then run that job whenever wished.

What I don't know is whether the threading implied by the job mechanism would break things. I can imagine lots of ways that it would. If it does, then the job would need to run in the foreground. That would be mildly annoying, but not a deal-killer.

Of course, if command line commands can be used reliably when the GUI is open (the DB supports access by multiple processes), then one could build real batch files. This sort of scheme wouldn't be as convenient, however, because the command line functions 'set' things instead of 'editing' them.

kovidgoyal · 03-28-2010, 08:23 AM

The db does support multi process access, with the proviso that the GUI has to be notified if its running (calibre contains IPC code to do that easily).

Oh and the jobs mechanism is actually multi-proc not multi-thread, for various reasons.

rchiav · 03-28-2010, 12:03 PM

Quote:

Originally Posted by chaley

@rchiav: Thank you for your thought-provoking comments. Being forced to think about something can be painful.

I had a similar set of exchanges when I asked about how to make calibre preserve my preferred author name ordering (last, first) when generating author_sort. To distill the responses harsher than they were, what I was told is that calibre works the way the developers want it to work, and why should they take time to build something of no interest to them? This attitude is eminently reasonable. Why should someone donate time and effort for something s/he won't use? Well, the problem annoyed me enough where I decided to look into fixing it. Being semi-retired (and soon fully retired) I had both the time and the desire to learn something new. (If you are curious about who I am, this is my home page.) Perhaps you are in the same situation, having desire, time and inclination.

I've gotten the same impression that the software works the way specific people want it to work and the answer to most questions seems to be to change how others think to conform to the software. Like you said, that is understandable, but it's also a shame that there's not more of a focus on accommodating the masses and how people who aren't developers think and work.

I'll look into what the plugin framework looks like and see if there's something I can do there. If that doesn't fit the bill, it looks like there are other options where I could create wrappers for a lot of the functionality and create a separate GUI.

kovidgoyal · 03-28-2010, 12:15 PM

Quote:

Originally Posted by rchiav

Like you said, that is understandable, but it's also a shame that there's not more of a focus on accommodating the masses and how people who aren't developers think and work.

Oh and you are the elected representative of the masses are you?

HarryT · 03-28-2010, 12:26 PM

Quote:

Originally Posted by rchiav

I've gotten the same impression that the software works the way specific people want it to work and the answer to most questions seems to be to change how others think to conform to the software.

Of course it does, and if you want a piece of software to work the way that you want, there's nothing to stop you from writing your ideal program. That's the way that many of us got into software development in the first place. My field is astronomy software, and I starting writing what's now become one of the leading programs in that field simply because I didn't like the way that any of the programs that were out there worked, and I wanted one that worked the way that I wanted it to.

rchiav · 03-28-2010, 01:26 PM

Quote:

Originally Posted by kovidgoyal

Oh and you are the elected representative of the masses are you?

Nope. It's the opinion I've developed through reading the message boards, seeing responses to peoples posts and using the application. The responses to people appear to generally be "Do it our way", not "Good idea, but I don't know when we'd be able to get to it". Maybe it's just me, but having to continually reeducate users on how to use an application says that there's a gap that isn't being bridged between the user and the software. To me, if the focus was on being more flexible to users needs then the responses would be entirely different. Users wanting to do something like sort last-name-first is a reasonable thing, but it sounds like because it's not something you want to do, it's not part of the application. And again, there's no expectation that you or anyone else is going to provide any type of functionality that anyone other than yourself wants. I've simply said that it's unfortunate that there isn't more of a focus on creating the flexibility to let people do things the way they'd like, Slippery Slope Fallacy aside.

With that, I'm done because this went exactly where I saw it going when I said I shouldn't have posted.

kovidgoyal · 03-29-2010, 04:28 AM

Have you actually bothered to read the calibre changelog. calibre is released every week with almost every new feature in every release in response to a user request. I defy you to find any software project of comparable size that is as responsive to user feedback.

You say you've read the message boards. The message boards are full of posts where users ask for help/new features, the vast majority of which are responded to positively. In fact, I urge you to go collect some hard data, I wager you'll find that upwards of 75% of user requests are honored.

Basically there is just one class of user request, that I refuse to consider. Requests from people that think they know the One True Way and even more egregiously imagine that the rest of the world shares their One True Way.

rchiav · 03-29-2010, 08:50 AM

Quote:

Originally Posted by kovidgoyal

Basically there is just one class of user request, that I refuse to consider. Requests from people that think they know the One True Way and even more egregiously imagine that the rest of the world shares their One True Way.

That's ironic, because that's exactly what you've done with tags. What I've said is unfortunate is that there's not more options for more people to perform common organizational tasks more quickly - i.e not One True Way. I've never said tags are bad or shouldn't be used, and I even gave an example of how tags could continue to be used, but the UI enhanced to allow people to work faster and perform a common or possibly several common organizational tasks faster and more intuitively. I also by no means suggested that there's only one way to do it. My basis of my whole point is that more options for people is better.

That doesn't seem to be a welcome opinion when it comes to how to organize books in Calibre though. And again, you don't have to implement any feature that anyone suggests. Just don't imply that I'm asking for something to be done One True Way when in fact I'm advocating more options than the One True Way you seem to get so worked up over defending.

itimpi · 03-29-2010, 08:57 AM

I would guess that if you can clearly state how you think it might be done and then raise that as a feature request then it would be considered.

The problem is to change things from a general idea to something that is detailed enough to be considered as something that could be implemented, and also could be done without predjudicing current functionality.

chaley · 03-29-2010, 09:34 AM

Having reread this thread, it seems to me that you get most of what you asking for with the following:

1. Define a set of tags for your workflow. Make them unusual and make them have a common prefix, such as WF:done, WF:step1, WF:step2, WF:notapplicable, WF:in_process

2. after importing books, search for (assuming the above tags) 'not tag:WF:'. This will find all books that do not have a tag with WF: in it. Select all the books, then use the bulk metadata editor to add WF:step1 (or what-have-you) to the books. Once saved searches are available, you will be able to select this search from the searches box or the tags pane.

3. When you process a book, change its workflow state using the bulk metadata editor instead of the single book metadata editor. The bulk editor is capable of adding and removing tags without needing to see/avoid all the other tags, and its completion mode makes it easier to identify the tags you wish to add/delete. You would remove the current step tag and add the next step tag.

4. You can set multiple tags to indicate substeps. For example, when you decide to start working on a book, you can add the WF:in_process tag without removing any other tag. This way you can rapidly find all the books that you are currently working on.

I understand that this doesn't define a real workflow system with nice buttons to move the book to the next state, and also that there is some typing involved to change states. It does (I think) solve your folder problem (where are the new books?) and partially solve the 'lots of clicks and tag editing' problem (bulk editor changes editing all tags to add/delete individual tags).

Starson17 · 03-29-2010, 10:28 AM

Quote:

Originally Posted by chaley

Having reread this thread, it seems to me that you get most of what you asking for with the following:

Chaley,

Your ln, fn tweak is the only "tweak" I've actually heard about. Is the "tweak" option flexible enough to add new buttons the way he seems to want?

As an aside, I'm really glad that your first thread, which seemed to go downhill (partly my fault), ended up not pissing you off, but instead ended with you becoming a valuable contributor. I use your exact search option a lot (I didn't know you wrote that one.) Did you also write the regex search option? I asked if that was possible and a few releases later, it was suddenly there. Whoever wrote that one gets my thanks, too.

When I started with Calibre, I can recall how I wanted ln, fn in the database. That's how I stored my books before Calibre and I thought it should be easy to simply track the firstname and lastname in separate fields. Having worked on the code for a while, however, I now realize how naive that was. The "author" field is used throughout the code and doing what I wanted would have required a major rewrite of almost every bit of code.

=X= · 03-29-2010, 10:40 AM

Quote:

Originally Posted by rchiav

Well I hope I'm still welcome.

I was just hoping I was missing something and there was some hidden options that I didn't find, because otherwise I really liked the app.

Yes you are still welcomed

There is no special feature you are missing in calibre. The way you are trying to use calibre is outside of it's scope. Calibre is a book management software designed for managing "production/released" books.

What you are looking for is a document management product of a different sort, one that manages the workflow process until a document is released or in production. I don't know of any open source project that does this, but there are several commerical ones, with a hefty price tag.

Calibre can be shoe horned to do this but as you noted it would require using tags and it would be a tedious process.

You might want to look at an open source version control software that might have the features your more interested in.

=X=

theducks · 03-29-2010, 10:42 AM

I will support Kovid's 75% number. I have made a few suggestions and have seen (maybe after some discussion) that they were implemented in some form (not always exactly), probably because they triggered the creativeness of the code writers.

Calibre is amazing. It runs on 3 operating systems, supports dozens of reader devices, interfaces with thousands of News Feeds, supports many document types,
All while being updated a couple of times a month. WOW.

When an idea surfaces, it gets filtered by the big picture of the impact to all the parts that work. Do this, and you break x,y and Z, which would take major effort to re-integrate if we made this change (AKA ripple effect).
Tags are evolving. Currently the use of " [tags] " are being implemented I hope that this class of (think of them as a control/note tag) tag will not be included in the "book tags" written into the "jacket section" when a file is converted.

03-28-2010, 08:23 AM	#34
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The db does support multi process access, with the proviso that the GUI has to be notified if its running (calibre contains IPC code to do that easily). Oh and the jobs mechanism is actually multi-proc not multi-thread, for various reasons. Last edited by kovidgoyal; 03-28-2010 at 08:46 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Disappointed with K3	jlmwrite	Amazon Kindle	86	10-09-2010 06:02 AM
Very disappointed	garyg007	Introduce Yourself	18	02-09-2010 01:47 PM
Any suggestions for organization?	kl_0	Calibre	4	02-08-2010 01:44 PM

03-28-2010, 01:11 AM	#32
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Incidentally, @rchiav, I do agree that if you have a lot of tags, the tag browser can be cumbersome, which is why one of the items on my serpentine TODO list is to implement a way of defining a set of "preferred tags" i.e. a set of tags that will be displayed in the tag browser and turned into collections on the device, etc.

03-28-2010, 08:15 AM	#33
chaley Grand Sorcerer Posts: 12,445 Karma: 8012886 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	@rchiav: Thank you for your thought-provoking comments. Being forced to think about something can be painful. I had a similar set of exchanges when I asked about how to make calibre preserve my preferred author name ordering (last, first) when generating author_sort. To distill the responses harsher than they were, what I was told is that calibre works the way the developers want it to work, and why should they take time to build something of no interest to them? This attitude is eminently reasonable. Why should someone donate time and effort for something s/he won't use? Well, the problem annoyed me enough where I decided to look into fixing it. Being semi-retired (and soon fully retired) I had both the time and the desire to learn something new. (If you are curious about who I am, this is my home page.) Perhaps you are in the same situation, having desire, time and inclination. There is a nice page in the manual that describes setting up a development environment. Unfortunately, that is the easy part. Calibre is large and complex, is built from large and complex tools, and (naturally) reflects Kovid's style of doing things which often doesn't match mine. It took me some days to be sure that a 5-line change to solve my problem was the right change. However, in the end I got the functionality I wanted, so it was worth it. Since then I have implemented more functions for my own use: tags with exact matching (released), an added GUI column that indicates whether a book is also on a plugged-in device (not released - someone is implementing it differently), and now saved searches (will be released). My next project will probably be tags-as-columns, assuming that Kovid agrees in principle to release it (I haven't asked him yet). The point of this screed is that if you can't convince someone to build what you want, 'use the source, Luke'. In my experience, Kovid is very open to changes, and will discuss alternatives if what you want to do doesn't fit into his model or future plans. @Kovid: what about building a 'job builder' that would permit a GUI user to run groups of 'batch' commands on a selected set of books. What I am imagining is something that provides a subset of that provided by the command line calibredb, ebook-meta and ebook-convert commands. As calibre already has a parser, setting up a batch language wouldn't be too difficult. One would probably also want a way to select a set of books -- to run a search. To use it, the user would save the list of commands under some job name, and then run that job whenever wished. What I don't know is whether the threading implied by the job mechanism would break things. I can imagine lots of ways that it would. If it does, then the job would need to run in the foreground. That would be mildly annoying, but not a deal-killer. Of course, if command line commands can be used reliably when the GUI is open (the DB supports access by multiple processes), then one could build real batch files. This sort of scheme wouldn't be as convenient, however, because the command line functions 'set' things instead of 'editing' them.

03-29-2010, 04:28 AM	#39
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Have you actually bothered to read the calibre changelog. calibre is released every week with almost every new feature in every release in response to a user request. I defy you to find any software project of comparable size that is as responsive to user feedback. You say you've read the message boards. The message boards are full of posts where users ask for help/new features, the vast majority of which are responded to positively. In fact, I urge you to go collect some hard data, I wager you'll find that upwards of 75% of user requests are honored. Basically there is just one class of user request, that I refuse to consider. Requests from people that think they know the One True Way and even more egregiously imagine that the rest of the world shares their One True Way.

03-29-2010, 08:57 AM	#41
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	I would guess that if you can clearly state how you think it might be done and then raise that as a feature request then it would be considered. The problem is to change things from a general idea to something that is detailed enough to be considered as something that could be implemented, and also could be done without predjudicing current functionality.

03-29-2010, 09:34 AM	#42
chaley Grand Sorcerer Posts: 12,445 Karma: 8012886 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Having reread this thread, it seems to me that you get most of what you asking for with the following: 1. Define a set of tags for your workflow. Make them unusual and make them have a common prefix, such as WF:done, WF:step1, WF:step2, WF:notapplicable, WF:in_process 2. after importing books, search for (assuming the above tags) 'not tag:WF:'. This will find all books that do not have a tag with WF: in it. Select all the books, then use the bulk metadata editor to add WF:step1 (or what-have-you) to the books. Once saved searches are available, you will be able to select this search from the searches box or the tags pane. 3. When you process a book, change its workflow state using the bulk metadata editor instead of the single book metadata editor. The bulk editor is capable of adding and removing tags without needing to see/avoid all the other tags, and its completion mode makes it easier to identify the tags you wish to add/delete. You would remove the current step tag and add the next step tag. 4. You can set multiple tags to indicate substeps. For example, when you decide to start working on a book, you can add the WF:in_process tag without removing any other tag. This way you can rapidly find all the books that you are currently working on. I understand that this doesn't define a real workflow system with nice buttons to move the book to the next state, and also that there is some typing involved to change states. It does (I think) solve your folder problem (where are the new books?) and partially solve the 'lots of clicks and tag editing' problem (bulk editor changes editing all tags to add/delete individual tags).

03-29-2010, 10:42 AM	#45
theducks Well trained by Cats Posts: 31,047 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I will support Kovid's 75% number. I have made a few suggestions and have seen (maybe after some discussion) that they were implemented in some form (not always exactly), probably because they triggered the creativeness of the code writers. Calibre is amazing. It runs on 3 operating systems, supports dozens of reader devices, interfaces with thousands of News Feeds, supports many document types, All while being updated a couple of times a month. *WOW. When an idea* surfaces, it gets filtered by the big picture of the impact to all the parts that work. Do this, and you break x,y and Z, which would take major effort to re-integrate if we made this change (AKA ripple effect). Tags are evolving. Currently the use of " [tags] " are being implemented I hope that this class of (think of them as a control/note tag) tag will not be included in the "book tags" written into the "jacket section" when a file is converted.

Advert

Advert