Graphing libraries for Qt?

kiwidude · 01-13-2012, 05:07 AM

I've got the beginnings of another plugin idea, which I am tentatively exploring the feasibility of. It is to do with producing statistical graphs/reports of books in your library. At its simplest it would be not dissimilar to what you get on Goodreads for instance allowing you to see books read by time periods (assuming you have a date read column), or books by date published, books by date added etc. In theory you could do books by genre tags or whatever etc as well, just really down to what people would find interesting.

However when it comes to graphing using Qt, according to the FAQ they say you are basically on your own, and suggest including some third party libraries. link

I don't think that me compiling up a Qt library and including in a plugin is likely to work very well, methinks there be dragons in there (compilation/cross platform etc) even if technically it did work. I've only bundled python libraries such as with Goodreads Sync previously, not C++ dlls.

So the question is most likely for Kovid - do you perchance already include any Qt graphing libraries currently in your calibre distribution, and if not would you have any interest in adding to your build process to do so? Perhaps you may have some feature in mind for calibre down the line which might use it? Or do you have any alternative suggestions?

If it all falls into the too hard basket I'll just forget the whole idea - I have close enough to what I need from Goodreads currently.

kovidgoyal · 01-13-2012, 07:06 AM

I dont distribute any graphing libraries with Qt. There are however python graphing libraries you could probably use to generate the graph as an image and then display it with Qt. Though I dont know if any of them are pure python, or if they include compiled code (if the latter, you will not be able to load the compiled parts directly from the zip file). Youd have to extract to file sytsem, load manually and then proceed.

kovidgoyal · 01-13-2012, 07:19 AM

I've used matplotlib in the past, to generate 3D realtime plots of a quantum computer simulation for my thesis, so it is definitely powerful enough

Whether it is usable from a zip file, I'm not so sure. I do know it has compiled code, but whether any of that is actually essential, I cannot recall. Try it and see.

kiwidude · 01-13-2012, 07:53 AM

Thanks Kovid, I will do some googling on the Python angle then to see what shakes out.

kovidgoyal · 01-13-2012, 08:12 AM

If you're serious about this plugin (which I think is a rather cool idea, something I always wanted to implement myself) and you cannot find a suitable pure python library, I'm willing to do the work to add some graphing library to the calibre distribution. The caveat being that in that case, I may want to end up adding your plugin to base calibre

kiwidude · 01-13-2012, 10:15 AM

I'll take a look over the weekend sometime and see what is out there, I hate getting in your way for stuff like this as know you have your own never-ending todo list. Appreciate the offer for sure, may well end up taking you up on that if it gestates beyond the current random idea stage.

If you or anyone has any suggestions about features or how it would work etc please feel free to chip in, it might help motivate me to pull it together. I have given no thought as yet as to how it might work etc. I just figure given many calibre users spend hours collecting minutae of data about their books and reading, that giving them some additional pretty output might be of interest. Certainly the stats pages on goodreads showing me how many pages I have read each year, how many books each year etc I find interesting, particularly if you happen to set yourself some goals for a bit of fun.

Clearly there will need to be some level of customisation allowing users to setup their own desired graphs, though where these will be rendered (modeless window) and what it all looks like is all up for grabs... ideas welcomed.

kovidgoyal · 01-13-2012, 12:31 PM

For me, I would like answers to questions along the time axis:

- How has my reading rate changed (proxied by number of books added per month or some such)

- How has the average rating of books I read changed

- How have the most frequent 5 tags in the books I read changed?

- Time charts showing what times of the year I tend to read more or less

Then there are static pictures, more detailed versions of the info you get from the tag browser

- Most popular tags, authors

- average rating of tags/authors

I could go on. Part of what has stopped me from doing this so far is that the list of such queries is endless and building something that is both powerful enough to do this generally and easy enough for casual use is daunting.

theducks · 01-13-2012, 01:18 PM

Many years ago I used a DOS program called Paradox (Relational Database). IMHO It's claim to was Query by example.
The query process presented the user with a form that represented the table(s) (yes there was an easy linking mechanism)

Why worry what the user wants stats on?
Just build the the query tool(s) and export as a CSV that could Open Office Calc or Excel.
Each table could have a Tab on the form (changes color/Bold when active) with fields and selection boxes/spin_wheels/criteria.
and a Run button

eat_pb · 03-13-2012, 06:58 PM

In the past, I've used a pretty good graphing library called QWT.

http://qwt.sourceforge.net/

The only issue I had with it was that it changed the cursor when you hovered over a graph, but I found a way of killing that. If you'd like to know a bit more I can dig through my old code to find out how I did it.

kiwidude · 03-14-2012, 07:13 AM

@eat_pb - welcome to MobileRead.

Thanks for the pointer to that library. As you may be able to tell from the lack of recent activity on this thread, implementing this plugin fell into the "too hard" basket for me which I suspected it would. There were not very many non-commercial Python based graphing libraries, and the ones I looked at had annoying long lists of dependencies (version X of A, version Y of B etc) which all just made it way more hassle than I wanted to deal with. I just wanted to download a zip file and have a go.

I don't recall coming across the qwt library though (my memory sucks) so at first glance it looks promising. The whole packaging drama would still have to be addressed (presumably platform specific C++ dlls) though Kovid did volunteer to assist with that.

Personally I think the demand for the plugin is there and it could add a bit of "glam" to the UI as well. However as I have recently created about 6-12 months of backlog for myself of other after hour development projects (non-calibre related) I don't see myself tackling this in the near future. If someone else is willing to carry the baton for it I would be delighted

DerManu · 04-25-2012, 10:29 PM

Hi,
If you're looking for a modern plotting/graph widget with, as you mentioned, no dependencies apart from Qt, you might be interested in my QCustomPlot component. It can be included directly as source file, so no need to juggle with libraries and linkage etc.:

http://www.WorksLikeClockwork.com/in...lotting-widget

It's by far not as large as QWT, but especially in your case, this can actually be an advantage. Oh yes, and I take the documentation and tutorials/examples very seriously.

kiwidude · 04-26-2012, 10:01 AM

@DerManu - welcome to MobileRead.

From my very quick glance at your post and the opening page that looks/sounds brilliant - exactly the sort of thing I was hoping to find. You may even have tempted me into thinking about implementing this plugin again - I shall take a look through your docs and have a play...

Edit: Ok I read further down the page - ummm, this isn't for Python is it?

DerManu · 04-27-2012, 09:01 AM

Yes, but the interface is all Qt, so isn't it possible to call it from python somehow? Maybe you do have to link it into a shared library in that case (which is probably what the other C++ plotting packages do...).
For python only, did you look at matplotlib?

kovidgoyal · 04-27-2012, 09:08 AM

You have to create a wrapper for the main class using SIP, to make it accessible from python. I had to do that for a couple of custom Qt components I wrote for calibre. It's not too hard if you have a bit of C++ experience.

kiwidude · 04-27-2012, 11:18 AM

It's been a long while since I looked but I'm sure I took a look at matplotlib, particularly since Kovid mentioned it. I can't remember why it didn't entice me - most likely it fell into the "you must also have version x of y and z which we don't include" type of packages but I could be wrong about that. Plugins are very much a labour of love and since this particular one already has the potential to be horribly time consuming in just decisions over UI/functionality let alone learning the graphing API I don't want to spend even more hours dealing with packaging issues.

Thanks also for the info Kovid, that's unfortunately not my wheelhouse. And I guess that even with a wrapper there comes the issues of dll loading/packaging from a plugin perspective, which in your case you avoid by being part of calibre itself...

01-13-2012, 05:07 AM	#1
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	Graphing libraries for Qt? I've got the beginnings of another plugin idea, which I am tentatively exploring the feasibility of. It is to do with producing statistical graphs/reports of books in your library. At its simplest it would be not dissimilar to what you get on Goodreads for instance allowing you to see books read by time periods (assuming you have a date read column), or books by date published, books by date added etc. In theory you could do books by genre tags or whatever etc as well, just really down to what people would find interesting. However when it comes to graphing using Qt, according to the FAQ they say you are basically on your own, and suggest including some third party libraries. link I don't think that me compiling up a Qt library and including in a plugin is likely to work very well, methinks there be dragons in there (compilation/cross platform etc) even if technically it did work. I've only bundled python libraries such as with Goodreads Sync previously, not C++ dlls. So the question is most likely for Kovid - do you perchance already include any Qt graphing libraries currently in your calibre distribution, and if not would you have any interest in adding to your build process to do so? Perhaps you may have some feature in mind for calibre down the line which might use it? Or do you have any alternative suggestions? If it all falls into the too hard basket I'll just forget the whole idea - I have close enough to what I need from Goodreads currently.

04-26-2012, 10:01 AM	#12
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@DerManu - welcome to MobileRead. From my very quick glance at your post and the opening page that looks/sounds brilliant - exactly the sort of thing I was hoping to find. You may even have tempted me into thinking about implementing this plugin again - I shall take a look through your docs and have a play... Edit: Ok I read further down the page - ummm, this isn't for Python is it? Last edited by kiwidude; 04-26-2012 at 10:05 AM.

04-27-2012, 09:01 AM	#13
DerManu Junior Member Posts: 2 Karma: 10 Join Date: Apr 2012 Device: none	Yes, but the interface is all Qt, so isn't it possible to call it from python somehow? Maybe you do have to link it into a shared library in that case (which is probably what the other C++ plotting packages do...). For python only, did you look at matplotlib? Last edited by DerManu; 04-27-2012 at 09:10 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
UK libraries	feenix1363	General Discussions	96	05-06-2012 07:26 AM
PRS-T1 Using two libraries on the T1	Avwriter	Sony Reader	1	10-12-2011 04:10 PM
Android Nice Scientific Graphing Calulator	obsessed2	enTourage Archive	4	05-14-2011 09:57 AM
Different libraries for different languages	philandjan	Library Management	7	02-13-2011 11:33 AM
Dual Libraries	latexsalesman	Calibre	4	09-03-2009 07:35 PM

01-13-2012, 07:06 AM	#2
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I dont distribute any graphing libraries with Qt. There are however python graphing libraries you could probably use to generate the graph as an image and then display it with Qt. Though I dont know if any of them are pure python, or if they include compiled code (if the latter, you will not be able to load the compiled parts directly from the zip file). Youd have to extract to file sytsem, load manually and then proceed.

01-13-2012, 07:19 AM	#3
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I've used matplotlib in the past, to generate 3D realtime plots of a quantum computer simulation for my thesis, so it is definitely powerful enough Whether it is usable from a zip file, I'm not so sure. I do know it has compiled code, but whether any of that is actually essential, I cannot recall. Try it and see.

01-13-2012, 07:53 AM	#4
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	Thanks Kovid, I will do some googling on the Python angle then to see what shakes out.

01-13-2012, 08:12 AM	#5
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	If you're serious about this plugin (which I think is a rather cool idea, something I always wanted to implement myself) and you cannot find a suitable pure python library, I'm willing to do the work to add some graphing library to the calibre distribution. The caveat being that in that case, I may want to end up adding your plugin to base calibre

01-13-2012, 10:15 AM	#6
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	I'll take a look over the weekend sometime and see what is out there, I hate getting in your way for stuff like this as know you have your own never-ending todo list. Appreciate the offer for sure, may well end up taking you up on that if it gestates beyond the current random idea stage. If you or anyone has any suggestions about features or how it would work etc please feel free to chip in, it might help motivate me to pull it together. I have given no thought as yet as to how it might work etc. I just figure given many calibre users spend hours collecting minutae of data about their books and reading, that giving them some additional pretty output might be of interest. Certainly the stats pages on goodreads showing me how many pages I have read each year, how many books each year etc I find interesting, particularly if you happen to set yourself some goals for a bit of fun. Clearly there will need to be some level of customisation allowing users to setup their own desired graphs, though where these will be rendered (modeless window) and what it all looks like is all up for grabs... ideas welcomed.

01-13-2012, 12:31 PM	#7
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	For me, I would like answers to questions along the time axis: - How has my reading rate changed (proxied by number of books added per month or some such) - How has the average rating of books I read changed - How have the most frequent 5 tags in the books I read changed? - Time charts showing what times of the year I tend to read more or less Then there are static pictures, more detailed versions of the info you get from the tag browser - Most popular tags, authors - average rating of tags/authors I could go on. Part of what has stopped me from doing this so far is that the list of such queries is endless and building something that is both powerful enough to do this generally and easy enough for casual use is daunting.

01-13-2012, 01:18 PM	#8
theducks Well trained by Cats Posts: 29,782 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Many years ago I used a DOS program called Paradox (Relational Database). IMHO It's claim to was Query by example. The query process presented the user with a form that represented the table(s) (yes there was an easy linking mechanism) Why worry what the user wants stats on? Just build the the query tool(s) and export as a CSV that could Open Office Calc or Excel. Each table could have a Tab on the form (changes color/Bold when active) with fields and selection boxes/spin_wheels/criteria. and a Run button

03-13-2012, 06:58 PM	#9
eat_pb Junior Member Posts: 1 Karma: 10 Join Date: Mar 2012 Device: Kindle 4	In the past, I've used a pretty good graphing library called QWT. http://qwt.sourceforge.net/ The only issue I had with it was that it changed the cursor when you hovered over a graph, but I found a way of killing that. If you'd like to know a bit more I can dig through my old code to find out how I did it.

03-14-2012, 07:13 AM	#10
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	@eat_pb - welcome to MobileRead. Thanks for the pointer to that library. As you may be able to tell from the lack of recent activity on this thread, implementing this plugin fell into the "too hard" basket for me which I suspected it would. There were not very many non-commercial Python based graphing libraries, and the ones I looked at had annoying long lists of dependencies (version X of A, version Y of B etc) which all just made it way more hassle than I wanted to deal with. I just wanted to download a zip file and have a go. I don't recall coming across the qwt library though (my memory sucks) so at first glance it looks promising. The whole packaging drama would still have to be addressed (presumably platform specific C++ dlls) though Kovid did volunteer to assist with that. Personally I think the demand for the plugin is there and it could add a bit of "glam" to the UI as well. However as I have recently created about 6-12 months of backlog for myself of other after hour development projects (non-calibre related) I don't see myself tackling this in the near future. If someone else is willing to carry the baton for it I would be delighted

04-25-2012, 10:29 PM	#11
DerManu Junior Member Posts: 2 Karma: 10 Join Date: Apr 2012 Device: none	Hi, If you're looking for a modern plotting/graph widget with, as you mentioned, no dependencies apart from Qt, you might be interested in my QCustomPlot component. It can be included directly as source file, so no need to juggle with libraries and linkage etc.: http://www.WorksLikeClockwork.com/in...lotting-widget It's by far not as large as QWT, but especially in your case, this can actually be an advantage. Oh yes, and I take the documentation and tutorials/examples very seriously.

Advert

Advert

04-27-2012, 09:08 AM	#14
kovidgoyal creator of calibre Posts: 43,843 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You have to create a wrapper for the main class using SIP, to make it accessible from python. I had to do that for a couple of custom Qt components I wrote for calibre. It's not too hard if you have a bit of C++ experience.

04-27-2012, 11:18 AM	#15
kiwidude Calibre Plugins Developer Posts: 4,636 Karma: 2162064 Join Date: Oct 2010 Location: Australia Device: Kindle Oasis	It's been a long while since I looked but I'm sure I took a look at matplotlib, particularly since Kovid mentioned it. I can't remember why it didn't entice me - most likely it fell into the "you must also have version x of y and z which we don't include" type of packages but I could be wrong about that. Plugins are very much a labour of love and since this particular one already has the potential to be horribly time consuming in just decisions over UI/functionality let alone learning the graphing API I don't want to spend even more hours dealing with packaging issues. Thanks also for the info Kovid, that's unfortunately not my wheelhouse. And I guess that even with a wrapper there comes the issues of dll loading/packaging from a plugin perspective, which in your case you avoid by being part of calibre itself...