Need advice for creating a slightly odd type of plugin

ldolse · 05-19-2011, 10:40 AM

I've got some friends doing archaeoastronomy on a research grant. They currently get datapoints in an extremely painstaking way from a NASA web application. When they described what they were doing I thought I could help them out with a script. While Calibre might not seem like the right place to create something like this, it made sense for a couple reasons:

Kovid's done a lot of heavy lifting around a very functional mechanize browser and other bundled functions
My friends aren't so techy I'd want to make them do CLI stuff all the time that a simple standalone script would require, and Calibre provides the heavy lifting here too with it's plugin UI.
Most importantly, Calibre can automatically convert the results to a variety of formats compatible with ipad, iphone, etc, a concept which my friends are big fans of

My questions are really how to go about making this work from a fine tuning perspective. I've already created a script I can execute from the CLI using calibre-debug that does all the heavy lifting. The user needs to configure a date in the past using UTC time for the celestial body they want to track, and then ideally an offset for the setting time. They also need to set the longitude/latitude coordinates. I plan to allow multiple events to be queried in one go.

I'm running into a couple issues and places where some advice would help.

Issue - When I tried using Calibre's parse_date function it converted UTC time to local time. NASA requires UTC. How do I keep Calibre using UTC for all date functions? Right now I'm just configuring start/end times by hand, but I want to start calculating offsets, which will require proper date handling.

Bonus points for this question - the dates my friends are using may go back far enough that they need to specify AD vs. BC - is that something Python's date functions would be up for handling? For now I'm just going to tell them to stick with dates past a certain year as they don't require special handling, but I may want to add support for this later. The NASA system also requires a specific format for BC dates, so that may be another hurdle.

Lastly I need advice on how to handle the actual plugin. My thinking is a file-type plugin. At first I thought I might have the user input the date/coordinates directly into a text file with a special extension, but now I'm thinking that would be more reliably handled in the plugin configuration dialog. That means to trigger the plugin one would just add a blank text file with a specific extension to Calibre and the rest would be done based on the config - does this make sense?

Once the input plugin is triggered I think I'll want to convert the results to a traditional format - probably zipped html - I believe an input format plugin can actually store a different format in Calibre's db/filesystem - I'm pretty sure the Topaz input plugin does this today.

Any thoughts on alternative approaches would be appreciated. Note I'm mostly out in the field with my friends at the moment, so any failures to respond reflect my lack of Internet access vs. lack of appreciation.

user_none · 05-19-2011, 12:05 PM

Why not make a GUI plugin? You would be able to have it open a dialog for them to put in date info for what they want to download. The click start and it runs.

I don't have access to my computer to give you specifics but... You can then have the GUI plugin use calibre's threaded job system to handle the download and then have it add the file to the library. Look at email.py and store_download.py in the gui2 folder for examples.

kovidgoyal · 05-19-2011, 12:07 PM

Date functions: Use the as_utc parameter to have them return UTC dates

I don't know for sure, but I highly doubt that Python handles BC dates very well. The default strftime function that comes with python doesn't even handle dates before 1900 (this is a limitation inherited from strftime in C libraries).

Why not use a interface action plugin. That will add an icon to the calibre toolbar. Your friends click on it, a dialog pops up they enter the data and the plugin does the rest.

Slightly more complex to create this kind of plugin, though.

ldolse · 05-25-2011, 09:18 AM

Thanks for the advice - I guess the reason I didn't think of an Interface Action plugin was that I didn't realize it could create a brand new book record. I'll go ahead and give that a shot using the examples user_none mentioned.

You mention the default strftime function doesn't handle dates before 1900 - can I infer that Calibre's no longer using that default? It seemed to handle dates before 1900 just fine when I tested it aside from the offset issue, which I'll use as_utc for. I'm back at my computer now, so have other code to use for a reference as well.

kovidgoyal · 05-25-2011, 11:31 AM

Yeah calibre has its own strftime function.

05-19-2011, 10:40 AM	#1
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Need advice for creating a slightly odd type of plugin I've got some friends doing archaeoastronomy on a research grant. They currently get datapoints in an extremely painstaking way from a NASA web application. When they described what they were doing I thought I could help them out with a script. While Calibre might not seem like the right place to create something like this, it made sense for a couple reasons: Kovid's done a lot of heavy lifting around a very functional mechanize browser and other bundled functions My friends aren't so techy I'd want to make them do CLI stuff all the time that a simple standalone script would require, and Calibre provides the heavy lifting here too with it's plugin UI. Most importantly, Calibre can automatically convert the results to a variety of formats compatible with ipad, iphone, etc, a concept which my friends are big fans of My questions are really how to go about making this work from a fine tuning perspective. I've already created a script I can execute from the CLI using calibre-debug that does all the heavy lifting. The user needs to configure a date in the past using UTC time for the celestial body they want to track, and then ideally an offset for the setting time. They also need to set the longitude/latitude coordinates. I plan to allow multiple events to be queried in one go. I'm running into a couple issues and places where some advice would help. Issue - When I tried using Calibre's parse_date function it converted UTC time to local time. NASA requires UTC. How do I keep Calibre using UTC for all date functions? Right now I'm just configuring start/end times by hand, but I want to start calculating offsets, which will require proper date handling. Bonus points for this question - the dates my friends are using may go back far enough that they need to specify AD vs. BC - is that something Python's date functions would be up for handling? For now I'm just going to tell them to stick with dates past a certain year as they don't require special handling, but I may want to add support for this later. The NASA system also requires a specific format for BC dates, so that may be another hurdle. Lastly I need advice on how to handle the actual plugin. My thinking is a file-type plugin. At first I thought I might have the user input the date/coordinates directly into a text file with a special extension, but now I'm thinking that would be more reliably handled in the plugin configuration dialog. That means to trigger the plugin one would just add a blank text file with a specific extension to Calibre and the rest would be done based on the config - does this make sense? Once the input plugin is triggered I think I'll want to convert the results to a traditional format - probably zipped html - I believe an input format plugin can actually store a different format in Calibre's db/filesystem - I'm pretty sure the Topaz input plugin does this today. Any thoughts on alternative approaches would be appreciated. Note I'm mostly out in the field with my friends at the moment, so any failures to respond reflect my lack of Internet access vs. lack of appreciation. Last edited by ldolse; 05-19-2011 at 11:09 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Trouble creating a plugin.	A0tmsk	Plugins	2	04-26-2011 12:19 AM
New Plugin Type Idea: Library Plugin	cgranade	Plugins	3	09-15-2010 12:11 PM
PDB to PMLZ File type plugin	pdurrant	Plugins	18	02-20-2010 02:19 PM
Display File Type and Sort by Type	pragmortal	Calibre	7	09-21-2009 08:07 PM
Advice on creating a PDF for eReader	charlieh	Introduce Yourself	3	08-14-2009 10:40 AM

05-19-2011, 12:05 PM	#2
user_none Sigil & calibre developer Posts: 2,488 Karma: 1063785 Join Date: Jan 2009 Location: Florida, USA Device: Nook STR	Why not make a GUI plugin? You would be able to have it open a dialog for them to put in date info for what they want to download. The click start and it runs. I don't have access to my computer to give you specifics but... You can then have the GUI plugin use calibre's threaded job system to handle the download and then have it add the file to the library. Look at email.py and store_download.py in the gui2 folder for examples.

05-19-2011, 12:07 PM	#3
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Date functions: Use the as_utc parameter to have them return UTC dates I don't know for sure, but I highly doubt that Python handles BC dates very well. The default strftime function that comes with python doesn't even handle dates before 1900 (this is a limitation inherited from strftime in C libraries). Why not use a interface action plugin. That will add an icon to the calibre toolbar. Your friends click on it, a dialog pops up they enter the data and the plugin does the rest. Slightly more complex to create this kind of plugin, though.

05-25-2011, 09:18 AM	#4
ldolse Wizard Posts: 1,337 Karma: 123455 Join Date: Apr 2009 Location: Malaysia Device: PRS-650, iPhone	Thanks for the advice - I guess the reason I didn't think of an Interface Action plugin was that I didn't realize it could create a brand new book record. I'll go ahead and give that a shot using the examples user_none mentioned. You mention the default strftime function doesn't handle dates before 1900 - can I infer that Calibre's no longer using that default? It seemed to handle dates before 1900 just fine when I tested it aside from the offset issue, which I'll use as_utc for. I'm back at my computer now, so have other code to use for a reference as well.

05-25-2011, 11:31 AM	#5
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Yeah calibre has its own strftime function.

Advert

Advert