python 2.X to 3.X migration?

KevinH · 11-12-2010, 12:59 PM

Hi Kovid,

I was reading the python developer's mailing list about how python 2.7 would be supported for up to 5 years and that the entire 2.x series was now in maintenance mode. I was wondering what your plan was for calibre in the long run? It seems that very few of the libraries have made the switch yet. I have played around with 2to3 a bit but it does not handle the string vs bytes issue as far as I can tell. I guess my only concern is how soon, if at all, would calibre plugin developers have to worry about shifting their plugin code to 3.x?

Thanks,

KevinH

kovidgoyal · 11-12-2010, 01:11 PM

I have absolutely no plans to migrate calibre to python 3. I see no real benefits at this point and migrating the half a million lines of python and C extension code in calibre is not to be lightly undertaken.

And that is not to mention all the external python libraries calibre depends on, most of which show no signs of wanting to migrate to python 3.

KevinH · 11-12-2010, 01:15 PM

Hi,

Thanks! That was the answer I was hoping for!

Take care,

Kevin

nyrath · 11-12-2010, 01:23 PM

Yes, I program in Python at my day job. Our shop is in no hurry to migrate to Python 3.0, mainly due to the issues Kovid stated. Especially the external libraries.

JimboDimbo · 02-13-2011, 10:08 AM

Absolutely agree that it's way too early to think about changing to P3.

"I see no real benefits at this point and migrating the half a million lines of python and C extension code in calibre is not to be lightly undertaken. "

But, maybe it's worth considering a plan to migrate over the next 2 to 3 years?
Eventually 2.x and support will peter out.
Ebooks are exploding (in popularity) now, so Calibre is becoming a critical resource. I can feel the commercial pressures building.... Good luck with that...
Although in IT, I am newish to Python and am enjoying learning. I fancy doing my own news recipes.
Calibre is unique in offering a development environment embedded together with a working program.

If you want Calibre to continue for several years, then (although daunting) there should be some longer term strategy to move to Python 3.
http://docs.python.org/dev/howto/pyporting.html talks about writing Python 2/3 source-compatible code (I'm too new to see if this could work). But of course there is the difficult external libraries issue to consider. Some may never make it across (e.g. BeautifulSoup)?

However, if the process is begun at least each of the issues can be taken one at a time. Several Calibre contributors can assist in the migration. With a new Sub-Group and Forum over the next few years. It can be Python 2to3 in 2to3 years (you see what I did there.....

).

Anyhow, enough already. Just a thought about considering a longer term strategy and utilising/exploiting/encouraging/growing/improving the loyal Calibre following/contributors.

Cheers,
Peter.

kovidgoyal · 02-13-2011, 04:41 PM

Most new calibre code is written in a way that is easy to port to 3 anyway.

For example, calibre has its own print function, unicode conversion functions and so on, which means that all you have to do is port those.

The problem is the half a million lines of legacy code. I for one do not have the time/motivation/desire to even contemplate porting that.

user_none · 02-13-2011, 07:36 PM

I foresee migration happening in 5 - 7 years because it will be absolutely necessary at that point. I also foresee that migration will also coincide with a massive refactoring / rewriting effort. There will most likely also be work toward integrating dependencies that are EOL themselves (though abandonment of some sort) into calibre's code base and reducing the number of dependencies.

kovidgoyal · 02-13-2011, 07:56 PM

I sometimes think it will be easier to just maintain python 2.x myself. I already patch and hand compile python 2.x for the calibre binaries. Cleaning up the language is all very nice, but I have yet to see an actual benefit to using python 3, given that I long ago trained myself to workaround the quirks in python 2 that python 3 addresses.

About the only thing that would convince me to start a migration effort is if PyQt dropped support for python 2.x. Or someone wrote a really large and really useful library for python 3 only, that I for some reason felt unable to port. I don't see either of those happening anytime in the forseeable future.

11-12-2010, 12:59 PM	#1
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	python 2.X to 3.X migration? Hi Kovid, I was reading the python developer's mailing list about how python 2.7 would be supported for up to 5 years and that the entire 2.x series was now in maintenance mode. I was wondering what your plan was for calibre in the long run? It seems that very few of the libraries have made the switch yet. I have played around with 2to3 a bit but it does not handle the string vs bytes issue as far as I can tell. I guess my only concern is how soon, if at all, would calibre plugin developers have to worry about shifting their plugin code to 3.x? Thanks, KevinH Last edited by KevinH; 11-12-2010 at 01:00 PM. Reason: fix typo

02-13-2011, 10:08 AM	#5
JimboDimbo Junior Member Posts: 1 Karma: 10 Join Date: Jan 2011 Location: South Wales - UK Device: Kindle 3 3G	Longer term strategy? Absolutely agree that it's way too early to think about changing to P3. "I see no real benefits at this point and migrating the half a million lines of python and C extension code in calibre is not to be lightly undertaken. " But, maybe it's worth considering a plan to migrate over the next 2 to 3 years? Eventually 2.x and support will peter out. Ebooks are exploding (in popularity) now, so Calibre is becoming a critical resource. I can feel the commercial pressures building.... Good luck with that... Although in IT, I am newish to Python and am enjoying learning. I fancy doing my own news recipes. Calibre is unique in offering a development environment embedded together with a working program. If you want Calibre to continue for several years, then (although daunting) there should be some longer term strategy to move to Python 3. http://docs.python.org/dev/howto/pyporting.html talks about writing Python 2/3 source-compatible code (I'm too new to see if this could work). But of course there is the difficult external libraries issue to consider. Some may never make it across (e.g. BeautifulSoup)? However, if the process is begun at least each of the issues can be taken one at a time. Several Calibre contributors can assist in the migration. With a new Sub-Group and Forum over the next few years. It can be Python 2to3 in 2to3 years (you see what I did there..... ). Anyhow, enough already. Just a thought about considering a longer term strategy and utilising/exploiting/encouraging/growing/improving the loyal Calibre following/contributors. Cheers, Peter.

02-13-2011, 07:56 PM	#8
kovidgoyal creator of calibre Posts: 43,857 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I sometimes think it will be easier to just maintain python 2.x myself. I already patch and hand compile python 2.x for the calibre binaries. Cleaning up the language is all very nice, but I have yet to see an actual benefit to using python 3, given that I long ago trained myself to workaround the quirks in python 2 that python 3 addresses. About the only thing that would convince me to start a migration effort is if PyQt dropped support for python 2.x. Or someone wrote a really large and really useful library for python 3 only, that I for some reason felt unable to port. I don't see either of those happening anytime in the forseeable future. Last edited by kovidgoyal; 02-13-2011 at 07:59 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre library migration	kiat	Calibre	4	10-22-2010 03:47 AM
Sony Bookstore - migration to ePub	amjbrown	Sony Reader	2	10-05-2009 12:43 PM
Other Non-Fiction D’Alviella, Count Goblet: The Migration of Symbols, v.1, 22 Nov 2008	vivaldirules	Kindle Books	0	11-22-2008 10:47 PM
Other Non-Fiction D’Alviella, Count Goblet: The Migration of Symbols, v.1, 22 Nov 2008	vivaldirules	BBeB/LRF Books	0	11-22-2008 10:45 PM
MobileRead migration complete / Small changes	Alexander Turcic	Announcements	88	07-09-2007 05:32 PM

11-12-2010, 01:11 PM	#2
kovidgoyal creator of calibre Posts: 43,857 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I have absolutely no plans to migrate calibre to python 3. I see no real benefits at this point and migrating the half a million lines of python and C extension code in calibre is not to be lightly undertaken. And that is not to mention all the external python libraries calibre depends on, most of which show no signs of wanting to migrate to python 3.

11-12-2010, 01:15 PM	#3
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Hi, Thanks! That was the answer I was hoping for! Take care, Kevin

11-12-2010, 01:23 PM	#4
nyrath Addict Posts: 281 Karma: 52007 Join Date: Jun 2010 Device: nook	Yes, I program in Python at my day job. Our shop is in no hurry to migrate to Python 3.0, mainly due to the issues Kovid stated. Especially the external libraries.

02-13-2011, 04:41 PM	#6
kovidgoyal creator of calibre Posts: 43,857 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Most new calibre code is written in a way that is easy to port to 3 anyway. For example, calibre has its own print function, unicode conversion functions and so on, which means that all you have to do is port those. The problem is the half a million lines of legacy code. I for one do not have the time/motivation/desire to even contemplate porting that.

02-13-2011, 07:36 PM	#7
user_none Sigil & calibre developer Posts: 2,488 Karma: 1063785 Join Date: Jan 2009 Location: Florida, USA Device: Nook STR	I foresee migration happening in 5 - 7 years because it will be absolutely necessary at that point. I also foresee that migration will also coincide with a massive refactoring / rewriting effort. There will most likely also be work toward integrating dependencies that are EOL themselves (though abandonment of some sort) into calibre's code base and reducing the number of dependencies.

Advert

Advert