Not sure if you're on a Mac or a PC. If you're on a Mac you might want to check out
this article. Apparently there is a bug in iTunes 9.1.1 (build 11) that was fixed in 9.1.1 (build 12). It involves constant re-syncs of music files. Might also contribute to lengthy backup times.