Library is what is in your iTunes library on your machine.
Store is a connection to the iTunes store, and contains links to all of the available purchasable content. If you buy something from the Store, it will briefly appear in Downloads, then be available in your Library.
With apps, you can redownload them for free as often as you like. For music and videos, you cannot, you need to keep your own backup copies.