Since I see from your sig that you use Blogger for your blog
this link would seem to have the relevant info on how to set up feeds.
As for what it is - well it's a way of publishing updates to people who are interested. In the old days of "web-logs" I used to regularly check a number of sites, cycling through my favourite bookmarks. If I forgot to visit a particular site for a few days then maybe there would be lots of new updates there. An RSS feed is a way of providing a summary and a link when new material is available. Various programs which understand RSS feeds can be used to follow them. So instead of regularly checking my bookmarks - which I may forget to do - I regularly check my RSS reader. Better still it can regularly check in the background and when I open it I can see that say, "Gregg's blog" has a new post and go and read it. So it feels more like it's updates being delivered to me.
Now I say RSS reader but these days I'm not sure how many people still use them. Most browsers will handle RSS feeds, there are a number of browser add-ons, email programs like Outlook and Thunderbird and so on. Personally I use Google Reader - which is a web-based RSS service.
Oh and RSS is generally how podcasts are delivered. The link of "new stuff" is a download for an mp3 (or other file) rather than a webpage but the principle's the same.
If you have an active blog you should definitely enable feeds as it's how many people will "follow" your updates.
HTH