How complicated do you want your interface with the site to be?
If you're happy with static HTML pages that you update manually, then all you need is a place that will host them for for a minor cost, and your own domain name, I suppose.
Personally, I'd recommend 1&1 for that: they have an
Instant Domain package that's included in all their domain name purchases and gives you 2GB of email storage, along with 100 MB of webspace for as long as your domain registration stays with them. I use that myself for my email. (Disclaimer: they technically have a referral program for their customers to bring them new customers, but I've no idea how it works and as far as I can tell, that's a clean link I've put up, or you can try browsing directly from the front of their site:
1&1 Internet Services. If anyone does want to try the referral thing if they're moved to choose 1&1, please PM me.)
If you want to run your own WordPress or other content management system so you don't have to create and update your pages manually, you'll need a host that will allow you to use CGI/PHP/other scripting whatnot, and will also allow you to do it for a low price.
For this, I personally recommend
NearlyFreeSpeech.net, which has an excellent pay-as-you-go plan which charges you for storage, server processes, and bandwidth used as you use it (you do have to deposit in advance) rather than at a fixed monthly rate. If you're not anticipating high traffic to your site, then this is very cheap indeed (and if you are, they have some kind of tie-in redirect option which let you link to external money-saving things like Amazon Web Services for your file-hosting, IIRC), and you can probably run your site for a year on what another place would charge you for a single month.
Again, I use them myself, and they're quite good and have very reasonable policies regarding usage (including as regards to free speech, responses to DMCA complaints, and hosting adult content, for those who might need a service provider who can do such) and have a comprehensive FAQ and helpful advice/help Forum where the members are really good at answering people's questions (both only available to view once you're signed up though, or I'd link them: here's the
admin blog so you can see a bit of what they're like).
If you're not quite up to setting a WordPress install up yourself, you can still use their free version and do all the stuff you mention above, as in addition to the regularly-updated blog function, WP lets you host downloadable files and set up static non-blog pages which you can put in the frontpage sidebar as links to permanent resources (as well as actual links).
It would probably be a good way to practice and get used to using it, and one of the advantages of WP.com vs Blogspot is that it's relatively easy to back up and move one's WP.com site over to one's own hosted install, once you're ready to do that.
Hope this helps.