I'm a California resident, but I have a New York Public Library Card. All I did was print out the application form on their website, fill it out and send it to them with a copy of my driver's license and a check for $100. They then sent me a card in the mail. The card is good for a year.
The check out procedure works just as everyone has described. It works great!
Natalie--not sure how it'd work for you--being that you're from Australia, but I seem to remember someone posting a while back that it was possible to get a NYPL card, even if you aren't from the US. Not sure about this though--you might want to send them an e-mail to see what they have to say. Before I joined I e-mailed them with a few questions and they responded back quickly.
The only thing I wish is that the NYPL offered ePub books--the only books they offer that work with the Sony are PDFs. They do have a large collection of these. I'm hopeful that they will begin offering ePub books shortly.
dordale