Disclaimer: I am not a WordPress expert. But as I learn the various mechanical techniques in working with WordPress.org, I am posting them here on Active Rain. Mainly so that I can find them again when I forget the steps involved.
Please don't go messing with your own files unless you feel comfortable with what you are doing.
Okay-dokey. You installed WordPress.org on your web host using Fantastico's one-click installer (described here: http://activerain.com/blogsview/323051/WordPress-org-One-Click ).
You signed on to your brand new WordPress.org blog, and you found this nagging little message banner across the top of your screen: "A new version of WordPress is available! Please update now."

First, there are three prerequisites before you handle this:
1. You need to have an UNZIP utility, and know how to use it.
WinZip is a popular unzip utility: http://www.winzip.com/index.htm , Cam UnZip is free: http://www.camunzip.com/ . You can Google for more.
2. You need to have an FTP utility, and know how to use it.
FTP stands for "File Transfer Protocol". You can think of it as just another method of uploading. CuteFTP http://www.cuteftp.com/products/ftp_clients.aspx and WS_FTP http://www.wsftp.com/products/ws_ftp_home/ are both popular FTP utilities.
3. You need to know the FTP address of your blog. It's probably ftp.myblog.com; and you need to know the login and password for the host account.
If you've got those three items handled, you're ready to proceed. If not, hang on, and I'll build some posts covering using these utilities soon. Or start Googling. :-)
If you're ready to proceed, go ahead and click on the words "Please update now." That will take you to the download WordPress page. Click the button to download the .zip file.

One important tip: When downloading anything, pay attention to where you save it. Some people like to save to their Desktop, some save to My Documents; I keep a folder titled "Download".
When the WordPress zip file finishes downloading, use your unzip utility to extract all the files, and yes, there are a lot of files. Second important tip: Pay attention to where the unzip utility places the extracted files. You can't install them if you can't find them.

Now, open up your FTP utility. Connect to the host account. Most FTP utilities will show the files on the host in the right hand panel, and the files on your computer in the left hand panel. Drill down to your WordPress files on the host, and to your newly extracted updated files on your computer.

Here's where WordPress experts will know more than me. I simply selected all of the files in the root WordPress folder on my computer and uploaded them to the host, overwriting the existing WorPress file.
Then I opened the wp-admin file on both the host and my computer and uploaded all files, overwriting the existing files. Same for wp-includes. I didn't do this with wp-content or any of the wp-admin of wp-includes sub-folders.
I'm guessing experts will tell me there were actually only a few files I needed to upload to do the update. But hey, I'm new at this!
Thank Goodness, no more "Please update" banner.

Jay McGillicuddy added these comments -- and this is so useful -- I've copied into them post itself:
There is a plugin that will automatically update for you. Press one button. Worked for me.
http://www.zirona.com/software/wordpress-instant-upgrade
I would suggest that you back up your blog first before upgrading. Here is an automatic back up plugin.
http://www.ilfilosofo.com/blog/wp-db-backup/
Thanks, Jay!!
Great information such detail abd effort...thanks for your time and thoughts...the updates too.