I would like to make a folder on the desktop and put stuff like text files and PDF files in it for easy access. I've been able to create the folder on the desktop, but I can't figure out where it is stored to put stuff into it. Any suggestions?

When you create a folder at the desktop, you can go to your all programs list and press and hold one icon, which will bring you to the Desktop again in the mode for placing the icon of that program to the Desktop. Then you can drag that program icon over folder icon and drop it there. Now you'll have that program in a quick-to-reach folder.
Tapping and holding over opened folder's name, will bring up the popup where you can rename the folder.

That's the furthest I got.

I guess the desktop folder structure is buried somewhere where it's required to have a rooted phone to be able to peek

If you install ASTRO file manager you will be able to put any file shortcut to the destop (or folder) by going to shortcuts and selecting astro.

Cheers

Thanks joolui, that's it!

When you locate the folder/file you want to place as a desktop shortcut, a long press, while still in ASTRO, on that one will give you a pop-up rename window.