Having lived briefly here in HK, I get a lot of questions about where a good place to stay might be, and other requests for travel advice.
For many travelers, Hong Kong (HK) is frequently always a stop on major international flights between Asia and Europe/Middle East or the Americas.
HK Island, for the most part, is not a cheap destination. In fact, in many ways it is a super luxury market. The outskirts away from the Island tend to be cheaper but you see less and it can take time to get to where you are going. If you are aiming for Shenzhen or the mainland, then that's great. If you're aiming for the financial center, then probably not.
HK is a place where the westerner can feel comfortable in Asia, where most things are or have a translation readily available in English. The Mass transit system is world class.
Where to stay
Here is Hostel in two locations - Hong Kong Side and Kowloon (Mong Kok) side - Yesinn - http://www.yesinspace.com/ I have not stayed here but heard it was better than others.
Hotel: 3 stars - Hong Kong Island side - http://www.mingleplace.com/ near Wellington st. This is the Midlevels region and highly recommended if you want to experience the Hong Kong ' Atmosphere' on the Island. Its also near major tourist sites.
In my head at least, Kowloon has always been the more industrial side, Hong Kong island side the manhattan side. Things have changed a lot in the past 25 years or so, with the modernization of Tsim Tsa Tsui (TST) in Kowloon but the gritty manufacturing side is still there on the Kowloon peninsula.
Hong Kong tends to have a very high people density in very close quarters (claustrophobic close), so in general I would avoid staying at a very cheap place. You will see people wearing face masks to prevent transmission of respiratory diseases in public areas. I personally can tolerate staying in hostels all over Africa, but hesitate inHong Kong because of previous SARS and respiratory disease outbreaks. It is better off for your health and safety to stay at least in a 3 star or above hotel.
Also a word of caution: On the Kowloon side avoid any Hostel on Nathan Road in TST. This whole area is a tourist trap. It is worth visiting TST if you have never been before, but please don't stay here if you are looking for a hostel. The Chungking Mansions on Nathan Rd are where most backpackers stay. I have been harrassed at the front of the building even though I'm Chinese and wasn't staying there. The local touts press visitors for cash and it is a maze of shops with slow as molasses odd/even alternating floor elevators. Great if you have the energy to deal with it, otherwise visit at your own peril.
Do not stay in Chung King Mansions or any place similar on Nathans Road at any cost. It is not worth your trouble and will wreck your HK experience.
Transit: the MTR subway system is really good. If you can, do get an Octopus card, instead of the single ride tickets because its very useful. The Octopus card is good for Starbucks, 7-11, Mcdonalds, Watson's, and a lot of other convenience stores. Helps you avoid carrying around spare change - the coins in HK are super heavy.
If you are flying out of the HK airport you can check your baggage in at the Central or Kowloon station instead of hauling it all the way to the airport. Checking in here with your Octopus card automatically gets you ticket to the airport ($90 HKD) Super awesome service.
Wifi: Free wifi can be hard to come by in Hong Kong for the short term visitor. Your best bet is Y5 internet, which gives you free internet for 20 minutes each day. you can get access at McDonalds or Starbucks. HSBC also has free wifi if you enter their building and just hang out.
