Re:New York city?
You can find cheap hotels in NJ close to various rail lines (commuter rail and light rail). Obviously, the further out, the lower the price. Also, up the Hudson Line in NY State. I've stayed in downtown Newark (which is a transitioning downtown) and could safely walk form hotel to Newark's Penn Station, and took rail to Manhattan. Easy, safe and convenient.
Also, most Newark airport hotels have free shuttles to train and airport. And you can always find deals at airport hotels.
The only problem with staying in Brooklyn or Queens, is paying all the toll bridge fees. But, there are cheap, name brand hotels in Queens that are near subway stations. I'm not, however, familiar enough with the neighborhoods in Queens to recommend a safe spot.