I love Haven Brothers, though I haven't been in a few years. They were of course a staple of misspent youth in Rhode Island. I also used to hit them earlier in the evenings when I would walk home - during the colder months they are out by City Hall by 5 or so, making it an easy stop before the evening scene developed. But there's still no place better to have a burger with drunks and bikers.
Seafood in Providence? You can get it, but I'd go along with wanderingjew and recommend looking out of town. I've never been to Duffy's, though I've been by it plenty of times and have heard only good things about their food. Other places I'd recommend, apologies for repeating some favorites:
Horton's Seafood, Broadway, East Providence - just seconds away from the East Side of Providence, across the Seekonk River, is this clam shack in an urban setting. Everything you'd expect from shore dining - fried and baked fish, clams, scallops, etc., clamcakes, stuffies and so on. Very good, reasonably priced. High quality root beer bottled for them, average french fries. Closed Sundays and Mondays, I think. Check them out at http://www.hortonsseafood.com/
Ye Olde English Fish and Chips, Market Sq., Woonsocket: The best fish and chips I have ever had or hope to have. Try the fish cake. Closed Sundays and Mondays, short to nonexistent evening hours most other days.
Skip's Dock, Old Succotash Rd., East Matunuck (South Kingstown) - It doesn't get any fresher. This is right on the docks, just across the water from Galillee. The boat comes up, unloads and the fish goes on sale. I've actually requested something and been asked to wait, until they could cut it, as they were just then unloading it. Skip's used to only operate as a fish market, but starting this year they are serving meals at small tables on the dock. Eat outside by the water, watching the boats, on great seafood; a bit more expensive, but worth it.
Main St. Seafood, Wakefield (South Kingstown) - another fish market/restaurant. In town (such as it is), pleasant indoor seating, great fish. The best fish chowder I've ever experienced, rich and full of seafood. I'm not sure about the hours.
Aunt Carrie's (reviewed on this site) is a fun place to go, with decent clam cakes, great atmosphere and a wonderful dining room, but the food is pricey and not really exceptional (though they have their moments.) Get clamcakes or something at the takeout and go down the street to Pt. Judith Lighthouse to snack. And there are tons of other clam shacks and so on to try, not to mention the plethora of Portuguese restaurants with good seafood once you head to the East Bay...but that's enough for now.