I know Case Shiller is spreading real estate doom and gloom across the state lines, but it’s summer and most of us have vacation on the mind. Here in Dallas, it is about 110 degrees in the shade, and anyone who doesn’t understand why I started a second home blog IN THIS CITY is suffering from heat stroke. A second home all but comes with a Texas drivers license, or at least, it should. The good news is that this really is the best time ever to buy that second vacation home, particularly in the middle market price range. While home prices are still falling in most regions, (I’m looking at you, Santa Fe) the luxury segment is picking up steam. Or, at least, buyers. If you measure traffic by calls and lookers (not hookers), brokers are reporting more inquiries than they have had in years. In bellwethers Vail, Hilton Head and Palm Beach, foot traffic has jumped by at least 30% this year, according to local real-estate agents.

Should you buy? Yes, IF you are not the one of every five Americans who is unemployed, IF you can afford it, and IF you plan to keep the property for a long time. In fact, the experts themselves are buying.

As I blogged from Inman Real Estate Connect in San Francisco on Friday,  Doug Duncan, VP and Chief Economist over at Fannie Mae spoke on an Economic Outlook panel and told us he’s just bought a home in Florida. Vacation markets that once were priced out of sight are coming back to a reasonable range. Take Hilton Head Island, S.C. The Wall Street Journal reports a three-bedroom home (between the Atlantic Ocean and Calibogue Sound) sold in April for $750,000, after having sold for $1.2 million in June 2006. In Vail, Colo., a three-bedroom home that fetched $3.3 million in 2008 sold in February for $2.5 million. I’m more in the under a million bracket, and I find these prices enticing.

Second home prices are definitely down. The National Association of Realtors says the median second-home price was $150,000 in 2010, down 11% from 2009, but down a whopping 25% from 2006 (peak of the steroidal market). What’s interesting is that while we all know the second home market is hurting more than even the primary home market, that’s still only three percentage points more of pain: the overall primary home market is down 22%. In fact Fannie Mae’s Duncan — the one with the new vacation home — says homes in the $5 million-plus range have held up the best.

 ”At the top of the market, particularly luxury homes, prices have proven very elastic, and have sprung upward quickly,” he says.

Here’s proof in the pudding: Palm Beach Island, Fla., sales were up 50% in the year ending June 30. Hamptons transactions, on New York’s Long Island, jumped 59% in the second quarter from a year earlier despite worries about state income taxes. And Aspen, Colo., as I have told you, is almost on fire, with sales ending May 31 up 10%.

Location, Location, Location

I’m not saying it’s Miller Time. But if you are going to buy, here’s how to secure your value should real estate dip majorly: only buy properties in prime locations—on the water (Beacon Hill, Lake Cypress Springs) or near a ski slope, ski in and out preferred. There’s got to be something fun to do there besides sit. Homes in less desirable spots tend to languish on the market. It is hard as you-know-what to obtain second home financing, as most banks are wary of second-home mortgages, particularly “jumbo” loans above federally guaranteed limits. Ten percent of banks raised their standards on such loans last year, according to the Federal Reserve. And who knows what our friends in D.C. will do to mortgage interest on second homes — my guess is they’ll say adios to “gig the rich”.

So how do you gauge which vacation home areas will hold up and not tank during a recession? Check geography — natural amentities, water access, ski in or out, and activities. Look at places that are not overbuilt –Hilton Head, S.C. benefitted from tough restrictions on building, which kept inventories manageable and prices there have risen by 4% during the past year. Foreclosures are a great buy, but if an area is loaded with foreclosures, prices will continue to fall. Example: Gulf Shores. Of course, get a good enough buy and hold long enough, you may not care.

Experts say that over time, vacation-home markets don’t appreciate noticeably better than primary-home markets, so do not let anyone talk you into buying a vacation home for future appreciation. Homes on Martha’s Vineyard appreciated by 40.9% over the past 10 years, edging out Boston’s 40.5%. But Hilton Head’s 15% gain was outshadowed by nearby Charleston, S.C.’s 25.4% rise.

A vacation home is an investment in your family and mental well-being. It’s where you gather to create memories. More than 80% of second-home buyers surveyed by the National Association of Realtors in May reported that they bought vacation homes for sheer, pure consumption: just live in the house and enjoy it.

If you can gather up the dry powder from savings, or the sale of your primary home, you will sidestep the cumbersome and pesky mortgage process. (Listen to me: calling the mortgage process cumbersome and pesky!) Last year, 36% of vacation-home transactions were all-cash deals, up from 29% in 2009, according to the National Association of Realtors. 68% of all Miami homes are cash sales this year, also an increase, most of those buyers foreigners from Brazil. If you are paying cash, do not be afraid to haggle. One agent told me how the South American buyers gives her indigestion.

“They are not afraid to offer us half of what we are asking,” she says.  “Then they act coy and pretend they don’t speak English very well. But they are driving hard, hard bargains when it comes to numbers.”

And, it appears, succeeding. Here’s a listing in Seaside for $750,000.