Massachusetts is one of the oldest states in the nation, and we've had a knack for serving up great food since the very beginning. From cozy colonial inns to dignified 19th-century restaurants, this state isn't lacking in amazing places to eat with plenty of rustic charm. Check out these places across Massachusetts that combine delicious food with incredible local history.

1. The Union Oyster House, Boston

The Union Oyster House is actually the oldest continuously operating restaurant in America. It has played host to founding fathers and even European royalty: while exiled, King Louis Philippe of France lived on the second floor of the restaurant in 1796 and made money by teaching French to young American women. Today, this famous seafood establishment is still one of the best places to order a round of fresh oysters in the city. (41 Union Street, Boston)

2. Old Yarmouth Inn, Yarmouth Port

Stop by the Old Yarmouth Inn for homemade comfort food and freshly-caught seafood. Housed in a building that's been around since 1696, this place strikes the perfect balance between coziness and elegance. Try the baked and stuffed lobster (pictured) for a delicious and hearty seafood treat. (223 Route 6A, Yarmouth Port)

3. Publick House Historic Inn, Sturbridge

This colonial-era inn is home to a cozy tavern that will make you feel like you've stepped back in time. With classic local cuisine like stuffed fish, perfectly seasoned grains and fresh vegetable sides, you'll feel right at home at this restaurant. Your bread basket will include the tavern's famous sticky buns, as well as cornbread and pumpkin nut bread. (277 Main Street,

Sturbridge)

4. Vienna Restaurant and Historic Inn, Southbridge

For authentic Austrian cuisine and amazing service, look no further than the Vienne Restaurant. The restaurant is housed in a historic inn, and has charm in spades. If you're looking for a break from colonial-style dining, this spot is perfect. Try their famous Linzertorte (pictured) while relaxing in the beer garden by the outdoor fire. (14 South Street, Southbridge)

5. The 1761 Old Mill, Westminster

The 1761 Old Mill restaurant is one of the most charming historic restaurants in the state. Housed in an old mill and overlooking a beautiful waterfall, this place specializes in heaping helpings of cozy comfort food. It's the perfect place for a romantic meal or a family outing. There's a hidden bar, several dining rooms and a lovely porch overlooking the flowing water. Don't miss the mill's signature pecan rolls, served hot and sweet. (69 State Road E, Westminster)

6. Longfellow's Wayside Inn, Sudbury

Literary giant Henry Wadsworth Longfellow named one of his most famous books of poems, "Tales of a Wayside Inn," after this charming colonial gathering place. With gorgeous original architecture – think dark wood, hand-hewn beams and low ceilings – this place looks like it could have tumbled straight out of the 18th century. Enjoy delicious entrees like lobster pie, and indulge in their tasty homemade bread and classic apple pie. (72 Wayside Inn Road, Sudbury)

7. The Red Lion Inn, Stockbridge

Another Stockbridge gem, The Red Lion Inn is a town landmark. It first opened in 1773 and has been operating ever since. This historic restaurant features a delightful front porch, a dining room filled with soft candlelight antique china, and a menu filled with classic New England fare. Try the roasted local turkey and finish up with a bowl of warm Indian pudding. (30 Main Street, Stockbridge)

8. The Salem Cross Inn, West Brookfield

Located on 600 acres of pristine farmland and woodlands, this colonial inn is a sprawling time capsule filled with delicious, traditional cuisine. The restaurant is housed in a restored 1700s-era farmhouse, and offers a slew of historically-themed dining events. Stop by for a Fireside Feast, complete with slow-roasted prime rib cooked in a massive hearth, deep-dish apple pie prepared in front of your eyes, and homemade clam chowder made in a large cauldron over the fire. (260 West Main St

West Brookfield)

9. Concord's Colonial Inn, Concord

This lovely colonial restaurant is located right in downtown Concord. The building was constructed in 1716 and has been an operational inn and restaurant since 1889. The dining room here is large, but someone still manages to be cozy and intimate. There are plenty of historical landmarks and Revolutionary War sites in the surrounding area, so you can make your visit here part of a larger tour. Be sure to sample the amazing chicken pot pie with its flaky, buttery crust. (48 Monument Square, Concord)

10. Rye Tavern, Plymouth

Are you looking for more amazing historical spots in Massachusetts? Check out these 15 historical landmarks you must visit before you die.