If you really want out of the city, rent a car and go to Lancaster, PA. It doesn't get much more country than that! I don't know if there's bus service from NYC, and I know there isn't easy public transportation in Lancaster (unless you want to take a horse & buggy) but it's fun!
Threaded Mode