How much do you know?
I think this is the interesting question for any culture that is living anything above sustenance levels.
Hunter-gather tribes in some jungle the arse-end of nowhere: ok fair enough. They could probably survive as they don't rely on any infrastructure. But the technological environment on which we survive is a house of cards in terms of infrastructure. Nobody really knows how everything works. Most people at a basic level couldn't farm or look after animals effectively, and then there are whole layers of complexity above that and I don't think there any single person who really knows how *everything* works.
If society crumbled, and you had a random sample of survivors, could they rebuild society to where it now? No way.
The more advanced we become, the more susceptible we are to being completely put back to nothing by a major insult. Technological advancement means we can cope with smaller problems better (minor epidemics, seasonal variation in weather) but the big stuff could really fuck it up. We're playing a game with big stakes.