There will be much debate in the halls of power and at cocktail parties as to the reason the Taliban were able to sweep through the Afghan National Army so quickly and why it caught the US military and the intelligence community so off guard. Having worked this problem in my years at the Pentagon I can assure you that it did not, at least not at the level of initial assessments. No one I worked with on our task force had any illusions about what Afghanistan was, a backward tribal conglomeration, not a nation in any way. As you got higher up the food chain assessments were changed to meet the desires of the decision makers, or at least the perceived desires. For what ever reason the United States spent 20 years trying to make Afghanistan a mirror image of the west. This was impossible. Afghanistan is a made-up country designed by the British to keep the Russian Empire from touching the British Empire. Even the shape of the Badakhshan province was created so that Russian did not touch India.
For a multiethnic country the United States has a hard time understanding the existence of other multiethnic countries. While the ethnic diversity in the US has grown to accept a national identity, the Afghans have not. While the Pashtun make up a plurality of the population, between 38% and 42%, they have been accepted as representative of the total population by the US. The rest of Afghanistan: Hazara, Uzbek, Baloch, Nuristani, Tajik, Hani and Kasi have different cultures and languages. There is no national identity or loyalty to a central government and for 20 years the US continued to treat the entire country as Pashtun. Afghans are tribal and adhere to their tribal cultures. The US leaders could never quite grasp this and tried to form a central government and national army. There was an attempt to make an ethnically diverse Army. Unless we were willing to accept Afghanistan for what it was, we were doomed.
Without this basic understanding of the people and culture we were never going to succeed. The only way for any success was to first let the military do what they are best at, killing people and braking things. Once that was done, we needed to accept that it would take generations to convert the people and during this time the military needed to stay in place. I feel for the people we left behind, especially for the female population. It would be interesting to read the After Action Reports and the Lessons Learned papers that will come out of this, but I doubt this will ever happen.