Take a bunch of really bright people, who are used to being the biggest frogs in their pond. Put them in a little itty bitty pond filled up with other, equally big frogs. (And self-esteem, unlike self-compassion, is based on how you rank compared to your peers...) Oh, and chop them off from whatever social network they had. (Going from undergrad school, which is one of the most networked places in the US, to your department's little itty bitty social bubble.) All working day, you never see anybody besides your students, your professors, and your colleagues/competition. Hope you've learned how to have a social life outside of school, while you were an undergrad! (Oh, you didn't, because you were too busy studying and hanging out with your college buddies?) Oh, and you're constantly observed and critiqued! Not a single little nuance of behavior goes unremarked!
It's possible to make it through grad school without breaking, but I think we *all* get bent. Keep in mind that you will almost certainly have some free visits to Student Psychological Services to use.