I agree and disagree. Breasts are the single most visible (usually) defining difference between girls and women and men and women, (outside of genitalia itself). The sight of them may, subconsciously, confirm the subject of desire is female and, therefor, fertile.
I am certainly not controlled by them. In my more immature days, I was more drawn to women with larger breasts...now, I notice much less the size and more whether they at least exist on the woman of interest. They are also an area that typically causes arousal for women which makes them even more desirable to men. I find them attractive, in all shapes and sizes, but, I am certainly not obsessed. And, when a women wears tops that are obviously designed to "put her BREASTS OUT THERE", I find it unattractive now. That wasn't the case in my early 20's but, I just find it...tacky, I guess.
Fact is though...big breasts, small breasts....doesn't matter. It comes down to personality combined with physical attraction. Both are necessary to remain attracted to an SO...Be it sexual or emotional attraction. I don't think most mature men are as obsessed as women may think.
My $0.02