Everyone puts up walls. It may be that what feels personal to one person is not so personal to another, but we all have our own boundaries and certain things which we feel safer keeping private. I don't think this is particularly an INFP issue, maybe just an issue you're having with your friend because he doesn't understand why you choose to guard the things you have appeared to guard.
It's also worth noting that everyone has a different identity around every individual or group of individuals with whom they interact. At that, it's natural to (subconsciously) take on things like others' accents, body language, and general mannerisms as a way of building rapport (the technical term is "mirroring"), and pretty much everyone does this, as well, although the better and less conspicuously you do it, the more receptive of you people tend to be.
I don't know what else to tell you. I mean, we all have "social selves," and we all have varying degrees of comfort as to how much we're willing to let people get to know us. I would think, as a general rule, introverts would have a lot of stuff going on that not even closest friends would know about, just because that's how introversion works--getting energy and ideas and so on when withdrawn from others and having a much richer inner life as opposed to what appears on the surface.