This research attempted to empirically investigate how individuals experience “self” and “others” according to the theory of sense of self from nonverbal aspects. In the survey, participants (N = 15) read a short story and discussed it with the researcher. All dialogs were video recorded and nonverbal behaviors of participants/the researcher in the obtained video data were coded based on analysis index. Quantified nonverbal behaviors were analyzed using moving correlation method, and time series changes of nonverbal behavioral synchronization between participants and the researcher were examined. Results revealed that in those with a high sense of self, active mutual exchange was maintained between self and others in the non-linguistic domain. Conversely, in those with a low sense of self, mutuality in the nonverbal area was weak. This result suggests the importance of nonverbal areas in considering the problems in sense of self of adults.