We model a community of individuals whose relationships are governed by the rules of the so-called Heider balance theory, but modified to address the impact of tolerating intolerant individuals. To consider tolerance toward a different group, the elements are assigned one of the two flags, A or B, and the elements of each group can be tolerant or intolerant. Two additional parameters, p and q, respectively, characterize the propensity of elements to cooperate and the propensity of tolerants to reject intolerant attitudes. We find that (1) parameter q does not affect the degree of conflict at the micro level, but has an important influence on the degree of conflict in the whole system; (2) segregation into two cliques occurs whenever there exists intolerants in both groups; (3) when intolerants are present in only one of the groups, segregation can be avoided for appropriate combinations of parameters p and q that depend on the fraction of intolerants and the size of the groups; (4) as the size of the system increases, two balanced solutions dominate: segregation into two cliques or the isolation of intolerants; and (5) endemic partially balanced configurations are observed in large systems.