The characteristic features of a two-sign point vortex system in positive and negative temperature states are examined by massive numerical simulations using MDGRAPE-2. The temperature is determined by a density of states for a microcanonical ensemble consisting of randomly generated $10^7$ states. Since the density of states has a single peak, the system has negative temperature states. The distributions of vortices in time-asymptotic equilibrium states in positive and negative temperature are obtained by time-development simulations. In positive temperature, both-sign vortices mix with each other and neutralize. In negative temperature, part of the vortices condense and form clumps exclusively consisting of the same-sign vortices, while the other part of the vortices distribute uniformly outside the clumps. It is found that the vortices inside the clumps gain energy and the vortices outside the clumps lose energy to keep the total energy constant. This suggests the common and essential role of the background vortices in the energy-conserving system that assists the formation of the clumps as well as the crystallization and generation of the symmetric configuration observed in the non-neutral plasma experiments.