Fossil groups origins. I. RX J105453.3+552102 a very massive and relaxed system at z~0.5

Context: The most accepted scenario for the origin of fossil groups is that they are galaxy associations in which the merging rate was fast and efficient. These systems have assembled half of their mass at early epoch of the Universe, subsequently growing by minor mergers, and therefore could contain a fossil record of the galaxy structure formation. Aims: We have started an observational project in order to characterize a large sample of fossil groups. In this paper we present the analysis of the fossil system RX J105453.3+552102. Methods: Optical deep images were used for studying the properties of the brightest group galaxy and for computing the photometric luminosity function of the group. We have also performed a detail dynamical analysis of the system based on redshift data for 116 galaxies. Combining galaxy velocities and positions we selected 78 group members. Results: RX J105453.3+552102 is located at ⟨z⟩ = 0.47, and shows a quite large line-of-sight velocity dispersion σv 1000 km/s. Assuming the dynamical equilibrium, we estimated a virial mass of M(<R_200) ~ 1 × 10^{15}/h_70 M_sun. No evidence of substructure was found within 1.4 Mpc radius. Nevertheless, we found a statistically significant departure from Gaussianity of the group members velocities in the most external regions of the group. This could indicate the presence of galaxies in radial orbits in the external region of the group. We also found that the photometrical luminosity function is bimodal, showing a lack of M_r~-19.5 galaxies. The brightest group galaxy shows low Sèrsic parameter (n~2) and a small peculiar velocity. Indeed, our accurate photometry shows that the difference between the brightest and the second brightest galaxies is 1.9 mag in the r-band, while the classical definition of fossil group is based on a magnitude gap of 2. Conclusions: We conclude that RX J105453.3+552102 does not follow the empirical definition of fossil group. Nevertheless, it is a massive, old and undisturbed galaxy system with little infall of L^∗ galaxies since its initial collapse.