Precise X-ray diffraction, heat capacity, magnetoresistance and magnetization measurements have been carried out on high quality Tm1-xYbxB12 single crystals with the goal to reconstruct the H-T and angular H-phi magnetic phase diagrams in the (110) plane of these antiferromagnets (AF) with dynamic charge stripes and Yb-ion valence instability. The analysis developed here allowed us to conclude in favor of essential changes in the filamentary structure of fluctuating charges, and the emergence and variation of stripe-induced magnetic and charge transport anisotropy that are controlled by lowering the temperature and ytterbium doping. It was found that local charge and spin fluctuations on Yb-sites suppress strongly the complicated AF state. However, the magnetic anisotropy is conserved, and the only moderate modifications of the Maltese Cross -type magnetic phase diagrams are detected in the range x < 0.2. We argue that the AF ordering of Tm3+ magnetic moments is the main factor, which determines the anisotropy in the Ne ' el phase of Tm1-xYbxB12 with carrier-mediated magnetic indirect RKKY exchange renormalized significantly by quantum fluctuations of the electron density along 110 directions. PACS: 73.22.-f, 75.47.-m, 71.27. + a.

E. S. Zhukova, B. P. Gorshunov, G. A. Komandin, L. N. Alyabyeva, A. V. Muratov, Yu. A. Aleshchenko, M. A. Anisimov, N. Yu. Shitsevalova, S. E. Polovets, V. B. Filipov, and N. E. Sluchanko 1 – Moscow Institute of Physics and Technology, 141700 Dolgoprudny, Moscow Region, Russia 2 – Prokhorov General Physics Institute of the Russian Academy of Sciences, 119991 Moscow, Russia 3 – Lebedev Physical Institute, Russian Academy of Sciences, 119991 Moscow, Russia and 4 – Frantsevich Institute for Problems of Materials Science, National Academy of Sciences of Ukraine, 03680 Kiev, Ukraine (Dated: November 2, 2018)