Competing spin-orbital singlet states in the 4d4 honeycomb ruthenate Ag3LiRu2O6

When spin-orbit-entangled d electrons reside on a honeycomb lattice, rich quantum states are anticipated to emerge, as exemplified by the d5 Kitaev materials. Distinct yet equally intriguing physics may be realized with a d-electron count other than d5. The magnetization, Li7-nuclear magnetic resonance (NMR), and inelastic neutron scattering measurements, together with the quantum chemistry calculation, indicate that the layered ruthenate Ag3LiRu2O6 with d4Ru4+ ions at ambient pressure forms a honeycomb lattice of spin-orbit-entangled singlets, which is a playground for frustrated excitonic magnetism. Under pressure, the singlet state does not develop the expected excitonic magnetism, but two successive transitions to other nonmagnetic phases were found in Li7-NMR, neutron diffraction, and x-ray absorption fine structure measurements, first to an intermediate phase with moderate distortion of honeycomb lattice and eventually to a high-pressure phase with very short Ru-Ru dimer bonds. While the strong dimerization in the high-pressure phase originates from a molecular orbital formation as in the sister compound Li2RuO3, we argue that the intermediate phase represents a spin-orbit-coupled singlet dimer state which is stabilized by the admixture of upper-lying Jeff=1-derived states via a pseudo-Jahn-Teller effect. The emergence of competing electronic phases demonstrates rich spin-orbital physics of d4 honeycomb compounds, and this finding paves the way for realization of unconventional magnetism.