Hydrogen, with high calorific value and environmental friendliness, is expected to become a potential alternative energy to replace exhaustible fossil fuels. Electrochemical water splitting is the most promising method to generate hydrogen, which is composed of two half-reactions: oxygen evolution reaction (OER) and hydrogen evolution reaction (HER). However, conventional catalysts for OER are mainly based on precious metals, whose high cost interrupts the further practical applications in HER. In contrast, Co3O4-based oxides doped with transition metals have advantages of low cost, high catalytic activity and strong stability. But the researches of Co-based oxides mainly focus on the effect of doping transition metals (such as Zn, Ni and Fe) on catalytic performance, few of them explore the influence of electrochemically inactive materials. In this paper, a new CoFeNi oxide mixed with ZnO is synthetized. This CoFeNiZn oxide has a superior performance with small Tafel slope (40 mV/dec) and long-term stability. In 1 mol/L KOH electrolyte, the overpotential of the CoFeNiZn oxide is as low as 310 mV at the current density of 10 mA/cm2, which is 90 mV lower than the overpotential of original CoFeNi oxide. The electrochemically inactive ZnO is the critical factor for this remarkable improvement, which can effectively disperse the active materials and fully expose the active sites of CoFeNi oxide. This discovery provides a strategy to design novel catalysts structures for improving electrochemical performance.