A novel phosphorus-containing hyperbranched polysiloxane (P–HSi) with a great amount of phosphaphenanthrene and silanol groups was synthesized by a hydrolysis of self-made phosphorus-containing triethoxysilane. Based on this, P–HSi was used to develop a new high performance flame retardant cyanate ester (CE) resin with simultaneously improved integrated properties. A small addition of P–HSi (5 wt%) to CE can remarkably increases the flame retardancy of CE resin, where the content of P element is only as low as about 1.8 wt%. More attractively, the incorporation of P–HSi to CE resin significantly improves the thermal stability and mechanical properties, completely overcoming the disadvantages of phosphorus flame retardants. Specifically, for the modified CE resin with 15 wt% P–HSi, its initial degradation temperature is about 58 °C higher than the corresponding value of original CE resin; moreover, its impact and flexural strengths are about 2.7 and 1.5 times of the corresponding values of CE resin, respectively. In addition, the P–HSi/CE resins have obviously decreased curing temperature and improved dielectric properties. These outstanding integrated properties of P–HSi/CE resins show that P–HSi is an effective and multi-functional flame retardant for developing high performance resins.