•We demonstrate a novel SERS/fluorescence dual-mode nano-sensing system in which the conformational switching of the human telomeric G-quadruplexes results in the signal transduction.•Its SERS/fluorescence dual-modalities nature effectively extends the response range of the nanosensor, leading to significant improvement in the sensing performance.•It exhibits a LOD as low as 5 pM for Hg2+ ions, which is ~100 times more sensitive than conventional optical sensors.DNA-metal nanoparticle conjugates have been increasingly exploited for sensing purposes over the past decades. However, most of the existing strategies are operated with canonical DNA structures, such as single-stranded forms, stem-loop structures, and double helix structures. There is intense interest in the development of nano-system based on high order DNA secondary structures. Herein, we propose a SERS/fluorescence dual-mode nanosensor, where the signal transduction mechanism is based on the conformational switching of the human telomeric G-quadruplex DNA. The nanosensor exhibits excellent SERS/fluorescence responses to the complementary strands of G-quadruplexes. Based on T–Hg2+–T coordination chemistry, this sensor is effectively applied to determination of Hg2+ in buffer solution and real samples. It achieves a limit of detection (LOD) as low as 1 ppt, which is ~100 times more sensitive than conventional optical sensors. We anticipate that the proposed G-quadruplex-based nanosensor could be applied to the analysis of other metal ions and small molecules in environmental samples and biological systems.