The potential of polymer nanocomposites as dielectric materials is an active area of research, due to the promising electrical properties that may result from nanostructuration. Recently, the effect of water absorption in nanocomposites has been considered and it is found that nanocomposites can absorb significantly more water than the equivalent unfilled polymer when exposed to humid environmental conditions. This is due, supposedly, to the presence of the nanoparticle-matrix interface, which could be a preferred location for the aggregation of water molecules. Nevertheless, from an application perspective, the presence of water is not welcomed as it might negatively affect the overall electrical properties. This paper reports on a dielectric spectroscopy investigation into polyethylene systems that contain different amounts of nanosilica with different surface chemistries. Results indicate that the permittivity and loss tangent of nanocomposites are higher than that of the unfilled polyethylene, especially in the low frequency range. Furthermore, water absorption tests show that nanocomposites do absorbed significantly more water than unfilled polyethylene, with the consequence that both the permittivity and loss tangent increase with increasing amounts of water. However, appropriate surface treatment of nanosilica was found to reduce the water absorption effect and to modify the dielectric response of nanocomposites compared with those containing untreated nanosilica. While water absorption may not be a technologically desirable characteristic, our results suggest that water molecules can act as effective dielectric probes of important factors.