Polymeric electrolytes have attracted recent research interest because they offer the advantages of being safe and non-flammable, having no dendrite formation, and having no possibility of leakage. The incorporation of synthetic polymers to gel electrolytes has numerous disadvantages: for instance, the required preparation time for creating gel electrolytes from synthetic polymers is dubious and lengthy. Additionally, the conventional pristine polymer gel electrolyte layer has been reported to have low ionic conductivity. This work is focused on preparing a thin flexible gel electrolyte layer by using a naturally occurring wood-based nanofiber cellulose (NFC) hydrogel, to overcome the energy and time consumption of conventional processes. In addition, we use polyvinyl alcohol (PVA) as an additive to the NFC hydrogel in controlled amounts to fabricate a stable thin gel electrolyte layer. By using x-ray diffraction, optical microscopy, and Fourier transform infrared spectra studies, we were able to further our understanding of the microstructure of the films: i.e., the penetration and cross-linking (changes in the bonding structures) of semi-crystalline PVA and hydrogel to form a flexible gel electrolyte layer. The NFC hydrogel-PVA films resulted in much higher ionic conductivity values when compared to other existing pristine polymer electrolytes. The addition of KOH to the NFC hydrogel-PVA further enhanced the ionic conductivity. The best ionic conductivity recorded was 75 mS/cm for films with thickness in the range of 200–350 µm, which is comparable to the highest reported ionic conductivity values of gel electrolytes.