Feature extraction is the key to the fault diagnosis problem, and the variable load of wind turbine gearboxes, large amount of vibration data and low information density make it difficult to balance the feature learning performance and computational cost of multimodal fusion diagnosis method. To this end, a wind turbine gearbox fault diagnosis method based on multimodal time-frequency map fusion is proposed. First, a wavelet packet decomposition algorithm is used to analyze the fault features of the gearbox vibration data set; then, the decomposed sub-signal and the original signal are transformed into two-dimensional time-frequency images to form complementary fault features reflecting the wavelet domain and the time-frequency domain; finally, a convolutional neural network (CNN) is used to learn the deep features of the image texture and fuse them separately to train a CNN-ViT (vision transformer) multimodal fault diagnosis model. The proposed model is validated with Case Western Reserve University gearbox bearing data, and has higher accuracy compared with other unimodal and multimodal methods under variable load and unknown faults, and the time-frequency map fusion method can realize the fault feature learning in both wavelet domain and time-frequency domain dual modality with lower computational cost.