Computer has been a bit wonky last few days with either Firefox crashing out or Explorer freezing. Malwarebytes didn't find anything, and I set Defender to do a full scan but this happened in the middle of that. Got an error pop-up from Google Drive, which I have set for file streaming. But I just decided to shut down the computer for now.
So I believe it was the Defender full scan crawling the Google Drives mapped drive letter and accessing every file it found causing every file to be downloaded and cached. So I'm rerunning the scan with all my cloud drive agents disabled.