The same thing literally just happened to me over this weekend. I had it plugged into a charger, and it shut off mid-game while playing in PCVR. When I booted it back up, it was in low-battery mode and turned back off, so I re-seated the charging cable and tried again. After half an hour of charging it was still in the battery-too-low-to-boot mode. Eventually it wouldn't even show the low-battery (charging icon) screen and only had a flashing red LED indicator. Then that stopped. I tried my 65W charger that was working with it before, my Steam Deck charger (45W), and even a slow charge overnight from USB-A to -C as others have suggested.
It took 3 days of back and forth with customer service having me try everything I've already tried (make sure charger is 45W, disconnect and reconnect the battery, etc.) but they're finally approving the RMA. I'm just waiting on the shipping label now. I'm glad I kept the box it shipped in.
I've had my Frame for about a month.
Edit: sorry for all the posts, I kept getting a connection error failed to post comment thing but I guess it was posting it anyway.