lol I randomly searched for a page from collabora about how they reverse engineer GPU drivers (since I've seen that they've done that for other GPUs) and the very first page I went to was written by this person. Apparently this was a side project while she did fairly similar work on Mali GPUs at Collabora.

Anyways, here's what she said about the process:

Typically, for Linux or Android driver reverse-engineering, a small wrap library will be written to inject into a test application via LD_PRELOAD that hooks key system calls like ioctl and mmap in order to analyze user-kernel interactions. Once the “submit command buffer” call is issued, the library can dump all (mapped) shared memory for offline analysis...

it took some elbow grease to identify the three critical calls: memory allocation, command buffer creation, and command buffer submission...

With those obstacles cleared, we can finally get to the shader binaries, black boxes in themselves. However, the process from here on out is standard: start with the simplest fragment or compute shader possible, make a small change in the input source code, and compare the output binaries. Iterating on this process is tedious but will quickly reveal key structures, including opcode numbers.

From https://alyssarosenzweig.ca/blog/asahi-gpu-part-1.html , there's a lot more details about the subsequent parts of the process in the next articles but they're harder to summarize