You probably want to look for books on reverse engineering. And a book on assembly for your CPU.

I learned assembly language for VAX-11 (this was like 30+ years ago) in a CS class. We also learned 6502 assembly in a computer engineering class. Neither book would help you. You want a book specific to whatever CPU you're using.

Now, I never took it, but friends in college took a CS Compilers course where they learned the basics of writing a compiler. But that's not what you're talking about though it might help.

Trying to understand what a program does is reverse engineering. And a tool like IDA Pro would help you understand subroutines, variables, flow, library calls, and so on.

A debugger will be invaluable for seeing a program execute one instruction at a time.

You would need to know the assembly language for your CPU. And it would help to become familiar with certain patterns. I haven't done much assembly (but I have done assembly on a few different CPUs) nor much reverse engineering so I'm not sure I can lend a whole lot of insight there.

As you learn assembly instructions, you will start to understand how loops, subroutines, if/then/else, and other things are accomplished for your CPU.

For example, if/then/else and loops are often accomplished with conditional branching. The conditions are based on CPU flags (bits in the Status Register) that are set by a comparison instruction. You'll start to recognize how if/then/else and loops and other things are commonly implemented in assembly (without necessarily having to study the compiler; it will be obvious without knowing anything but assembly).

Another example might be how C structs are implemented. Some CPUs provide convenient memory addressing modes for structs, some don't. Nearly all I am familiar with provide a convenient way to reference arrays with a simple index.

Subroutines are jumps to a set location and at the end of that code is a return instruction. Usually registers have to be saved when jumping and restored when returning. Arguments to the subroutines are pushed on to the stack either by value or by reference. Return value is provided through some convention (machines with lots of registers might always use one particular one for return).

I guess bottom line, learn assembly for your particular CPU, then take a crack at using a debugger and disassembler / reverse engineering tool.

I'm not entirely sure I follow why that is needed to learn how to do branching in forth but I only vaguely remember that language. Maybe if I did it would be more clear.

Anyway I hope this helps at least a little.