When you pay principal, you are gaining that much value back as equity. It makes more sense if you think of a loan for something physical like a mortgage. If you pay $100 of principal on your mortgage, that money turns into equity that you own in your home so that when you sell you get that much more (in a simplified way).

You aren't losing the $100 you pay in principal, it's just transferring into an asset rather than liquid cash. With a student loan, that asset is your degree/education. It's a little different than a mortgage because the bank can't repossess your degree, but the underlying logic is the same.

You could also think of it like paying for your degree on a payment plan. You wouldn't expect to get a tax writeoff on your couch just because IKEA let you pay in monthly installments.