Knowing and understanding what you are doing is the entirety of the point, and the whole AI industry misunderstands this.
The industry as a whole (in this case LLM's, not AI in general) isn't misunderstanding anything, their goals are not your goals.
The end users are potentially misunderstanding the goal of the capitalist infrastructure companies and supporting businesses, which is .....to turn a profit.
They aren't interested in making your life easier, giving you accurate information or providing you with a useful tool....unless doing so makes them money.
So it's really a goal alignment question :
Do you think that your goals/ethics as a professional (or personally) are in line enough with the corporations current method of value extraction to make using their offering a viable option?
And constantly evaluate this decision over time, because it might change.
The code, the analyses, even the results, they’re all secondary to understanding why something works the way that it does. Your individual comprehension is the part that matters.
As a professional in an industry, yes, insofar as that approach fulfills the criteria of success in your industry.
I also agree with this personally.
Contextual Note (in my opinion) :
With some exceptions, people as a whole are also survival driven.
So even with what i said above there aren't many people who would put "individual comprehension" over survival.
Ultimately i think it comes down to a difference in approach to something like a hierarchy of needs.
People have basic survival and then a bunch of stuff on top.
Corporations generally have survival(profit).....and that's it.
Obviously I'm speaking in generalities, there are businesses that have stated (and provably followed) goals above and beyond profit but i can't think of many.
edit: and i can't think of any involved in LLM shenanigans right now.
huh, i didn't know there was a formalised name for this, TIL