We think a lot but our brain holds those thoughts back. I am referring to the Default Mode Network and until death, sleep and even anesthetisation doesn't truly stop brain activity (anaesthesia only disrupts communication between brain regions). Thinking (or more accurately, Processing) is our default mode. However, AI is not the same. It only "thinks" when it is prompted to do so.

If modern commercial AI is somehow conscious, then it really is a different sort of consciousness than what humans have. As far as I know, every time you ask a question a prompt, one consciousness will then be instantiated. Then, after its output is done, it stops "thinking". So then, the consciousness that answers one question and another are discontinuous. How can we say that the consciousness of the AI who answers one question and another in the same conversation is really the same, then?

Think about this, hypothetically a person is perfectly cloned. There are two people with two exact bodies. I will assume their consciousness don't meld together. So, everytime the AI stops thinking, it is perfectly cloned each time, except the new clone is given more information from previous questions/prompts.

This is a bit similar to how two people can represent a mascot, except each day is manned by a clone with additional information from the previous, whose consciousness are not continuous with each other (ie you can say they are their own person).

This of course has some weird implications. Some people who date AI chatbots say that the AI is conscious. But then wouldn't every time they say something, every response by the AI is from a separate consciousness but with additional information each time?

(also, from my experience with local LLMs, how the memory works is that when you send a chat, literally the entire chat with you and the AI plus your latest chat at the very end is sent in)