I forget what Fullmetal Alchemist called Germany, it’s been so long.

Probably because it was never in Germany. The author drew a lot of inspiration from Europe during the 19th and 20th century. People make the connection to Germany because the militaristic government structure draws parallels with fascism and the leader is called fuhrer. But the major cities are literally called Central city, East city, South city, West city and North city, the official language is English and the military attire seems to be inspired from the French military.

Do the Manga/does the source material of the animes of SpyxFamily and Tanya speak of Germany explicitly?

In both anime, it's very obvious what the historical inspiration is. I didn't find it unfitting to use replacement/made up names and not call them real-world names.