you really interrogate it closely enough you might not be able to meaningfully differentiate it from biological consciousness or thought (which if you really wanna argue could also be reduced to "it's just math" as well, just math that is way beyond the ability of people to determine

Here's one easy way to differentiate it: my brain is wet and runs on electrochemical processes powered by food. Is that a "significant" difference? That depends on what you think is worth tracking! Defining what counts as "functionally identical" requires you decide which features of a system are "functional" and which are "mere" cosmetic differences. That differentiation isn't given to us by nature, though, and already reflects a hefty series of evaluative judgements. By carefully defining our functions, we can call any two things "functionally identical." There's no right answer, which is both a strength and a limitation of this kind of functionalist framework. Both the AI boosters and the AI "impossibilists" miss this point: functional identity is perspectival, and encodes a bunch of evaluative assumptions about which differences do and don't matter. That's ok--all model building does that--but it's important not to confuse the map and the territory, or think we're identifying some kind of value-independent feature of the world when we attribute functional identity.