idk about the glass cleaner, but I know Wipe-Out can be safely used to clean frog terrariums and water dishes. Since they absorb chemicals directly through the mucous membrane of their skin, their sensitivity is kind of on par with a fish, which is breathing it.
You could do rinses with scalding water, followed by scrubbing and another rinse to be absolutely sure.