Do I owe US state tax while living overseas?

Maybe. Some states release residency when you sever ties; others (e.g. certain high-tax states) may still treat you as a resident if you maintain domicile. State rules are separate from federal FEIE.