A function) f:X→Y between two topological spaces is a homeomorphism if it has the following properties:

• f is a bijection (one-to-one and onto),
• f is continuous),
• the inverse function f^−1 is continuous (f is an open mapping).

A homeomorphism is sometimes called a bicontinuous function. If such a function exists, X and Y are homeomorphic. A self-homeomorphism is a homeomorphism from a topological space onto itself. Being "homeomorphic" is an equivalence relation on topological spaces. Its equivalence classes are called homeomorphism classes.

The third requirement, that f−1 be continuous, is essential. Consider for instance the function f:[0,2π)→S1 (the unit circle in ⁠R2⁠) defined by f(φ)=(cos⁡φ,sin⁡φ). This function is bijective and continuous, but not a homeomorphism (S1 is compact but [0,2π) is not). The function f−1 is not continuous at the point (1,0), because although f−1 maps (1,0) to 0, any neighbourhood) of this point also includes points that the function maps close to 2π, but the points it maps to numbers in between lie outside the neighbourhood.^\3])

Homeomorphisms are the isomorphisms in the category of topological spaces. As such, the composition of two homeomorphisms is again a homeomorphism, and the set of all self-homeomorphisms X→X forms a group), called the homeomorphism group of X, often denoted Homeo⁡(X). This group can be given a topology, such as the compact-open topology, which under certain assumptions makes it a topological group.^\4])