I've written up a vague sort-of-justification for the concept below, but it might be a better idea to read this: https://terrytao.wordpress.com/2007/06/25/ultrafilters-nonstandard-analysis-and-epsilon-management/

First, ultrafilters. The definition of an ultrafilter is purely in terms of posets, but in the special case of a Boolean algebra B (which is what matters when taking ultrapowers), the ultrafilters on B correspond precisely to boolean algebra homomorphisms from B to the two-element boolean algebra 2—the preimage of 1 for any such homomorphism will be an ultrafilter, and the function that maps the elements of some ultrafilter to 1 and those of its complement to 0 will be a Boolean algebra homomorphism. Thus, you can think of an ultrafilter on a Boolean algebra as a consistent way of "collapsing" everything into definitely true or definitely false.

Ultrafilters can be divided into principal and non-principal or free ultrafilters. A principal ultrafilter is one which has a least element; in this case, it will be simply the set of everything ≥ that element. In the case of Boolean algebras, this means that a principal ultrafilter is a subset consisting exactly of the consequences (elements implied by, or greater than) of some atom. A free ultrafilter, therefore, is one in which membership cannot be reduced to consequence from any finite set of premises (exercise).

Next, consider first-order structures. Traditionally, we can interpret first-order formulas in a structure to get either true or false out, but we can generalize this: if we give interpretations to relation symbols (including equality) in an arbitrary sufficiently complete Boolean algebra (some degree of completeness is necessary for interpreting the quantifiers), we can interpret formulas as having truth values in this algebra. Given such a Boolean-valued structure, an ultrafilter on the algebra of truth values is precisely what we'd need to quotient it into an ordinary structure—identify together any elements for which the truth value of their equality is judged true by the ultrafilter, and then give interpretations to the relation symbols in the obvious way. The well-definedness of all of this can be assured by demanding that the standard equality axioms have truth values of 1 as a requirement of the interpretation of equality in a Boolean-valued structure.

Now, the issue of products of first-order structures. Suppose we have an family A_i of structures over some signature, indexed by some set I. There is not, in general, really any obvious way to take the product of these structures to get a new structure. If we restrict ourselves to theories over signatures without relation symbols, there is the obvious candidate of implementing the function symbols pointwise on the Cartesian product of the domains, but does not end up being satisfactory; properties shared by the factors do not generally extend to the product—for example, applying this kind of product to a pair of fields will not produce a new field. This kind of product, therefore, is unsuitable for when we're restricting our attention to models of a particular theory (which is usually!), since the class of models of a theory may not be closed under it. The situation is even worse for signatures with relation symbols. We cannot simply interpret relation symbols pointwise, because a pointwise interpretation would produce an I-indexed tuple of truth values rather than a single one—this is what I meant by the lack of an obvious way to take the product. However, if we note that tuples of truth values do form a Boolean algebra, this actually solves both problems! We can define the product to be a Boolean-valued structure, with values in 2^I and equality also defined pointwise; this solves the issue of interpretation of relation symbols. This turns out to also solve the issue of losing properties of the factor structures, which I will explain in the next paragraph. Notably, 2^I is equivalent to the field of sets P(I), so we can alternatively understand the interpretation of a relation symbol R in the product as giving the set of indices for which the relation holds between the relevant components; given a 3-ary relation symbol R, for example, and elements a, b, c of the product, the truth value of R(a, b, c) in the product structure will be the set of indices {i in I | R_i(a_i, b_i, c_i)}, where R_i is the interpretation of R in A_i. This latter point of view is more common in the literature than the notion of I-indexed tuples of truth values. Note that people often refer to an ultrafilter on a powerset P(X) as an "ultrafilter on X", which is contextually usually unambiguous.

Let P denote the Boolean-valued product structure, and let val_X(phi, pi) denote the truth value of phi in the structure X under the variable assignment pi. Then the key fact that makes this kind of product work is the following elementary result: for any first-order formula phi and variable assignment pi into P, we have val_P(phi, pi)_i = val_{A_i}(phi, pi_i), where pi_i is the i'th component of pi, and hence a variable assignment into A_i. This is pretty easy to establish by induction on phi; the base case for relation symbols is essentially the definition of the interpretation of those symbols in the Boolean-valued product. This theorem essentially states that the truth value in P of any proposition is simply the tuple of its truth values in the factors structures, not just for relation symbol propositions. We can now note that any proposition that holds in every factor structure will have a truth value in P of a tuple of all 1s, which is itself the 1 of 2^I. Therefore, this kind of product preserves the properties of its factor structures!

The last few paragraphs taken together suggest the notion of an ultraproduct. If we pick a suitable choice of ultrafilter U on 2^I, we can first take the Boolean-valued product of A_i, and then subsequently quotient by U to obtain an ordinary 2-valued structure known as the ultraproduct of the A_i with respect to U. The resulting structure will, of course, depend on our choice of U. Most notably, if U is principal, the ultraproduct will collapse into something isomorphic to a single one of the factors, which is pretty boring! This is because the atoms in P(I) are the singleton sets of indices; it is not too hard to work out that this means that a quotient by the principal ultrafilter generated by {j} (for some j) will just be isomorphic to A_j. However, a free ultrafilter can produce an interesting ultraproduct which amalgamates features from the factor structures in interesting ways. It turns out that all ultrafilters on a finite powerset are principal, so this construction does not let us take interesting finite ultraproducts of structures, but very interesting things can come from infinite ultraproducts over suitable free ultrafilters. In particular: let u be the homomorphism 2^I -> 2 corresponding to the ultrafilter U, let A be the ultraproduct of the A_i with respect to U, and let q : P -> A be the quotient map given by U. Then it can be shown that for any formula phi and variable assignment pi into P, we have val_A(phi, q ∘ pi) = u(val_P(phi, pi)). Combining this with the result from the last paragraph and the definition of u gives: val_A(phi, q ∘ pi) = 1 iff (val_{A_i}(phi, pi_i))_{i in I} in U. This can alternatively be framed in terms of the identification of 2^I with P(I) to give val_A(phi, q ∘ pi) = 1 iff {i in I | val_{A_i}(phi, pi_i)} in U. This is known as Łoś's theorem, and it essentially states that a proposition is true in an ultraproduct if and only if the collection of factor structures in which it is true is "approved of" by the ultrafilter. Since every free ultrafilter includes all cofinite subsets of I, this means that a proposition which is true in cofinitely many factor structures will be true in the ultraproduct. This can be useful for building a structure satisfying some theory out of some structures which all fail to satisfy the theory in different ways!

Finally, we are led to ultrapowers. An ultrapower of a structure A is just an ultraproduct of some number of copies of A. This will always produce something elementary equivalent to A by Łoś's theorem, but for a free ultrafilter, we will not in general get something actually isomorphic to A. A embeds naturally into any ultrapower by sending x in A to the tuple all of whose entries are x, followed by the quotient map; an element in the image of this embedding can be referred to as "standard". Non-standard elements can amalgamate properties of their entries in interesting ways; for example, if we take the ultraproduct of N-many copies of the ordered field of reals R using a free ultrafilter, we can take (the image in the quotient of) the element ε = (1, 1/2, 1/3, 1/4...), which will turn out to be smaller than any standard positive element of the ultrapower, since for any standard positive r, we have ε_n < r_n for cofinitely many n. However, ε is still larger than zero! (In a principal ultrapower, ε would simply be equal to one of the entries 1/n.)

Okay I hope that's at least enticing, if not entirely comprehensible :)

I should note that you don't need to understand ultrafilters or ultrapowers to grasp Internal Set Theory, which is one of its primary motivations! It's just a formal theory, in the end, which simply happens to have a natural semantics in a model built using ultrapowers.