Equivalently, using the notation for vincular patterns, a Baxter permutation is one that avoids the two dashed patterns and .
For example, the permutation in (written in one-line notation) is not a Baxter permutation because, taking
, and , this permutation violates the first condition.
In fact, this formula is graded by the number of descents in the permutations, i.e., there are
Baxter permutations in with descents.
[3]
Other properties
The number of alternating Baxter permutations of length is , the square of a Catalan number, and of length is
.
The number of doubly alternating Baxter permutations of length and (i.e., those for which both and its inverse are alternating) is the Catalan number .[4]
Baxter introduced Baxter permutations while studying the fixed points of commuting continuous functions. In particular, if and are continuous functions from the interval to itself such that for all , and for finitely many
in , then:
the number of these fixed points is odd;
if the fixed points are then and act as mutually-inverse permutations on
and ;
the permutation induced by on uniquely determines the permutation induced by
on ;
under the natural relabeling , , etc., the permutation induced on is a Baxter permutation.
^Ackerman, Eyal; Barequet, Gill; Pinter, Ron Y. (2006), "A bijection between permutations and floorplans, and its applications", Discrete Applied Mathematics, 154 (12): 1674–1684, doi:10.1016/j.dam.2006.03.018, MR2233287.