OUP user menu

The Phylogenetic Position of Cetaceans: Further Combined Data Analyses, Comparisons with the Stratigraphic Record and a Discussion of Character Optimization

Maureen A. O'Leary
DOI: http://dx.doi.org/10.1093/icb/41.3.487 487-506 First published online: 1 June 2001


A recent total evidence analysis of the position of cetaceans (whales, dolphins and porpoises and extinct relatives) among mammals indicated that the phylogeny of these taxa remains poorly resolved. Molecular data show that 1) the order Artiodactyla (even-toed ungulates) is paraphyletic unless whales are included within it and 2) that the traditional relationships of clades within Artiodactyla are not supported. This controversy also affects the position of a wholly extinct clade, Mesonychia, which has been argued to be the group of terrestrial mammals most closely related to whales. Here I update a previous total evidence analysis by adding several hundred new informative molecular characters from the literature. Even with the addition of these characters the phylogeny remains unresolved. All most parsimonious trees, however, indicate a paraphyletic Artiodactyla with conflict existing over the exact sister taxon of Cetacea. Congruence between different equally parsimonious cladograms and the stratigraphic record as measured using the modified Manhattan Stratigraphic Measure shows that all of the competing topologies, including those with a paraphyletic Artiodactyla, are significantly congruent with the stratigraphic record. In fossil taxa cladistic optimization can be used as an alternative to “argument from design” (Lauder, 1996) to reconstruct behavior and soft tissues that do not fossilize or osteological characters that have not preserved. In certain scenarios of cetacean phylogeny, optimization indicates that taxa such as the archaic whales Ambulocetus and Pakicetus, and possibly mesonychians, are more correctly reconstructed without hair.


Living cetaceans (whales, dolphins and porpoises) represent some of the most bizarre mammals in the extant biota. They are morphologically distinct from other placental mammal clades. They lack hair, sweat glands and sebaceous glands and have a dorsal fin; they cannot engage in terrestrial quadrupedal locomotion, and they achieve among the largest body sizes ever known in animals (Nowak, 1991). Positioning these strange creatures in the phylogenetic tree of mammals has frustrated even such accomplished paleontologists as G. G. Simpson (1945, p. 213) who commented that, “Their [cetaceans'] place in the sequence of cohorts and orders is open to question and is indeed quite impossible to determine in any purely objective way. There is no proper place for them in a scala naturae.

Since Simpson's remarks, however, the study of cetacean phylogeny has been advanced by an impressive series of new paleontological discoveries (e.g., Ting and Li, 1987; Gingerich et al., 1990, 1994; Thewissen et al., 1994, 1996, 1998; Zhou et al., 1995) and the description and reevaluation of fossils collected over the last century (e.g., Van Valen, 1966; O'Leary and Rose, 1995; Rose and O'Leary, 1995; Rose, 1996; Luo, 1998). The field has also been drastically reshaped by the advent of modern phylogenetic systematics methods (Hennig, 1966) and their application to this problem (e.g., Prothero et al., 1988; Thewissen, 1994; Geisler and Luo, 1998; O'Leary, 1998; O'Leary and Geisler, 1999; O'Leary and Uhen, 1999; Luo and Gingerich, 1999; Luo, 2000). Particularly important has been the rapid increase in molecular sequence data available for phylogenetic research (e.g., Gatesy, 1998, 1999; Gatesy et al., 1999a; Gatesy and Arctander, 2000) and the incorporation of these data into total evidence analyses (e.g., Gatesy et al., 1999a, b; O'Leary, 1999).

Due to the extreme anatomical and physiological modifications present in extant cetaceans, evolutionary biologists have traditionally turned to the fossil record to discover intermediate forms that would untangle the complicated ancestry of this clade. Van Valen (1966) argued from paleontological evidence that cetaceans, including the archaic whales known as the Archaeoceti (now recognized to be paraphyletic, see references above), are not most closely related to carnivoran mammals, but instead to ungulates or hoofed mammals. This conclusion derived from his observation that archaeocetes share many characteristics with mesonychians, an extinct clade of Early Tertiary mammals that exhibit a mixture of carnivoran and ungulate features. Mesonychians have sharp teeth with tall protoconids like carnivorans and cetaceans but also hooves like ungulates (Van Valen, 1966; O'Leary and Rose, 1995; O'Leary, 1998; Gatesy and O'Leary, 2001).

The discoveries of Basilosaurus (Gingerich et al., 1990) and especially, Ambulocetus (Thewissen et al., 1994, 1996), two archaeocetes so primitive that they still retain hind limbs with feet, not only corroborated the hypothesis that cetaceans were ungulates but also suggested that among living ungulates, cetaceans were closest to artiodactylans (the clade that includes hippopotamids, pigs, camels, ruminants and their relatives, Fig. 1A). This hypothesis emerged in part from the observation that these stem cetaceans exhibited the paraxonic foot posture that characterizes living and extinct artiodactylans.

Fig. 1. Cladograms depicting competing phylogenetic hypotheses for the position of cetaceans among ungulates. (A) traditional hypothesis of relationships with a monophyletic Artiodactyla containing two major clades and the extinct (†) clade Mesonychia is the sister taxon of cetaceans; also the hypothesis supported by data that fossilize (i.e., osteological and dental data, see O'Leary, 1999); and (B) the cladogram supported by molecular data (e.g., Gatesy et al., 1999a), “Artiodactyla” is paraphyletic. Molecular sequence data alone cannot inform on the position of the wholly extinct clade Mesonychia

Molecular sequence data collected in recent years have not only supported a close relationship between cetaceans and artiodactylans but have also defined an alternative phylogenetic hypothesis as well: that cetaceans are nested within Artiodactyla as the sister taxon of hippopotamids (Irwin and Árnason, 1994; Gatesy et al., 1996, 1999a, b; Gatesy, 1998; Gatesy and Arctander, 2000). This hypothesis runs counter to the traditional notion of artiodactylan monophyly with respect to other placental clades and the idea that within Artiodactyla there are two clades: 1) hippopotamids + pigs, peccaries and their relatives, and 2) camels + ruminants and their relatives. The molecule-based cladograms rearrange the internal structure of Artiodactyla (Fig. 1B) and suggest homoplasy of such structures as the ruminating gut (Gatesy and Arctander, 2000).

Total evidence

Cetacean phylogenetics intersects with debates about the best methods of reconstructing phylogenies. There are numerous philosophies about how this should be done; a review of these is well beyond the scope of this paper and currently very much a topic of debate. The method applied here, total evidence (Kluge, 1989) or simultaneous analysis (Nixon and Carpenter, 1996), relies exclusively on character congruence: all data are considered together in a simultaneous maximum parsimony analysis. Kluge and Wolfe (1993) explained why this method, as opposed to taxonomic congruence, in which a consensus tree is derived from two or more trees generated from separate data partitions (e.g., molecules and morphology), is more consistent with the logic of parsimony analysis. The results of character congruence and taxonomic congruence often differ greatly (Kluge and Wolfe, 1993).

An important corollary to arguments for total evidence is that data partitions (e.g., postcranial data, dentitions, gene “A,” retroposon “Q,” to name a few) used widely in systematics research, have not been shown to have any biological reality (Kluge and Wolfe, 1993). While data partitions are useful for bookkeeping purposes, it does not make sense to investigate such things as a “dental partition” or “a dental signal,” if we do not know a priori whether having a spatulate incisor correlates more with the anatomy of the premaxilla, some genetic information, or the third molar. Indeed Farris (1969, p. 374) articulated over thirty years ago that functional correlation and phylogenetic correlation are not the same thing stating that “even if we could find some way to measure … the “functional importance” of a character, it would remain to be demonstrated that the measure of functional importance was correlated with the utility of the character for purposes of cladistic inference.” Nonetheless it is not uncommon to see in modern cladistic analyses that investigators conflate the two. We need only look at the consistency index of most data partitions analyzed individually to see that it is virtually never equal to one, indicating mixed phylogenetic signal occurs within frequently designated partitions. Thus, from the standpoint of testing phylogenetic hypotheses with our observations, the total evidence approach makes few assumptions and treats each character as an independent test of relationships by examining all characters simultaneously. Gatesy et al. (1999b) and Gatesy and Arctander (2000) have found that hidden support, that is, support contributed by different data partitions, figures importantly in the determination of the most parsimonious tree.

A total evidence analysis of the position of Cetacea that combined molecular, soft tissue, and osteological data (O'Leary, 1999) indicated that the sister taxon of Cetacea remained unresolved and that artiodactylans, cetaceans and mesonychians formed a polytomy to the exclusion of perissodactyls. New molecular data from Gatesy et al. (1999a) can now be incorporated to test this result further.

Congruence of a phylogenetic hypothesis with the stratigraphic record provides some information about where our taxonomic sampling may be poorest. O'Leary and Uhen (1999) used the Manhattan Stratigraphic Measure (MSM) (Siddall, 1998) to assess congruence between temporal information and the phylogenetic hypothesis derived from morphological data (O'Leary and Geisler, 1999). They found that the hypothesis that a monophyletic Mesonychia is the sister taxon of Cetacea, and a monophyletic Artiodactyla is the extant sister taxon of Cetacea was significantly congruent with the stratigraphic record. They did not, however, apply the MSM to alternative topologies. Subsequent to that analysis, Pol and Norrel (2001) introduced the modified Manhattan Stratigraphic Measure (MSM*), which prohibits reversals in the age character, making the measure more sensitive to conflicts between temporal and topological information.

With a phylogeny constraining hypotheses of character transformation, optimization of characters on that phylogeny permits reconstruction of ancestors, as well as reconstruction of characters that are not preserved in certain terminal fossil taxa (e.g., both soft tissues and osteology). Here I add recently collected molecular data (Gatesy et al., 1999a) to the analysis of O'Leary (1999) and use the modified Manhattan Stratigraphic measure (Pol and Norrel, 2001) to test the congruence between the result and the stratigraphic record. I also discuss implications of the phylogenetic hypotheses for reconstructing character evolution, in particular, the reconstruction of soft tissue and behavioral characters in certain pivotal fossils.


Morphological (osteological and soft tissue) data were taken from O'Leary and Geisler (1999), and the ordering of certain multistate characters in that analysis was also applied here (all character numbers in this paper follow O'Leary and Geisler [1999]). Several new character codings were added to that matrix; they are: character 12 for Tapirus (state 0; AMNH [American Museum of Natural History] 2592); character 873 for Tursiops (state 0; AMNH 35421 and AMNH 35428); character 89 Agriochoerus (AMNH 38843), Sinonyx (state 0 IVPP [Institute of Vertebrate Paleontology and Paleoanthropology, Beijing] V10760), Tapirus (state 0 [AMNH 2592]), and Phenacodus (state 0 [Thewissen, 1990]). Data were combined with molecular data from the Gatesy et al. (1999a) “Whippo-2” data matrix, which included mitochondrial gene sequences for cytochrome b, 12S ribosomal (r) DNA, 16S rDNA, and nuclear gene sequences for β-casein exon 7 + intron 7, κ-casein exon 4, γ-fibrinogen exons 2–4 + introns 2–3, protamine P1 exons 1–2 + intron 1 + 5′-3′—noncoding regions, IRBP exon 1, vWF exon 28, α-lactalbumin exons 1–3 + introns 1–2. Data on retroposon insertions came from Shimamura et al. (1997).

The taxonomic sample was drawn from O'Leary and Geisler (1999): extinct and extant artiodactylans, perissodactylans and cetaceans as well as a number of extinct stem taxa including a number of primitive ungulates often referred to as “condylarths.” Contra O'Leary and Geisler (1999), however, Didelphis and Asioryctes were not included in the analysis because other cladistic studies (e.g., Rougier et al., 1998) indicate that these taxa are not at all closely related to the ingroup (artiodacylans, cetaceans, perissodactylans and their extinct relatives). Indeed these taxa are not even placental mammals, which means there are potentially several thousand taxa separating them from the ingroup. Leptictis, which is included here and is used to root the trees, is also most likely very distantly related to the ingroup, even within Eutheria (Novacek, 1986), and its impact on the phylogeny should be tested in future analyses by the inclusion of other placentals.

As in O'Leary (1999) several explicit assumptions of monophyly were made for certain taxa in which more than one genus was combined to make one operational taxonomic unit for the analysis. These taxa were: the artiodactylans Camelus + Lama; the extinct tapiroid Heptodon + Tapirus; and the odontocetes Tursiops + Physeter + Delphinapterus. The Whippo-2 matrix contains other extant taxa that have not yet been scored for morphological characters. Only extant taxa that overlapped in both matrices (i.e., that were scored for morphology), however, were included so as not to inflate the amount of missing data for osteological characters simply because these data have not yet been collected (as opposed to missing data occurring because a character is inapplicable or not yet found in a fossil specimen).

Parsimony analyses were performed in PAUP* 4.0b 8 (Swofford, 2001). All searches were heuristic with 10,000 replicates, using random addition and TBR branch swapping, maxtrees setting was 100 with instructions to increase by 100 once 100 was reached. If a taxon was coded as having multiple states for a particular character these data were treated as polymorphisms, and gaps were treated as missing data. Optimizations of certain characters were performed in MacClade, version 4 (Maddison and Maddison, 2000). All tree statistics reported are calculated in PAUP* 4.0b 8. Two searches are presented: one based on fossilized evidence alone and one based on a total evidence analysis.

The modified Manhattan Stratigraphic Measure (MSM*) (Pol and Norell, 2001), was calculated in PAUP* 4.0b 8 (Appendix 1) for all equally parsimonious trees and for all trees generated from an examination of the fossilized data only (numbers of replicates for the Lm = 10; the Lo = 10; and the significance test = 1,000).

Optimizations of certain characters onto trees generated from the total evidence analyses were performed in MacClade Ver. 4 (Maddison and Maddison, 2000) using both the ACCTRAN algorithm (which favors reversals over parallelisms when the choice is equally parsimonious) and the DELTRAN algorithm (which favors parallelisms over reversals) (Farris, 1970; Swofford and Maddison, 1987; see also Wiley et al., 1991). Several other methods of character optimization have been developed (see Omland, 1999 for a recent review) many of which require a priori knowledge of how a character evolves. Because evidence has not been presented to inform on how these characters evolve, ACCTRAN and DELTRAN are the most appropriate algorithms to use to reconstruct characters. Ancestral states or reconstructed terminal states found using both algorithms are presented as unequivocal. Figure 2 shows the difference between an equivocal optimization of character state in a terminal taxon that is a fossil. The character in question could be either an unpreserved osteological character or a soft tissue character.

Fig. 2. Optimization of missing data for fossils that are terminal taxa on a cladogram (e.g., skeletal data that have not been recovered or soft tissue, behavioral or molecular data that rarely, if ever, fossilize). Each example shows the optimization of the color black (squares represent observations), unknown in the fossil taxon. A) Unequivocal optimization of black for the fossil taxon using both ACCTRAN and DELTRAN optimization algorithms; B) equivocal optimization for the fossil taxon using either ACCTRAN or DELTRAN both of which reconstruct the ancestral node differently because the two taxa bracketing the fossil in question have different character states


The total evidence matrix contained 8,258 characters of which 5,815 are constant, 2,443 are variable but not parsimony informative and 1,485 are parsimony informative. In comparison to the total evidence analysis of O'Leary (1999) where the number of informative characters ranged from 819–1,034 (depending on the alignment), the current analysis contains between 451–666 more informative molecular characters from Gatesy et al. (1999a).

The combined data analysis resulted in 33 most parsimonious trees (length = 4020; CI = 0.752; RI = 0.578; RC = 0.435; HI = 0.261), the strict consensus of which shows a polytomy of artiodactylans, cetaceans, mesonychians and perissodactylans (Fig. 3). The total evidence strict consensus results are somewhat similar to those of O'Leary (1999) that included many fewer molecular data in that they indicate that the phylogeny is largely unresolved. In all most parsimonious trees cetaceans (extant and extinct) are nested inside artiodactylans.

Fig. 3. Strict consensus of 33 most parsimonious trees generated from simultaneous parsimony analysis of all characters (total evidence analysis). Extinct taxa in plain typeface, extant taxa in bold. “Artiodactyla” is in quotations because it cannot be described as monophyletic without including taxa not traditionally part of the clade (i.e., cetaceans). All taxa from Pakicetus through Balaenoptera are considered part of Cetacea (following McKenna and Bell, 1997; Geisler and Luo, 1998; O'Leary and Uhen, 1999) even though many of these taxa are stem taxa that fall outside the crown clade Cetacea (here represented only by Tursiops and Balaeonoptera)

The incongruence among equally parsimonious trees hinges in part on the unresolved position of Mesonychia, which falls both outside (Fig. 4A) and inside (Fig. 4B) of Artiodactyla. There are a number of other taxa that are unstable on the tree. These include: Archaeotherium, which is sometimes the sister taxon of cetaceans or part of a clade with hippopotamids as the sister taxon of cetaceans and Andrewsarchus; Elomeryx which is sometimes closely related to cetaceans, hippopotamids or suids; Andrewsarchus which is sometimes closely related to cetaceans or mesonychians; and Basilosaurus and Dorudon which compete either to be the sister taxon of extant Cetacea or a clade that forms the sister taxon to extant Cetacea. In short, there are a number of unstable taxa but the content of the clades Cetacea and Mesonychia (excluding Andrewsarchus [O'Leary, 1998]) (Fig. 3) remains stable in all equally parsimonious trees.

Fig. 4. (A) One of 33 most parsimonious trees generated in the total evidence analysis that shows Mesonychia outside of Artiodactyla and relatively distantly related to Cetacea; (B) a second most parsimonious tree from the same analysis showing Mesonychia inside of “Artiodactyla” as the sister taxon of Cetacea; (C) strict consensus of two trees generated from parsimony analysis of fossilized data only. Extinct taxa in plain typeface, extant taxa in bold. Artiodactyla is monophyletic and comprised of two clades containing extant forms as well as stem taxa, Mesonychia is the sister taxon of Cetacea and perissodactylans and their relatives fall outside this clade. Cetacea, Artiodactyla and Perissodactyla here again include stem taxa to the crown clade, not just the crown clade itself

Analysis of fossilized data alone, 116 characters of which 115 were parsimony informative, yields two most parsimonious trees (length = 494, CI = 0.401; RI = 0.706; RC = 0.283; HI = 0.686) that show a monophyletic Artiodactyla as the sister taxon of a mesonychian + cetacean clade, the strict consensus of these trees (Fig. 4C) yields a hypothesis congruent with traditional ideas of ungulate phylogeny (Fig. 1A).

The MSM* applied to each of the two trees that result from analysis of the fossilized data alone (Fig. 5) is 0.1987 and is significantly congruent with the stratigraphic record (p = 0.001). This corroborates results from O'Leary and Uhen (1999), that a tree consistent with the traditional notions of relationship is also congruent with the stratigraphic record. All 33 equally parsimonious trees that result from the total evidence analyses (Fig. 5A and B), however, which have MSM* values ranging from 0.1252–0.1641, are also significantly congruent with the stratigraphic record (P = 0.001). All these trees show cetaceans, and sometimes also mesonychians, inside of Artiodactyla. The tree that most closely resembles the traditional hypothesis of relationship (Fig. 5C) has the highest MSM score, indicating the greatest degree of congruence with temporal data; trees that place Mesonychia outside of Artiodactyla (Fig. 5A) have the second highest MSM* values and those placing Mesonychia inside Artiodactyla (Fig. 5B) as the cetacean sister taxon have the lowest MSM* values.

Fig. 5. Trees from Figure 4 plotted against the stratigraphic record. Extant taxa in bold, extinct taxa in plain type. Black dots indicate the first appearance of a taxon (dates from McKenna and Bell [1997]; see also O'Leary

Optimizations (Fig. 6) of three osteological characters that have been attributed functional significance, character #12: presence or absence of pachyostosis of the auditory bulla (e.g., Thewissen et al., 1996; Luo and Gingerich, 1999); character #87: presence or absence of elongate toothwear facets (O'Leary and Uhen, 1999); character #89: relative size of the sacro-iliac articulation (Thewissen et al., 1996; O'Leary and Uhen, 1999); and one soft tissue character (#121) have been generated using ACCTRAN and DELTRAN algorithms. As noted above these algorithms can reconstruct character states in terminal taxa and ancestral nodes. Figure 6 consists of pruned trees based on fossilized data alone (Fig. 6A); and two examples from among the 33 most parsimonious trees from the total evidence analysis.

Fig. 5. Continued. and Uhen [1999]). A and B are two of 33 most parsimonious trees resulting from the total evidence analysis. C reflects the traditional hypothesis of relationships (with artiodactylan monophyly) and is the strict consensus of two most parsimonious trees from the analysis of fossilized data alone

Fig. 6. Three different tree topologies (modified from Gatesy et al., 1996) for cetacean phylogeny with silhouettes of the bodies of extant taxa and preserved bones of fossil taxa. Distributions of four different characters on the tree; those in bold have actually been observed; those in italics are optimized (using both ACCTRAN and DELTRAN). In all trees the ancestral cetacean is reconstructed unambiguously as having a pachyostotic bulla, an elongate molar shear facet and a broad sacro-iliac articulation, the last of these is also reconstructed in the terminal taxon, Pakicetus, using optimization. Reconstruction of the soft tissue, hair, however, differs depending on the tree topology. In A where Artiodactyla is monophyletic, this optimization is ambiguous for stem cetaceans and for mesonychians, in B both mesonychians and stem cetaceans are reconstructed unambiguously as having no hair, and in C stem cetaceans are reconstructed as having no hair and mesonychians as having hair. The character state “narrow sacro-iliac joint” listed here as present in extant cetaceans was actually scored as a “?” in O'Leary and Geisler (1999). The “narrow sacro-iliac joint” occurs in stem taxa to extant whales and extant whales are highly transformed so as to make the character difficult to score. The information is shown this way for brevity and to indicate at the same time that Ambulocetus and Pakicetus differ from the morphology of most members of the cetacean clade

No pelvis is known for Pakicetus. However, if this taxon falls between Ambulocetus, which has a broad sacro-iliac contact (Thewissen unpublished data reported in O'Leary and Uhen, 1999) and other taxa (e.g., mesonychians or hippopotamids) which have the same condition, Pakicetus, and the ancestral cetacean (Fig. 6A–C), can be reconstructed as having a broad sacro-iliac articulation.

An elongate molar shear facet and the pachyostotic bulla also characterize the ancestor of cetaceans and can be scored directly in Pakicetus and other more highly nested cetaceans. This feature is not present in other closely related taxa. O'Leary and Uhen (1999) hypothesized that this character state is found in stem whales that engage in aquatic feeding because stomach contents are known in stem cetaceans that have these wear facets (e.g., Basilosaurus). Their hypothesis can be tested by the discovery of fossilized stomach contents for Pakicetus.

Optimization of soft tissue characters that rarely if ever fossilize present different problems than optimization of osteological characters which do fossilize. As explained by Witmer (1995) the former requires knowledge of extant taxa or fossil Lagerstätten that flank a fossil of interest (Fig. 2A). To ask the question, did taxa like mesonychians or stem whales have hair, sweat glands, sebaceous glands, internal testes, and nurse in water (Gatesy et al., 1996), we must initially investigate whether these characters optimize unequivocally for the extinct taxa. For example, whether a hippopotamid clade or Artiodactyla as a whole forms the other half of the extant phylogenetic bracket (Witmer, 1995) with the Tursiops/Balaenoptera clade results in very different optimizations (Fig. 6).

Hippopotamids share many soft tissue features with whales not found in other artiodactylans (as discussed in Gatesy et al., 1996; Gatesy and Arctander, 2000) such as those just noted above. Figure 6 shows three possible optimizations of the presence of hair, which incidentally has a similar optimization to several other soft tissue and behavioral characters, initially discussed by Gatesy et al. (1996, p. 961) as “aquatic characters”. Figure 6B (which positions mesonychians, cetaceans and artiodactylans as in Fig. 4A of Gatesy et al., 1996) shows mesonychians as more closely related to cetaceans than are hippopotamids, and hippopotamids as the closest extant taxon to cetaceans. Optimization of the soft tissue characters shared by hippopotamids and cetaceans indicate unequivocally that these characters are present in mesonychians and in stem cetaceans and were inherited from the common ancestor of this clade and hippopotamids. This is a different interpretation of Figure 4A from that of Gatesy et al. (1996), a tree that shows essentially the same topology. In Figure 6A (Gatesy et al., 1996, p. 4C) mesonychians and cetaceans are more closely related to each other than either clade is to any artiodactylan, and artiodactylans form a monophyletic clade. In this scenario optimization of the soft tissue characters is equivocal for mesonychians and for stem cetaceans because the bracketing taxa (cetaceans and hippopotamids) have opposing character states for these characters. Finally, Figure 6B shows mesonychians outside of an artiodactylan clade that includes cetaceans (Gatesy et al., 1996; Fig. 4B). Here soft tissues can be reconstructed unequivocally for both stem cetaceans and mesonychians. These characters are, however, reconstructed differently in each case; mesonychians do not have the “aquatic characters” like the absence of hair but stem cetaceans do.


Cetacean evolution is one of several transitions across the land-water barrier documented in the history of vertebrate evolution (Carroll, 1997). Through the fossil record and the living biota we can observe the hypothesized extremes of the continuum: a pig-sized, four-footed, mesonychid or artiodactylan, and an immense, hairless baleen or toothed whale with no functional hind limbs. These extremes beg the questions: when did such a transition happen, how long did it take, what character systems changed first, what did the intermediate forms look like, how did they move, what did they eat, and how were they distributed throughout the globe? Answers to these questions rely on having a highly corroborated hypothesis of phylogenetic relationships that becomes the building blocks or pattern upon which we base specific hypotheses of descent with modification. The diagnosis of clades through synapomorphy becomes the starting point for the investigation of functional, temporal, adaptive and biogeographic questions.

The position of cetaceans among mammals is currently unresolved. Many types of data still need to be collected for taxa relevant to this problem including osteology, soft tissues, behavior and molecular sequences. All 33 most parsimonious trees from the total evidence analysis indicate that among living taxa, artiodactylans are the closest relatives of cetaceans. This is not upheld in the strict consensus because of the instability in the positions of certain fossil taxa, in particular, the extinct clade Mesonychia.

In the total evidence analysis crown clade cetaceans, that is, the clade formed by the ancestor of living cetaceans and all of its descendants, are associated in all trees with a number of fossil stem taxa, from Pakicetus through Basilosaurus. These taxa are considered to be whales (McKenna and Bell, 1997), albeit primitive forms that left no living descendants. All 33 most parsimonious trees from the total evidence analysis also show cetaceans nested within Artiodactyla and are essentially congruent with the molecule-based topology of Gatesy et al. (1999a) in terms of the relative positions of the extant taxa. Ultimate resolution of the problem of the position of Cetacea with additional data does not necessarily mean that one of the 33 most parsimonious trees from the total evidence analysis will necessarily be upheld: it could be an entirely different tree.

Results presented here and in Gatesy and O'Leary (2001) argue against the recent comment that “artiodactyl (including whale) relationships are the best-resolved portion of the mammalian tree” and that the support for the tree is so strong that many mammalogists now consider this a “ ‘virtually known’ phylogeny” one that is “so well supported by multiple analyses that no reasonable person would question [its] resolution” (Hillis, 1999, p. 9979). This conclusion is clearly premature based on evidence presented here and summarized elsewhere (Gatesy and O'Leary, 2001). Furthermore, even if the phylogeny of this clade becomes better established, many systematists remain uncomfortable with the notion of using stable phylogenies as a means of testing methods.

It is critical to temper haste for answers with recognition of how few data, both taxa and characters, have been collected pertaining to the question of the phylogenetic position of Cetacea. This may be surprising given the long history of comparative anatomy and paleontology. Cladistic methods have, however, only been available for about 35 years (Hennig, 1966) and have only been in widespread use with the assistance of computer-based algorithms for half that long. Although morphologists have made inferences about ancestry and phylogeny for centuries, they have not worked with modern methods for any longer than molecular biologists have. For all genera of mammals we are only beginning to see the emergence of detailed databases to test (and often corroborate) with parsimony analysis some traditional hypotheses of relationship.

The cetacean phylogeny example helps outline the way large blocks of missing data develop in total evidence analyses of clades that have both a good fossil record and numerous extant members. Taxa historically thought to be most relevant to phylogenetic analysis of Cetacea (O'Leary and Geisler, 1999; Gatesy and O'Leary, 2001) are: all extinct and extant cetaceans, artiodactylans and perissodactylans, and a number of extinct ungulate clades (e.g., mesonychians, triisodontines). A very conservative estimate (i.e., a count of genera listed in a recent classification [McKenna and Bell, 1997]) indicates that the ratio of extant:extinct taxa is 1:9 (O'Leary and Geisler, 1999). Figure 7 indicates roughly which data have been collected so far for cetacean phylogeny.

Fig. 7. Cartoon of the universe of character data available and/or collected pertaining to the phylogenetic position of cetaceans among mammals. Gray blocks indicate data of a given type that have already been collected. White areas indicate data that could be collected but that have not been collected, and black block indicates data that may never be collected because they do not fossilize. The latter category currently describes the majority of cells (O'Leary, 1999). The gray box in the category of behavior for an extinct taxon refers to the preservation of stomach contents in extinct Cetacea. This figure does not show inapplicable characters, which would increase the amount of missing data even further and would affect all different types of data. Ancient DNA from extinct organisms has yet to be collected or incorporated into phylogenetic analyses of the position of cetaceans. The figure is not to scale in terms of the relative numbers of molecular characters to other types of characters

Given the great extinction in this clade a very large block of data, arguably the majority of cells, can probably never be scored at all because they do not fossilize. No ancient DNA has yet been used for cetacean systematics. Amounts of missing data can run as high as 75% of cells in a matrix in combined analyses (O'Leary, 1999). Discussion of this phenomenon has usually been phrased in terms of incomplete taxa (i.e., fossils) but could equally fairly be described in terms of incomplete characters (i.e., those that do not fossilize). This cartoon (Fig. 7) emphasizes the very regular pattern of missing data that results from extinction (and the subsequent selectivity of fossilization). The large block of missing data would not, however, be a reason against conducting total evidence analyses.

The densest samples of extant taxa (Gatesy et al., 1999a) and extinct taxa (O'Leary and Geisler, 1999; Luo and Gingerich, 1999) do not yet approach a comprehensive sampling of the clade, and indeed even these published analyses are still in the process of being combined. Fossilized (largely osteological) characters remaining to be collected include those describing vertebral, carpal and tarsal morphology. Other improvements include better taxonomic representation for the characters already described. Basicranial characters, although particularly well-documented by Luo and Gingerich (1999), need to be scored for many other taxa, in particular, artiodactylans. Because morphology includes much more than osteology it is important that other data sets such as that of Langer (2001), which codify data on the histology of the digestive system in these taxa, be combined with other data to assess their overall contribution to this question. Some soft tissue characters of interest not yet extensively explored for systematics include: musculature, physiology, neurology, and embryology. Finally, arguments that behavioral data are fundamentally no different from other kinds of anatomical or molecular data used for systematics (De Queiroz and Wimberger, 1993) should also impact cetacean phylogenetics. Gatesy et al. (1996) suggest that bioacoustical data in particular could also be exploited for cetacean systematics.


Application of the modified Manhattan Stratigraphic Measure (MSM*) to all of the equally parsimonious trees generated by the total evidence analysis and to two trees derived from an analysis of fossilized data alone indicates that all of the trees are significantly congruent with the stratigraphic record. The hypothesis that agrees with traditional ideas of relationship (i.e., a monophyletic Artiodactyla) has a slightly higher MSM* value, indicating slightly better agreement with the stratigraphic record. The significance test, however, adds an important perspective that all the competing hypotheses are relatively congruent with temporal data. Gatesy et al. (1996) commented that the tree that nests both mesonychians and cetaceans within Artiodactyla (here tree 6B) is the hypothesis that promotes the greatest amount of conflict with the stratigraphic record. The MSM* values confirm this but also show that the increase in incongruence does not cause this topology to clash significantly with information on divergence times.

The divergence of cetaceans from other placental mammals is hypothesized to have occurred in the Paleocene or Eocene using ghost lineage estimates (O'Leary and Geisler, 1999; O'Leary and Uhen, 1999; Gatesy and O'Leary, 2001), the Eocene, using likelihood-based estimates (Gingerich and Uhen, 1998); or the Cretaceous, using molecular clocks (Kumar and Hedges, 1998). The data presented here are consistent with an Early Tertiary origin of Cetacea.


Phylogenetic hypotheses form the framework for asking questions about adaptation as outlined by Lauder (1996). Atomization and study of the origin of different characters in any clade of organisms is the starting point for the interpretation of whether or not that character may have been acquired by selection. To this end we should ultimately aim to incorporate as many different types of data and taxa as possible in phylogeny reconstruction. Such accumulations of data allow reconstruction of soft tissues and behaviors in fossil taxa through optimization. These reconstructions also allow us to investigate whether a structure and a soft tissue are associated in all cases, an observation that is minimally important for arguing that a structure necessarily implies a given soft tissue or behavior.

Optimization of several different osteological characters onto different equally parsimonious trees generated in the total evidence analysis becomes a basis for reconstructing the ancestral whale. O'Leary and Uhen (1999) discussed that although the sacro-iliac articulation is not preserved in taxa like Pakicetus, due to the stable phylogenetic position of the earliest cetaceans relative to extant whales (i.e., they are in the same position in the strict consensus tree), optimization indicates that they had a sacro-iliac articulation equivalent in relative breadth to that of many terrestrial quadrupedal mammals. Directly observable from fossils, the ancestral cetacean also had a pachyostotic bulla and elongate molar shear facets.

The paleontological literature is replete with functional inference from structure or Lauder's (1996) “Argument from Design.” Some of these inferences are so second nature or reasonable that they are rarely questioned such as whether a torpedo-shaped body in a vertebrate indicated anything other than an aquatic lifestyle. These inferences, however, often occur in situations where an optimization is ambiguous (e.g., O'Leary and Uhen, 1999 on terrestrial quadrupedalism or aquatic predation). Nonetheless no one has observed swimming in an animal like Basilosaurus, and no Lagerstätten exists as testimony to it. It is important to recognize when such statements are 1) inferences from design or environment and/or 2) are explicit a priori character correlation arguments because not all such inferences in fossils are untestable using cladistics.

The functional inference from the pelvis character just mentioned for Cetacea is that basal whales were capable of terrestrial quadrupedal locomotion because a joint of this size is present in many extant mammals with that capacity and tendency (Thewissen et al., 1996; O'Leary and Uhen, 1999). At present, however, this behavioral character is an ambiguous optimization. It is a behavioral hypothesis that ultimately can be tested through ichnofossils: if footprints of a pakicetid or ambulocetid were found this discovery would test the hypothesis with cladistic character data. Inclusion of such data for Ambulocetus and Pakicetus would change the optimization to unequivocal terrestrial quadrupedal locomotion.

Stem cetaceans also possess an elongate molar toothwear facet not present in closely related taxa outside the clade. O'Leary and Uhen (1999) argued that this facet suggests aquatic predation because it is also present in fossil taxa that have stomach contents preserved containing many fish. The optimization of the character, presence of aquatic predation, would be equivocal on trees presented here because in all cases taxa bracketing stem cetaceans would lead to an equivocal optimization. Like the hypothesis about locomoter behavior just described, this hypothesis about dietary behavior is also testable by the recovery of stomach contents for an ambulocetid or a pakicetid.

Finally, the hypothesis presented here that ambulocetids and pakicetids are, under some topologies, unambiguously reconstructed as having hair, is also testable given that occurrences of fossilized hair (Meng and Wyss, 1997) and skin (Chiappe et al., 1998) have been reported in fossils from sediments even older than those that yield ancient whales.

The hypothesis that pachyostosis of the auditory bulla occurs as an adaptation to processing sound in a dense medium (water), is widely considered to be reasonable (Thewissen et al., 1996; Luo, 1998; Luo and Gingerich, 1999). This inference is similar to inferences made about the bony hearing apparatus in fossil bats and whether the presence of particular bony characters is indicative of an ability to echolocate (Simmons and Geisler, 1998). In both bats and cetaceans optimizations for the behaviors themselves are ambiguous for many fossil stem taxa of interest. No one has suggested what a cladistic test of this hypothesis would be for fossil taxa. Without such a test we are left with character correlation argument only or an inference from design or environment based on the characteristics of the organisms that happen to be alive. While questions about stem taxa to major clades are often some of the most interesting, they can expose areas where it is very difficult to test functional and behavioral inferences in fossils with cladistic character data.

Some may argue that the geological context of a fossil is also important information that should factor into tests of fossilized behavior. If a species is recovered from near shore marine rocks we should infer that it is semi-aquatic. If it is found in deep ocean rocks then we should infer that is was fully aquatic. Like the presence of wings in extant animals that do not fly, several Early Tertiary taxa, which are not typically considered to be anything other than terrestrial have been found in marine or near shore marine sediments (Rose, 1999, 2000), to name one example of incongruence between an inference of behavior and geological context. What are we to make of these exceptions? When do we decide that geology can tell us behavior and when do we ignore it as confounding? Such questions demand further investigation, which will hopefully be aided by initial examinations of cladistic optimizations of behavior and soft tissues in fossils.


I thank D. Swiderski for the invitation to contribute this article to the SICB symposium on cladistics and optimization. This research was supported by NSF Grant #DEB-9903964. For helpful criticism I thank J. Gatesy and D. Pol. J. Geisler and one anonymous person reviewed the manuscript.


  • 1 From the Symposium Beyond Reconstruction: Using Phylogenies to Test Hypotheses About Vertebrate Evolution presented at the Annual Meeting of the Society for Integrative and Comparative Biology, 4–8 January 2000, at Atlanta, Georgia.

  • 2 E-mail: moleary{at}mail.som.sunysb.edu

  • 3 Although Andrewsarchus has been synonymized with Paratriisodon (see Van Valen, 1978; O'Leary, 1998) illustrations of the lower dentition of Paratriisodon (Chow, 1959) are inadequate to allow assessment of character 87 and the original in the Institute of Vertebrate Paleontology and Paleoanthropology, Beijing has not been examined therefore this character is scored as “?” here.


View this table:
View this table:
View Abstract