Criteria for greatness in a western movie:

• Setting:  It almost goes without saying that a western is set in the American west, almost always during the period of greatest western expansion and settlement (around 1840 to 1890).  But that isn’t invariably true.  Last of the Mohicans, for example, is set in the east, in upstate New York, when that was on the frontier of white settlement.  The same is true of Drums Along the Mohawk.  There have been westerns set in the far north (The Spoilers and North to Alaska), even in South America (Savage Pampas, Way of a Gaucho) and Australia (Quigley Down Under and The Man from Snowy River).  There are many westerns set during the Civil War, which took place almost entirely east of the Mississippi River.  There is a subgenre of westerns set in the modern west but dealing to some degree with traditional western themes (Bad Day at Black Rock, Lone Star, No Country for Old Men, Wind River).  Some science fiction and samurai movies seem to be westerns.  It is one of several criteria, and it has some flexibility.

• Use of traditional western themes:  Similarly, there is a wide range of western themes, many of which expand into universal human themes such as the search for family in an unfriendly world.  They include clashes of races (usually whites and Native Americans), clashes between social and commercial objectives (law vs. anarchy and unrestrained self-interest, ranchers vs. farmers, for example, or cattlemen vs. sheepmen), man against nature, self-reliance and the place of violence in society.  They are too numerous to make any comprehensive list.  At times “a western theme” almost seems like Justice Potter Stewart’s elusive definition of pornography:  “I know it when I see it.”
• Story and performances:  It’s hard to have a great western without a good story and strong performances.  As with any other cinematic genre, directors can sometimes get distracted by the magic of movie-making at the expense of story.  (Here’s looking at you, Sergio Leone.)  While that can lead to some interesting movies, they’re not generally as strong as those where story is more at the front.
• Visual sweep and impact:  Good use of the American west’s big skies, open plains, towering mountains and general natural splendor is a hallmark of great westerns, such as Stagecoach, My Darling Clementine, Red River and, more recently, Jeremiah Johnson and The Horse Whisperer.  Westerns offer a unique scope for the physical background to become another character.  In fact, the west’s visual magnificence can become distracting if the director isn’t careful.  But there have been great westerns shot on back lots in Hollywood that make virtually no use of traditional western scenery beyond a western town set—High Noon, for example.

• Re-watchability:  In a way, this is the Cimarron vs. Stagecoach question.  Is this something you welcome the opportunity to see again, even though you’ve already seen it before?  Yes for Stagecoach; not so much for Cimarron.  However, this question is not determinative—it’s one of several.  Some of the great westerns one just doesn’t go back to as often as others, and that doesn’t mean they’re not great.  Unforgiven and Dances With Wolves are great westerns, but once seen one might not re-watch them as often as, say, Rio Bravo or The Outlaw Josey Wales.
• Film-making skills for its time:  Most movies are inseparable from the time they were made.  The state of movie-making technology, the social attitudes, the fashion and actors’ makeup, the approach to story and cliché—all these and many other things speak of the time a movie was made.  You probably wouldn’t confuse 1939’s Stagecoach with a movie made only eleven years later in 1950, although black-and-white was still common for westerns in 1950.  Is the director making good use of what’s available to him at the time he made the movie?  If the direction isn’t remarkable in any way, is there something else that’s carrying the movie so you don’t think about it much?
• Impact on the genre in its time:  How was the movie received upon its original release?  How are westerns different (if at all) because of this movie?  Unless you’re interested in film history, this might only be a subtext, and a subconscious one at that.  Sometimes you can even forgive a clunky story if a movie changed the course of what was made later.  That’s one reason to watch some of the influential silent movies, like The Iron Horse or the films of William S. Hart.  If it is just influential and not of continuing greatness on a number of levels, however, you won’t tend to go back to it.  Sometimes it is the body of a director’s work that seems to shift the course of westerns made after.  Think, for example, of the films of John Ford, Anthony Mann and Sergio Leone.
• How does it age?  (The Test of Time):  Is it still a good and enjoyable film notwithstanding its age or the time at which it was made?  Cinematic and societal fashions change.  Although one can’t really separate a film from the time it was made, is there something that pulls you back to it?

• Historicity—How true is it to its supposed time?:  This one is tricky, in part because standards of historicity change.  In 1939, for example, historicity was all but ignored in the service of what was thought to constitute a good story.  Jesse James is much more interested in perpetuating the outlaw’s myth and getting people into theaters than it is in telling the actual story.  Compare it with The Long Riders, Walter Hill’s 1980 retelling of the story of the James Gang, which has a much stronger historical sense.  My Darling Clementine, however, works magnificently as a film even though it’s a very garbled retelling of the Wyatt Earp myth that worries very little about what actually happened and feels free to toss in invented characters, such as the titular Clementine.  In westerns from the 1950s, western clothes (hats in particular) and women’s garb tend not to be very accurate historically.  Recent decades (since the 1980s, perhaps) seem more concerned with historical accuracy, although those financing the movie will seldom let the actual facts get in the way of the movie they want to make.

• Good introduction of non-traditional elements? Sometimes the ability to think beyond traditional western elements broadens the scope of the movie.  Sometimes it’s just distractingly anachronistic.  Some consider McCabe and Mrs. Miller a great western because it approached some subjects (such as prostitution) in a different way than previous movies set in the west had.  It is very much a movie of its own time (the anti-authoritarian early 1970s, with a grainy-ish look) and its director (Robert Altman, who made only one other western even more explicity revisionist).  The nihilism and explicit violence of The Wild Bunch (combined, of course, with terrific performances and stunning cinematography) have affected every western made since, although they were mostly new to westerns when the movie was made.
• Use of supporting elements (musical score, writing, supporting performances, cinematography, etc.):  Movie-making is such a collaborative art that it’s kind of amazing when it all comes together to produce a great film.  That’s as true of westerns as it is of any other kind of movie.  Good writing (think Butch Cassidy and The Sundance Kid, for example) produces lines that the movie fan mutters to himself as he watches the film again; writing is always more important than movie makers (especially the financial backers and the studio) think.  Music undeniably influences one’s memory of the film (the Tex Ritter version of the theme from High Noon, Dmitri Tiomkin’s “Deguello” from Rio Bravo, Bruce Broughton’s stirring theme from Silverado, and Basil Pouledoris’ rollicking theme from Quigley Down Under all come to mind), and become inseparable from the experience of that movie.  Sometimes the statement is made by not having music, as in Westward the Women.  John Wayne’s bravura performance as the Ringo Kid in Stagecoach wouldn’t have been as effective without the talented ensemble of supporting actors (Claire Trevor, Thomas Mitchell, John Carradine, Donald Meek, George Bancroft and others).  In fact, Wayne is more one of the ensemble than the star.  Wayne and James Stewart are giants in The Man Who Shot Liberty Valance, but it wouldn’t have worked as well without the despicable malevolence of Lee Marvin as Liberty Valance.  You may or may not overtly notice the brilliant cinematography in Red River, She Wore a Yellow Ribbon, The Wild Bunch or Jeremiah Johnson, but it’s part of what makes you think you’ve seen a great western.