In the tradition of the last few centuries at least, public papal Masses were much fewer and further between by comparison with present times. A solemn papal Mass was a liturgical rarity, offered only on the most solemn occasions int he life of the Church (such as Christmas, Easter, the feast of Ss. Peter and Paul and the canonizations of new saints for example) no doubt in part due to the practical ceremonial requirements that came with organizing one of them, and possibly informed too by other considerations such as a general concern for the usually advanced age of a Roman pontiff and the toll such ceremonies might take on his health, and perhaps too it was intended to add increased gravitas to those solemn occasions when the pope did indeed publicly celebrate (for it is certainly true that the more regular and routine something becomes, the more our human nature tends to devalue it).
 |
| Easter |
 |
| Solemn Papal Mass of the Canonization of the Ugandan Martyrs |
Outside of the solemn papal Mass proper, the pope also participated more semi-privately/publicly in Masses held in the Sistine chapel -- which you might consider to be something like a "parish church" of the Vatican where the pope would gather with the papal court and a select number of guests. Here, the popes were not the primary celebrant strictly speaking. The popes would participate in what was called the "Missa coram Summo Pontifice" -- a Pontifical Mass offered in the presence of the Pope. This form of Mass would see the Roman pontiff, vested in mantum (a longer-trained cope) recite certain introductory prayers with the cardinal celebrant at the foot of the altar, but for the most part the pontiff would 'hear Mass' from the papal throne -- effectively attending in choro. It was in contexts such as these that solemn celebrations such as those of Holy Week were offered. The mantum indeed only came in red and white.
So does this mean traditionally popes barely ever said Mass in recent centuries? Not at all. Popes would also offer private papal Low Masses. For these occasions, the popes would vest in whatever the liturgical colour was called for according to the rubrics of the particular Mass. Evidently, these occasions, more private and small by nature and they were rarely seen. However, in modern times we do have some photos of such occasions, as well as photos of some of the papal chasubles used for them.
 |
| Pope John XXIII celebrating a papal Low Mass, wearing a violet chasuble. |
Here is the a violet chasuble seen in use by Pope Paul VI within the context of a private, papal Low Mass. It was originally made for Pope Pius IX and used the same design for three liturgical colours: red, rose and violet as seen here:
Another well documented example is this rose chasuble of Pope Pius XI:
And a green chasuble gifted to Pius XII:
So then, one might say that if there was a liturgical colour that wasn't worn by the pope himself, at least in 'modern' times (i.e. the past few centuries) it would be black since red was the pope's colour of mourning. However on this point I'd be quick to follow up that we should not make yet another faulty deduction and thus assume from everyone else should dispense with black as well. Black is of long standing liturgical use and was, in point of fact, considered one of the principal liturgical colours even prior to violet; a point testified to by Pope Innocent III (+ 1216) and at one times Roman pontiffs did indeed wear this colour.
At any rate, circling back to the main point, while it is not strictly correct to say that popes traditionally only wore red and white vestments, it would be accurate to say is that these were the liturgical colours they happened to generally be seen in by the general Catholic populace.
