In Dragonball Z, the Ultra Super Saiya-jin (A.K.A. Ultra Super Saiyan, though it is never actually called this in the English dub) is a level of Super Saiyan beyond the Super Saiyan; but it is not to be confused with the truly Ascended Super Saiyan, who can maintain his Super Saiyan power without the strain of actually being "powered up," or Super Saiyan 2, in which the fighter gains strength and speed by a huge exponential factor and has blue electricity arcing around his body. I know, sounds silly, but...
It is characterized by an immense increase in
size and
strength, with the
muscles of all parts of the
Saiyan's body bulging massively. However, the
fatal flaw in this
incarnation is the
extreme decrease in the fighter's speed.
Trunks discovers the weakness of the Ultra Super Saiyan the hard way during his battle with Perfect Cell. The fight had been going moderately well, at least not disastrous, and according to Trunks' plan, his new powerful form that, as he had observed, his father could not attain, would enable him to defeat Cell.
Almost tragically, Trunks did not foresee that the USSJ form would be as slow as it was, and Cell dashingly and cleverly points out to Trunks that, while USSJ Trunks may have the strength to tear him apart, he will never actually be able to touch his superlatively swift self.
Trunks later has the stunning revelation that his father must have been able to reach USSJ, but that he refrained from doing so and so was defeated because he knew that going USSJ would be an exercise in futility against Cell. We never actually get confirmation from Vegeta on this, but it is nevertheless believable that Toriyama envisioned that Vegeta actually did experience this thought process.
The likelihood that Vegeta knew of the USSJ is increased by the fact that we are allowed to see Son Goku himself display the USSJ form he independently discovered to Son Gohan while in The Room of Spirit and Time; Goku immediately recognizes the debilitating defect of the transformation and informs Gohan that it would be wise to skip it altogether. Goku and Gohan move on in their training and eventually become truly Ascended Super Saiyans.
By the way, I found out that the form described heretofore was called Ultra Super Saiya-jin from the now defunct DB/Z/GT fansite Planetnamek.com. I believe the term was introduced in one of the Japanese Daizenshuu, or the informative books about the series.