Russell Laroy Holland appeals his conviction and sentence for mailing threatening communications and threatening the President of the United States. He maintains that the district court judge who imposed the sentence should have recused himself after Holland obtained the judge’s home telephone number and left at least one threatening message prior to his sentencing. We hold that the district judge reasonably construed Holland’s threatening phone message as an attempt to manipulate the court system which did not warrant his sua sponte recusal. Accordingly, we affirm the judgment.
I
Russell Laroy Holland pled guilty to one count each under 18 U.S.C. § 876 and 18 U.S.C. § 871 for mailing threatening communications and threatening the President *1122of the United States. He agreed to plead guilty in exchange for the Government dismissing the other four counts of the indictment. On April 13, 2006, Appellant was sentenced to seventy-eight (78) months incarceration followed by three (3) years of supervised release, to begin running after Holland served his current state sentence. Holland is projected to be released from Federal custody sometime during the year 2012, when he will be 51 years old.
Holland has a lengthy criminal record and has been incarcerated for most of his adult life for crimes ranging from armed robbery, petty theft, escape from prison, assault by a prisoner, grand theft, robbery, prisoner possession of a weapon, assault with a deadly weapon, threats against state officials, and injury to jails. At some point during the proceedings in this case, Holland obtained the sentencing judge’s home telephone number, which he called, leaving more than one threatening message on an answering machine. Before sentencing, the district court revealed that he had received these threatening messages from Holland. The district court, however, dismissed them as attempts to “manipulate the system.” Specifically, the judge remarked:
For the record, too, the Court is also aware that Mr. Holland did, in fact, leave voice messages at the Court’s personal residence that to some people could be construed as threatening. However, in my judgment, Mr. Holland’s history clearly demonstrates that these are attempts to manipulate the criminal justice system rather than threats as such and the Court is just simply not going to allow Mr. Holland to manipulate the system. So the Court has chosen to go forward with this sentencing at this time.
Later in the proceeding, the judge observed that Holland had a history of violent and assaultive crimes and “if he is given the opportunity, he has the ability to carry out his threats.... [I]t is clearly important ... that the Court imposef ] a sentence for the protection of society.” Holland did not object to the sentencing judge’s decision to proceed with the hearing or request that the judge recuse himself.
II
We are confronted with a narrow question: When does a judge have an obligation under 28 U.S.C. § 455 to recuse himself sua sponte in response to threats made against him, his family members or associates? Because the issue was not raised before the trial court, we review for plain error. Jones v. United States, 527 U.S. 373, 388, 119 S.Ct. 2090, 144 L.Ed.2d 370 (1999); United States v. Olano, 507 U.S. 725, 736, 113 S.Ct. 1770, 123 L.Ed.2d 508 (1993); United States v. Ortiz, 362 F.3d 1274, 1278 (9th Cir.2004).1 In Noli v. Comm’r of Internal Revenue, 860 F.2d 1521, 1527 (9th Cir.1988), we held that “[fjailure to move for recusal at the trial level ... does not preclude raising on appeal the issue of recusal under § 455.” Id. “ ‘Nonetheless, if no motion is made to the [trial court] judge ... a party will bear a greater burden on appeal in demonstrating that the judge ... [erred] in failing to grant recusal under section 455.’ ” Id. (quoting United States v. Sibla, 624 F.2d 864, 868 (9th Cir.1980)); see also Pau v. Yosemite Park and Curry Co., 928 F.2d 880, 885(9th Cir.1991).
We begin with the general proposition that, in the absence of a legitimate reason to recuse himself, a judge has a duty to sit in judgment in all cases coming before him. See Laird v. Tatum, 409 U.S. 824, 837, 93 S.Ct. 7, 34 L.Ed.2d 50 (1972); Sensley v. Albritton, 385 F.3d 591, 598-99 (5th Cir.2004); United States v. Snyder, 235 F.3d 42, 46 (1st Cir.2000); Nichols v. Alley, 71 F.3d 347, 351 (10th Cir.1995). The duty inheres in the “judicial Power” with which we are vested. See U.S. Const. art III, § 1. It is reflected in our oath, by which we have obligated ourselves to “faithfully and impartially discharge and perform [our] duties” and to “administer justice without respect to persons, and do equal right to the poor and to the rich.” 28 U.S.C. § 453. Without a duty to sit, we could recuse ourselves for any reason or no reason at all; we could pick and chose our cases, abandoning those that we find difficult, distasteful, inconvenient or just plain boring. Our mythic Justice, represented by a blindfolded figure wielding a balance and a sword, hears all cases coming before her, giving no preference— whether in priority or result — to the station or economic status of such persons. We are duty-bound to sit in all cases — not just the ones we would like to hear— except those in which our “impartiality might reasonably be questioned.” 28 U.S.C. § 455(a); see also id. § 455(b)(enu-merating circumstances requiring recusal).
We also recognize that the security of the nation’s judiciary is a serious concern. We are, unfortunately, reminded from time to time that threats against the judiciary have been carried out.2 We stress that, despite the duty to sit, if a judge feels that his personal safety or the safety of his family is in danger, he may always recuse himself sua sponte from a matter. Recusal in such situations is left to the discretion of the threatened judge, who is in the best position to evaluate the advisability of recusal; we generally do not review a judge’s decision to recuse himself.
The question before us, however, is different. Here we must decide when a judge must recuse himself sua sponte in response to threats even if he would prefer to continue his work on the case. Here, Holland contends that under 28 U.S.C. § 455, the district judge should have re-*1124cused himself sua sponte after receiving his threatening phone calls.
Section 455(a) of Title 28 reads: “Any justice, judge or magistrate judge of the United States shall disqualify himself in any proceeding in which his impartiality might reasonably be questioned.” In 1974 Congress amended the statute to replace the subjective test of the previous version of the statute with an objective test based on public perception.3 The Supreme Court discussed this change in Liljeberg v. Health Servs. Acquisition Corp., explaining that, “The goal of section 455(a) is to avoid even the appearance of partiality. If it would appear to a reasonable person that a judge has knowledge of facts that would give him an interest in the litigation then an appearance of partiality is created even though no actual partiality exists.” 486 U.S. 847, 860, 108 S.Ct. 2194, 100 L.Ed.2d 855 (1988) (internal quotation marks omitted). We have restated § 455(a) and ask “whether a reasonable person with knowledge of all the facts would conclude that the judge’s impartiality might reasonably be questioned.” Clemens v. U.S. Dist. Ct., 428 F.3d 1175, 1178 (9th Cir.2005) (internal quotation marks and citation omitted); cf. United States v. Cooley, 1 F.3d 985, 993 (10th Cir.1993) (articulating a similar standard). “Section 455(a) asks whether a reasonable person perceives a significant risk that the judge will resolve the case on a basis other than the merits.” In re Mason, 916 F.2d 384, 385 (7th Cir.1990). The “reasonable person” is not someone who is “hypersensitive or unduly suspicious,” but rather is a “well-informed, thoughtful observer.” Id. at 386. The standard “must not be so broadly construed that it becomes, in effect, presumptive, so that recusal is mandated upon the merest unsubstantiated suggestion of personal bias or prejudice.” Cooley, 1 F.3d at 993.
Disqualification under § 455(a) is necessarily fact-driven and may turn on subtleties in the particular case. Consequently, “the analysis of a particular section 455(a) claim must be guided, not by comparison to similar situations addressed by prior jurisprudence, but rather by an independent examination of the unique facts and circumstances of the particular claim at issue.” United States v. Bremers, 195 F.3d 221, 226 (5th Cir.1999); see also Clemens, 428 F.3d at 1178. In general, the conscientious judge should also bear in mind that § 455 is limited by the “extrajudicial source” factor which generally requires as the basis for recusal something other than rulings, opinions formed or statements made by the judge during the course of trial. Liteky v. United States, 510 U.S. 540, 554-56, 114 S.Ct. 1147, 127 L.Ed.2d 474 (1994). Put differently, the judge’s conduct during the proceedings should not, except in the “rarest of circumstances”4 form the sole basis for recusal *1125under § 455(a). Id. at 555, 114 S.Ct. 1147.5
B
Applying these general principles to situations where the judge receives a threat, suggests a three-step process for evaluating whether recusal is required under § 455. First, the judge must evaluate the threat itself to determine how much risk there is that it may be carried out and how much harm there would be if it were. In deciding whether recusal is appropriate, the judge may wish to consider the following factors:
(1) The defendant’s capacity to carry out the threat Has the defendant taken concrete steps to carry out the threat? Does the defendant have a history of violence or has he previously been successful in carrying out other threats? Is he a member of a gang or does he have accomplices or contacts who could carry out the threat on his behalf?
(2) The defendant’s demeanor and the context of the threat. Was the threat made in a fit of passion or intended as a joke? Does the judge believe that the defendant is serious in carrying out the threat? Does the judge have any prior dealings with the defendant that make the threat more or less likely to be carried out?
(3)The perceived purpose of the threat. Was the threat made in open court or did the judge become aware of the threat only through the fortuity of a law enforcement investigation? Does the judge believe it is an attempt to force recusal and manipulate the judicial system?
The final factor is, perhaps, the most important. Not every threat made against a judge should force recusal. If so, defendants could readily manipulate the system, threatening every jurist assigned on the “wheel” until the defendant gets a judge he preferred. Also, the defendant could force delays, perhaps making the cases against him more difficult to try, perhaps putting witnesses at greater risk. Such blatant manipulation would subvert our processes, undermine our notions of fair play and justice, and damage the public’s perception of the judiciary.6 We agree with the Eighth Circuit that “recusal is not *1126automatic on the mere basis of the judge’s knowledge of the threat.” United States v. Gamboa, 439 F.3d 796, 817(8th Cir.2006).
The second step is for the judge to determine whether he can be truly impartial when trying case. If he feels he cannot hear the case without bias, on account of the threat, then the judge has a duty to recuse himself irrespective of how it looks to the public. Section 455 of Title 28 requires recusal where the judge “has a personal bias or prejudice concerning a party.” 28 U.S.C. § 455(b)(1); see also 28 U.S.C. § 144 (addressing recusal upon motion by a litigant); Code of Conduct FOR United States Judges Canon 3 (2000) (imposing upon judges a duty to recuse themselves where they are either impartial or when their impartiality may reasonably be questioned). This step is highly personal in nature and requires each judge in such a situation to set aside emotion and thoughtfully examine his ability to impartially “administer justice without respect to persons.” 28 U.S.C. § 453. If he feels there is a risk of prejudice, it is incumbent on him to recuse himself from the case; failure to do so would amount to an abdication of duty and be in clear derogation of the solemn promise he made when he took his oath of office.
The third step in evaluating whether recusal is required is to apply the “objective” standard articulated in Liljeberg, 486 U.S. at 860, 108 S.Ct. 2194. The standard requires recusal if a reasonable third-party observer would perceive that there is a “significant risk” that the judge will be influenced by the threat and resolve the case on a basis other than the merits. The reasonable third-party observer is not a “partly informed man-in-the-street,” but rather someone who “understand^] all the relevant facts” and has examined the record and law. LoCascio v. United States, 473 F.3d 493, 496 (2d Cir.2007); see also Clemens, 428 F.3d at 1178(“The reasonable person in this context means a well-informed, thoughtful observer, as opposed to a hypersensitive or unduly suspicious person.” (internal quotation marks and citation omitted)); but see In re Nettles, 394 F.3d 1001, 1002 (7th Cir.2005) (“We must bear in mind that these outside observers are less inclined to credit judges’ impartiality and mental discipline than the judiciary itself will be.” (internal quotation marks and citation omitted)). The “objective” standard is a check to avoid even the “appearance of partiality,” Liljeberg, 486 U.S. at 860, 108 S.Ct. 2194, and insure that the judge’s decision is reasonable to an informed observer.
Although the Supreme Court has explained the § 455 standard in objective terms, a threat to the judge’s person or family necessarily involves the judge’s personal sentiment. The judge must take into account his own perception of the situation and personal feelings and ask himself if there is a significant risk that the threat will influence his decision. Put differently: Would a reasonable person understanding the judge’s perspective accept the judge’s decision to recuse (or not to recuse) himself? If the judge cannot, in good faith, answer “yes,” then he should revisit his analysis under the first two steps. If it is a close case, the balance tips in favor of recusal. United States v. Dandy, 998 F.2d 1344, 1349(6th Cir.1993).
These considerations are, necessarily, deeply personal to the judge. Only the judge himself, viewing the threats in context, can differentiate between threats that the author intended to be taken seriously and those made to anger or to manipulate the system. And, only the judge can determine whether he took them seriously or dismissed them. Although § 455 requires an objective standard on recusals, only the *1127judge — -who is the sole person in possession of all the relevant facts — must decide whether a reasonable person should be concerned with risk that the threats will taint the proceedings or create a perception of partiality.
C
Turning to the facts of the case at hand, it is apparent to us that the district judge did not plainly err when he did not recuse himself sua sponte after receiving Holland’s threatening phone message. The precise nature of the threat is not clear from the record, but the district judge described it as a message “that to some people could be construed as threatening.” The district court, however, did not consider the threats or Holland’s capacity to carry them out serious enough to refer the incident to the FBI, nor did he request additional security from the U.S. Marshal’s service.
Additionally, the judge carefully considered Holland’s extensive (and curious) history of making threats. Holland was before the district court to plead guilty of violating 18 U.S.C. § 871, making “knowingly and willful” threats against the President of the United States. Section 871 does not require proof of the defendant’s capacity, or evidence that the defendant has taken any affirmative steps, to carry out such threats. Holland’s current legal troubles also involved mailing threatening letters to: (1) a state' court judge (he later mailed another letter apologizing for' the threats); (2) to the prosecutor involved in a prior criminal proceeding; and (3) the President of the United States in violation of 18 U.S.C. § 876. When initially confronted with the letter to the President, Holland told an Idaho Department of Corrections lieutenant assisting the Secret Service that he did not want to remain in state custody any longer and that he wanted to go to federal prison instead. He thought he could get to federal prison by committing a federal crime. Holland also threatened a jail nurse in an apparent attempt to be transferred to the prison’s medical facility and threatened his former defense lawyer in an attempt to get him to mail a second letter from Holland threatening the President. There is no evidence in the record that Holland took any steps in furtherance of any of his threats.
The district judge found that “these [threats] are attempts to manipulate the criminal justice system.” This finding is amply supported by the record, including Holland’s behavior in the courtroom. At the beginning of the proceedings, Holland blurted out, “The detectives told me yesterday that I threatened you [the district judge] at home. Did I threaten you at home?” Later, when given an opportunity to address the court, Holland stated “I am not as bad as my paperwork says.... There ain’t no way I am going to look for a judge. There ain’t no damn way I am going to look for a President of the United States or anybody.... I have never hurt a victim.” He attempted to explain his threats as part of the prison culture: “Maybe it’s because of the lifestyle, the way prison is. Everybody threatens, T am going to do this, I am going to do that.’ It has got to the point where it is natural to do that.... But as far as carrying out the threats, no. There ain’t no way. Who would be stupid enough?” It was not until well after Holland offered this explanation, and was subsequently removed from the courtroom by refusing to be quiet during sentencing,7 that the judge addressed the *1128issue of Holland’s threatening phone messages.
Holland directs our attention to a comment the district court made suggesting that Appellant’s threats should be taken seriously. The court stated, “As I pointed out to him, many of his crimes are violent and assaultive in nature. So contrary to what he'says, if he is given the opportunity, he has the ability to carry out his threats.” Holland argues that this statement suggests that the district court may have taken the threats seriously, which would weigh in favor of recusal.8 However, there is compelling evidence that the district court dealt with Holland in an even-handed manner. It considered his motions, allowed Appellant to speak freely at sentencing, and treated him in a courteous manner. When reviewing the situation as a whole, we cannot conclude that a reasonable person in possession of all the facts would determine that the district court based its sentence on anything but the merits of the case. Consequently, we find no plain error in the district court’s decision not to recuse itself sua sponte under § 455. The judgment is
AFFIRMED.