OPINION
Yusuf Abdur-Rahman appeals from the trial court’s order amending a domestic violence order to include Yusuf and Nicole Peterson’s minor child. After a thorough review of the parties’ arguments, the record, and the applicable law, we find reversible error and, accordingly, reverse and remand this matter to the trial court for further proceedings.
The parties have never been married but have a minor child together. The parties have been before the family court several times based on Nicole’s alleging domestic violence. On November 12, 2009, the court entered a domestic violence order (“DVO”) on behalf of Nicole. Thereafter, on May 11, 2010, Nicole filed a motion to include the parties’ minor child in the DVO. The court held a hearing on May 20, 2010.
At the hearing Nicole testified that the minor child was returned to her from Yu-sufs care with a rash, two burns on her leg, and a bump on her head. Nicole also stated that on Sunday, May 9, 2010, Yusuf brought the minor child to Nicole’s mother’s house for Mother’s Day. While there, Nicole noticed a rash on the minor child and requested that Yusuf take the child to the pediatrician the next day. Nicole testified that Yusuf acted suspiciously and that he would not allow anyone to be alone with the child that day. When Yusuf returned the child to Nicole on May 11, 2010, Nicole noticed two burns on the child’s leg and Yusuf could not provide a plausible explanation for the burns. Nicole took the child to the hospital.
Yusuf testified that he did not know where the burns came from but that he thought they occurred at Nicole’s mother’s house, since after their visit he noticed the burns. Yusuf added that there *825were curling irons at Nicole’s mother’s house and that he heard the child cry while she was with Nicole’s sisters. Yu-suf treated the burns with aloe vera, Neosporin, and a bandage. When he took the child to the pediatrician, the doctor treated the rash and not the burns. Yu-suf introduced numerous handwritten notes and e-mails from Nicole to Yusuf that expressed her gratitude for his being a good father. After hearing this evidence, the court concluded that the child’s burns were worrisome and severe, and that Yusufs indifferent attitude toward the cause of the child’s burns and obtaining treatment for her injuries was sufficient for the court to extend the DVO to the child.
In its order amending the DVO, the trial court found that Nicole had established by a preponderance of the evidence that Yu-suf had committed an act of domestic violence or abuse and that it may occur again, that the DVO was extended to the minor child, and for Yusuf to have supervised visitation for three hours each Saturday. Yusuf filed a motion to alter, amend, or vacate the court’s DVO, which the court denied. It is from this amended DVO that Yusuf now appeals.
On appeal, Yusuf presents four arguments, namely, (1) the trial court erred when it denied Yusuf a full evidentiary hearing by refusing to hear the testimony of two witnesses, one of whom would have testified that the burns to the minor child had to have happened at the home of Nicole’s mother while both parents were present; (2) the trial court refused to allow testimony regarding Nicole’s mental health history into evidence when Nicole had attempted to commit suicide while pregnant less than nineteen months prior to the hearing, and custody was at issue in the hearing; (3) the trial court erred when the court made medical findings regarding the degree of the burns and the appropriate treatment of said burns when there was no medical proof presented by testimony or through documents; and (4) the court erred by entering a DVO because the facts do not support a DVO for the minor child.
At the outset, we note that Kentucky Revised Statute(s)(KRS) 403.750 provides for the issuance, reissuance, and amendment of DVOs.2 The procedures a court must undertake to issue, reissue, or amend a DVO differ significantly. In order to issue a DVO, the court must conduct a hearing as provided for in KRS 403.745 and may only enter a DVO if the petitioner shows that by “a preponderance of the evidence that an act or acts of domestic violence and abuse have occurred and may again occur.” KRS 403.750(1); Bissell v. Baumgardner, 236 S.W.3d 24, 29 (Ky.App.2007). The preponderance of the evidence standard is met when the evidence establishes that the petitioner “was more likely than not to have been a victim of domestic violence.” Commonwealth v. Anderson, 934 S.W.2d 276, 278 (Ky.1996); Baird v. Baird, 234 S.W.3d 385, 387 (Ky.App.2007). The definition of domestic violence and abuse, found in KRS 403.720(1), includes “physical injury, serious physical injury, sexual abuse, assault, or the infliction of fear of imminent physical injury, serious physical injury, sexual abuse, or assault between family members.”
As to amending a DVO, as in the case sub judice, KRS 403.750(3) simply states that “[u]pon proper filing of a motion, either party may seek to amend a domestic violence order.” The statute does not state if or when a hearing must be held or what evidence the court may or should *826consider before amending a DVO.3 Moreover, KRS 403.750 permits the trial court to extend the protection of a DVO to a minor child of the petitioner if it finds from a preponderance of the evidence that an act or acts of domestic violence and abuse have occurred and may again occur. See KRS 403.750(l)(d). However, when the minor child is the child of the respondent, the court must look to KRS 403.3204 which states in part: “(2) If domestic violence and abuse, as defined in KRS 403.720, has been alleged, the court shall, after a hearing, determine the visitation arrangement, if any, which would not endanger seriously the child’s or the custodial parent’s physical, mental, or emotional health.” KRS 403.320(2). Thus, pursuant to KRS 403.320(2), if a DVO concerns a parent’s visitation with a minor, then a hearing shall be conducted and a finding made whether the visitation would seriously endanger either the child’s or custodial parent’s physical, mental or emotional health.
We bear in mind that in reviewing the decision of a trial court the test is not whether we would have decided it differently, but whether the findings of the trial court were clearly erroneous or that it abused its discretion. Cherry v. Cherry, 634 S.W.2d 423, 425 (Ky.1982); Kentucky Rules of Civil Procedure (CR) 52.01; Reichle v. Reichle, 719 S.W.2d 442, 444 (Ky.1986). Findings are not clearly erroneous if they are supported by substantial evidence. Moore v. Asente, 110 S.W.3d 336, 354 (Ky.2003). Substantial evidence is evidence of sufficient probative value that permits a reasonable mind to accept as adequate the factual determinations of the trial court. Id. A reviewing court must give due regard to the trial court’s judgment on the credibility of the witnesses. Id. “Abuse of discretion in relation to the exercise of judicial power implies arbitrary action or capricious disposition under the circumstances, at least an unreasonable and unfair decision.” Kuprion v. Fitzgerald, 888 S.W.2d 679, 684 (Ky.1994) (citations omitted).
*827As pointed out in Rankin v. Criswell, 277 S.W.3d 621, 624 (Ky.App.2008), domestic violence proceedings are not criminal matters, but the consequence for both parties are very significant. And, as quoted in Wright v. Wright, 181 S.W.3d 49 (Ky.App.2005):
The filing of a DVO petition has enormous significance to the parties involved. If granted, it may afford the victim protection from physical, emotional, and psychological injury, as well as from sexual abuse or even death. It may further provide the victim an opportunity to move forward in establishing a new life away from an abusive relationship. In many cases, it provides a victim with a court order determining custody, visitation and child support, which he or she might not otherwise be able to obtain. The full impact of EPOs and DVOs are not always immediately seen, but the protection and hope they provide can have lasting effects on the victim and his or her family.
On the other hand, the impact of having an EPO or DVO entered improperly, hastily, or without a valid basis can have a devastating effect on the alleged perpetrator. To have the legal system manipulated in order to “win” the first battle of a divorce, custody, or criminal proceeding, or in order to get “one-up” on the other party is just as offensive as domestic violence itself. From the prospect of an individual improperly accused of such behavior, the fairness, justice, impartiality, and equality promised by our judicial system is [sic] destroyed. In addition, there are severe consequences, such as the immediate loss of one’s children, home, financial resources, employment, and dignity. Further, one becomes subject to immediate arrest, imprisonment, and incarceration for up to one year for the violation of a court order, no matter what the situation or circumstances might be.
Id. at 52. The Wright Court sympathized with the family courts and the heavy dockets under which they must perform their duties and the need to operate with speed and efficiency. Nevertheless, it cautioned that courts are required to provide each party with a full evidentiary hearing. Rankin, 277 S.W.3d at 625 (citing Wright, 181 S.W.3d at 53). With this jurisprudence in mind we turn to Yusufs arguments.
Yusuf first argues that the trial court erred when it denied Yusuf a full evidentiary hearing by refusing to hear testimony of two witnesses, one of whom would have testified that the burns to the minor child had to have happened at the home of Nicole’s mother while both parents were present. Yusuf requested that he be permitted to call his instructor at Batterer’s Intervention, Maurice Williamson, because Williamson would testify that Yusuf was a model student with a consistent attendance record. The trial court determined that such testimony would not be relevant to the issue before the court, namely whether to amend the DVO in light of the injuries received by the minor child. Yusuf also requested that he be permitted to call Desiree Crank, Yusufs partner, who had changed the minor child prior to the child’s going to the home of Nicole’s mother and did not notice the burns until the minor child returned from the home. The trial court determined that it did not need Crank to testify based on the evidence before it.
Kentucky Rules of Evidence (KRE) 402 provides that, with some exceptions, all relevant evidence is admissible. That all evidence must be relevant in order to be admissible is perhaps the most fundamental rule of evidence. See KRE 402; see also Robert G. Lawson, The Kentucky Ev *828 idence Law Handbook, § 2.00 (4th ed. 2003) (“The first critical determination to be made concerning the admissibility of any item of evidence is its relevance; no other principle or concept is of any significance in the absence of a positive determination on this issue”). KRE 401 defines relevant evidence as “evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence.”
Relevant evidence may, nevertheless, be inadmissible “if its probative value is substantially outweighed by the danger of undue prejudice, confusion of the issues, or misleading the jury, or by considerations of undue delay, or needless presentation of cumulative evidence.” KRE 403. In both respects, we review a trial court’s determination for an abuse of discretion. See Love v. Commonwealth, 55 S.W.3d 816, 822 (Ky.2001) (citing Commonwealth v. English, 993 S.W.2d 941, 945 (Ky.1999); Barnett v. Commonwealth, 979 S.W.2d 98 (Ky.1998)).
In the case sub judice, the trial court determined that testimony from Williamson, Yusufs instructor at Batterer’s Intervention, concerning his attendance at class and his model behavior in class, was not relevant. As noted we review this determination for an abuse of discretion. Based on the issue before the court as to whether to amend the DVO to include the minor child, we do not find an abuse of discretion in the court’s exclusion of Williamson’s testimony because Yusufs class attendance and conduct while in class have little relevance to his conduct out of class.
In regard to the testimony of Crank, we agree with Yusuf that the trial court exceeded its discretion in denying Yusufs request that he be permitted to call Crank, given that she had changed the minor child prior to the child’s going to Nicole’s mother’s house and did not notice the burns until the minor child returned from there. Certainly her testimony would have been instrumental in Yusufs attempt to establish the time of the injury to the minor child. The trial court should have considered her testimony prior to determining whether Yusuf had committed an act of domestic violence which warranted the issuance of a DVO concerning the minor child. This error was not harmless error. Accordingly, we reverse the trial court’s decision for failure to include Crank’s testimony.
Yusuf next argues that the trial court refused to allow testimony regarding Nicole’s mental health history into evidence when Nicole had attempted to commit suicide while pregnant less than nineteen months prior to the hearing, and custody was at issue in the hearing. The trial court determined that Nicole’s mental health history prior to the birth of the child was not relevant to the issue at hand. We do not find abuse of discretion in the trial court’s determination, given the length of time that had passed since the child had been born and that there were no allegations of mental health issues concerning Nicole’s unfitness to parent after the birth of the child. As such, we find no error.
We now turn to Yusufs third claimed error, namely, that the trial court erred when the court made medical findings regarding the degree of the burns and the appropriate treatment of said burns when there was no medical proof presented by testimony or through documents.
Upon hearing the testimony presented, and after viewing the picture of the burns on the minor child, the court issued its ruling from the bench. The trial court discussed the severity of the burns and opined that such burns were third-degree *829burns and required treatment from a medical provider. At the crux of the trial court’s ruling was not the degree of the burns but the lack of concern expressed by Yusuf over the origin of the burns and the lack of consideration he gave for the need of medical treatment. Accordingly, we find any error by the court regarding the medical findings of the burns to be harmless error. While it is true that a trial court could not determine the degree of the burns without medical proof, and to the extent it did so was error, its ruling focused on Yusufs lack of concern for the origin of the burns and Yusufs failure to seek medical treatment for what appeared to be severe burns. Thus, we find no reversible error on this issue. See CR 61.01.
Yusuf last argues that the court erred by entering a DVO because the facts do not support a DVO for the minor child. We decline to address this argument because it is moot in light of our decision to remand for the failure of the trial court to hear the testimony of Desiree Crank, discussed infra. Upon remand, the trial court shall again conduct a hearing and include all testimony that is relevant and properly admissible.
Accordingly, we reverse and remand for proceedings not inconsistent with this opinion.
COMBS, Judge, concurs.
KELLER, Judge, concurs in part, dissents in part, and files separate opinion.