The opinion of the court was delivered by
This case involves an attempt to remodel a nonconforming structure and use, a rooming and boarding house, in a residential zone in Atlantic Highlands. The appeal is from a ruling in the Law Division dismissing plaintiffs’ complaint in lieu of prerogative writ and affirming the denial of zoning relief by the Atlantic Highlands Zoning Board of Adjustment. We essentially affirm the result reached in the Law Division, but for somewhat narrower reasons than expressed by the judge in her written opinions of January 3, 1990 and July 10, 1990.
John and Helen Lehen own an improved property located at 89 Center Street in the Borough of Atlantic Highlands. The property is located in an R-l residential area. The Victorian-style dwelling has been used as a rooming house and multi*395family dwelling and was a valid nonconforming use since well before the Lehens’ purchase of the property in 1980.
At the time of the Lehens’ purchase of the property the structure consisted of ten rental rooms and two apartments. According to the Lehens, each habitable room or apartment is designated by the State as a “dwelling unit.” N.J.A.C. 5:28-1.2. Boarding rooms may be occupied only by one person; apartments may be occupied by more than one person.
In 1981 or 1982, the Lehens requested and received construction permits from the Borough permitting them to change the number of units from ten rooms and two apartments to five rooms and five apartments. No proposed plans were submitted during this permitting process. All of the renovations were internal.
In 1987 the Lehens decided to remodel the third floor of the house, which has been referred to as “a V2-floor.” A steeply-sloped roof with five peaks limited available space for the third-floor tenants. Up until this time the third floor had consisted of one apartment and two boarding rooms with a shared kitchen. The Lehens proposed to convert the two boarding rooms on the third floor into another apartment, reducing the number of total dwelling units from ten to nine but increasing the number of apartments from five to six. The proposed renovations would “square off” the roof in order to create more room.
The Lehens contacted Alfred Katz, the Borough Zoning Official, and told him of their plans “to open up the third floor and make more head room and expand the third floor area.” According to the Lehens, Katz requested a survey and photographs of the building. After taking time to “check it out,” Katz told the Lehens that “he didn’t feel there would be any problem with it.” The Lehens then applied for and received a construction permit. The construction permit described the work as: “square off rooms and apartments on 3rd floor and replace roof.”
*396Katz testified at the prerogative writ hearing that in 1987 he had told Mrs. Lehen that the use of dormers to make more room on the third floor could, under a strict or technical interpretation of the ordinance, be considered an expansion of the nonconforming building. Katz thought, however, that it was “an intelligent thing to do” and had no problem with the proposed construction. He testified that he understood the Lehens would be raising the roof line, but not altering the peak of the roof.' Based upon his understanding of the proposed construction, Katz preliminarily approved the orally described plans and allowed the Lehens to bypass approval from the Zoning Board.
A construction permit was issued in December 1987 which stated, as noted: “Description of Work—Square off rooms and apartments on 3rd floor and replace roof.” The permit was signed by the construction official, Gerald Menna. No plans regarding the proposed construction were ever requested or submitted. Menna testified that he did not get a plan from the Lehens; he admitted that it was “very irregular that I would have issued a building permit without a plan.” Menna stated that it “could have been a mistake,” and that prior to his taking over the construction office, a few weeks before, the office had been a “very loose operation.” Nevertheless, on the basis of the construction permit, the Lehens also received the necessary plumbing and electrical permits.
The Lehens began construction on the third floor in June 1988. They did not use a contractor. While the roof was being removed, Mr. Lehen noticed that the corners of the building and roof line were “rotted” and “charred.” As a result of the deterioration, Lehen decided to remove and rebuild the entire third floor instead of simply remodeling. A new platform, or floor, for the third floor was installed which extended out three to four feet beyond the first and second floors on three sides. The square footage of the third floor increased from 1,226 to 1,384 square feet, an increase of 158 square feet.
*397On July 19,1988 defendants Katz and Menna came to the site and ordered the work stopped. At that point, the floor had been framed and partially sheeted and the roof was under construction. The Lehens were told to come to the Borough Hall to get revised building permits. They were permitted to finish closing up the roof to protect the construction from the weather. A written stop order was issued that day. The reasons given for the stop-work order were: (1) an approved construction plan was required, and (2) the construction was in violation of the building permit.
Alfred Katz sent a letter to the Lehens on August 3, 1988 advising them that under the Atlantic Highlands Zoning Ordinance a nonconforming use cannot be altered or enlarged without approval from the Zoning Board. He admitted that at the time they had initially talked he did not feel that the Lehens’ planned renovations were significant enough to require Zoning Board approval. The present undertaking, however, exceeded the scope of what Katz anticipated and now required official action.
The Lehens first challenged the stop-work order by appealing to the Monmouth County Construction Board of Appeals. The case was heard on September 29, 1988. The Construction Board affirmed the stop-construction order and “strongly” recommended that structural plans be submitted before work continued. The Lehens submitted a sketch to the Borough officials but never received a revised permit to continue.
Meanwhile, the Lehens applied to the Zoning Board for a variance and an interpretative ruling on whether a variance was required. The Zoning Board heard the matter on November 5 and 16,1988 and rendered a decision in the form of a resolution on December 21, 1988. The Zoning Board relied upon Article XVII of the Atlantic Highlands Zoning Ordinance governing nonconformance. It found that the building contained two nonconforming uses,, rental rooms and apartments. Even though the number of actual units in the nonconforming struc-
*398ture would be decreased, the Zoning Board concluded that the addition of another apartment expanded the nonconforming apartment use in violation of the ordinance. Finding that the .Lehens had presented no “special reasons” for granting the variance, the Zoning Board denied the application. The Lehens then filed this complaint in lieu of prerogative writ alleging that: (1) the denial of the variance by the Zoning Board was arbitrary, unreasonable and capricious; (2) as a matter of law, the nonconforming use was being decreased, not expanded, and no variance was required under the Zoning Ordinance; and (3) the municipality was estopped from revoking the construction permit because of the Lehens’ expenditure of $15,000 and good faith reliance upon the issued permits. They later abandoned their appeal from the Zoning Board’s denial of the request for a variance.
The Law Division judge heard testimony on December 20, 1989. She adopted the reasoning of the Zoning Board in deciding that the renovations constituted an expansion or extension of a nonconforming use for which a variance was required. She reasoned that although the nonconforming use of the building as a single-room boarding house would decrease with the renovations (five to three units), the nonconforming use as a multi-family apartment dwelling would increase (five to six units), as could the number of people who might potentially reside in the building.
Turning to the Lehens’ estoppel argument, the judge made findings of fact and concluded that the Lehens never explicitly told either Katz or Menna that the “squaring off” of the third floor would involve a complete redesign and rebuilding of the third floor and roof with an expansion of 158 square feet on the third floor. This work was “neither anticipated nor authorized at the time the permit was issued.” For these reasons, the judge dismissed the complaint with prejudice. We accept the judge’s fact-finding as well-supported in the record. Rova Farms v. Investors Ins. Co., 65 N.J. 474, 483-484, 323 A.2d 495 *399(1974). We also accept her legal conclusion denying any relief on this application and on this record.
We are satisfied that no view of this record supports the Lehens’ contention that they are entitled to increase the floor space in the third-floor level by 158 square feet by the use of the three to four foot overhangs. This extension was definitely beyond the scope of the construction permit issued with such informality in December 1987. There was no justification for this expansion, either in fact or in law. See Burbridge v. Mine Hill Tp., 117 N.J. 376, 568 A.2d 527 (1990); N.J.S.A. 40:55D-5 & 68; Atlantic Highlands Zoning Ordinance, Article XVII, § 146-68(B.) and (C.).
The linear and cubic foot expansion of the third floor markedly changed the appearance and size of that floor and the house. The photographs in evidence, which we have viewed, amply demonstrate this inappropriate aesthetic effect. Nonconforming uses may not be enlarged as of right except where the change is so negligible or insubstantial that it does not warrant judicial or administrative interference. Town of Belleville v. Porrillo’s, 83 N.J. 309, 316, 416 A.2d 388 (1980). If there is doubt whether the extension of the use is substantial, the extension should be disapproved. The inquiry must focus on “the quality, character and intensity of the use, viewed in [its] totality and with regard to their overall effect on the neighborhood and the zoning plan.” Id. at 314, 416 A.2d 388. This involves an appraisal of the use, both before and after the change.
We conclude that any future plans for construction or renovation on this building must revert to the original floor space on the third-floor level. The expansion of the third floor by the overhangs cannot be justified on any theory of estoppel. We do disagree with the judge’s view that the proposed alteration in the ratio of single-room rentals to apartments, i.e., from 5:5 to 3:6 is an increase in the nonconforming aspects of the use. We conclude from this record that any prediction on the *400impact of this change on the number of occupants, increase in parking, and so forth, is completely speculative. Technically, there is actually a decrease in the number of living units. Whether the actual number of occupants will increase is debatable since the new apartments are one-bedroom units. We find that the change in ratio of rooms to apartments alone is not sufficient in these circumstances to deny the building permit and require the Lehens to get variance relief. See Black v. Montclair, 34 N.J. 105, 118, 167 A.2d 388 (1961); State v. Wagner, 81 N.J.Super. 206, 211, 195 A.2d 224 (Law Div.1963).
In respect of the Lehens’ estoppel claim, we conclude that the judge was on solid ground in rejecting this contention in regard to the floor-space enlargement. The appellants clearly went beyond the scope of the description of the work in the December 1987 construction permit in this respect. The doctrine of equitable estoppel is hesitantly applied against public entities but it will be invoked against them where interests of justice, morality and common fairness dictate. Palisades Properties, Inc. v. Brunetti, 44 N.J. 117, 207 A.2d 522 (1965); Juliano v. Borough of Ocean Gate, 214 N.J.Super. 503, 520 A.2d 418 (Law Div.1986). “A prerequisite of equitable.estoppel, however, is the good faith reliance by one party on the conduct of another to the detriment of the relying party.” Lizak v. Faria, 96 N.J. 482, 499, 476 A.2d 1189 (1984).
We are satisfied on this record that the Lehens proceeded in good faith and in reliance on the construction permit to “square off the rooms and apartments on the third floor and replace the roof.” To this extent, the municipality is estopped to deny appropriate permits for future construction and occupancy. See Hill v. Board of Adjustment, Borough of Eaton-town, 122 N.J.Super. 156, 164-165, 299 A.2d 737 (App.Div. 1972). Denial of relief, at least to this extent, would impose too much hardship on the Lehens who proceeded in good faith as to this part of their remodeling endeavor and invested substantial funds.
*401In conclusion, if a future proposal to complete the work is (1) limited to the original third-floor space, (2) restricts the construction to squaring off the existing third-floor rooms, and (3) includes replacing the roof, the municipality may not deny necessary permits, even though the rental room:apartment ratio is altered from 5:5 to 3:6.
Affirmed, as modified.