Fisher's financial circumstances were probably such that "servicing the loan" (i.e. making required monthly payments on the loan) required a minimum revenue stream from tenants of the building.  A building in poor condition, or a building that would be difficult to convert from single tenant to multi-tenant use, would affect the long-term ability of Fisher to attract tenants willing to pay rent sufficient to generate that minimum revenue stream.  If, because of a declinining revenue stream from rents, Fisher were to default in making payments on the loan, the lenders could foreclose on (i.e. force the sale of) the real property by virtue of the deed of trust, but a building in poor condition might not generate enough at the foreclosure sale to pay off the loan, causing a loss to the pension trusts that Kennedy represented.