Debt is always cheaper source of finance because of following reasons.
a) Tax benefit: The firm gets an income tax benefit on the interest component that is paid to the lender. Dividends to equity holders are not tax deductable.
b) Limited obligation to lenders: In the event of a firm going bankrupt, which is what happened with Lehman Brothers, equity holders lose everything. But, debt holders have the first claim on company assets (collateral), increasing their security. So since debt has limited risk, it is usually cheaper. Equity holders are taking on more risk, hence they need to be compensated for it with higher returns.
c) Limited upside: Since the equity holder has a stake in the business; he can actually participate in the potential upside in earnings. PE, Venture Capital funds usually buy stakes in high potential companies at cheap valuations, and since they have a minority stake in the company, they are entitled to a share of the profits. Plus they can exit after a few years at a fantastic premium. On the other hand debt holders have an upside limited to the fixed rate of interest they receive every year.