Transforming growth factor-beta 1 inhibited proliferation of a human ovarian carcinoma cell line (NIH-OVCAR-3). The inhibition of NIH-OVCAR-3 cell proliferation was accompanied by a decrease in clonogenic potential, evidenced by the reduced ability of TGF-beta 1-treated NIH-OVCAR-3 cells to form colonies on a plastic substratum. This rapid decrease of clonogenic potential, which was detected 6 h after addition of TGF-beta 1 was dose-dependent (IC50 = 4 pM). Fluorescence microscopy of DAPI-stained cells supported by electron-microscopic examination showed that TGF-beta 1 induced chromatin condensation and nuclear fragmentation. In addition, oligonucleosomal-sized fragments were detected in the TGF-beta 1-treated cells. These features indicated that TGF-beta 1 induced NIH-OVCAR-3 cell death by an apoptosis-like mechanism. This TGF-beta 1 apoptotic effect was subject to modulation by cell density. It was observed that an increase in cell density (up to 20 x 10(3) cells/cm2) protected NIH-OVCAR-3 cells against apoptosis induced by TGF-beta 1. Conditioned medium from high-density cultures of NIH-OVCAR-3 cells did not inhibit apoptosis induced by TGF-beta 1 on NIH-OVCAR-3 cells cultured at low density, suggesting that the protective effect of cell density was not related to the cell secretion of a soluble survival factor.