Down syndrome is the most common chromosomal condition that presents characteristic facial morphology and texture patterns. The early detection of Down syndrome through an automatic, non-invasive and simple way is desirable and critical to provide the best health management to newborns. In this study, we propose such a computer-aided diagnosis system for Down syndrome from photography based on facial analysis with ensemble learning. First, geometric and texture facial features are extracted based on automatically located facial landmarks, followed by feature fusion and selection. Then multiple classifiers (i.e. support vector machines, random forests and linear discriminant analysis) are adopted to identify patients with Down syndrome. An accurate and reliable decision is finally achieved by optimally combining the outputs of these individual classifiers via ensemble learning that captures both the shared and complementary information from different classifiers. The best performance was achieved by using the median ensemble rule with 0.967 accuracy, 0.977 precision and 0.933 recall.