There is an increasing scholarly interest on federalism studies among the scholars of political science. Such a surge coincides with the re-emergence of the importance of issue in the political field particularly among the politicians of multi-ethnic societies. Federalism and other devolved forms of government have began to be discussed as alternative types of political organization among these societies and as a remedy to the conflicts and problems arising mainly from ethnic diversity. In such an environment, an analysis of Africa’s federations may contribute to the production of this literature on federalism. The aim of this comparative analysis may also contribute to the discussions on the proper system of government for the African state in the context where a growing number of African scholars and politicians are looking for a panacea for the problems that they have been experiencing since their independence. The major purpose of this paper is, therefore, to analyze if federal or other devolved forms of government can hold a promise for African countries. Today, Ethiopia, Comoros and Nigeria are the only three constitutional African federations while the status of other federal or quasi-federal states of Africa is contentious. The paper will first focus on the relevance of federal systems for Africa’s “fragile” states after briefly displaying the main characteristics of federalism as a governmental system. Before proceeding to the cases, namely, Ethiopia, Nigeria, South Africa, Comoros and Tanzania, the paper will then present a brief analysis about federalism debates while, at the same time, elaborating the main theses of both the adherents and critics of federalism. The following last part of the paper will examine whether federalism could perform its attributed merits in the African state.