Appointing an imam is obligatory. The consensus (ijmā‘) of the Sahabah and the Tabi‘īn shows that (the caliphate) is necessary according to the sacred law. At the death of the Messenger of Allah (ﷺ), the men around him proceeded to render the oath of allegiance (bay‘ah) to Abu Bakr (رضي الله عنه) and to entrust him with the supervision of their affairs. And so it was at all subsequent periods. In no period were the people left in a state of anarchy. This was so by general consensus (ijmā‘), which proves that the position of imam is a necessary one.
Some people have expressed the opinion that the necessity of appointing an imam is indicated by the intellect (rational reasons), and that the consensus which happens to exist merely confirms the authority of the intellect in this respect. As they say, what makes (the position of imam) intellectually (rationally) necessary is the need of human beings for social organization and the impossibility of their living and existing by themselves. One of the necessary consequences of social organization is disagreement, because of the pressure of cross-purposes. As long as there is no ruler who exercises a restraining influence, this (disagreement) leads to trouble which, in turn, may lead to the destruction and uprooting of mankind. Now, the preservation of the (human) species is one of the necessary objectives (maqāsid) of the sacred law.
This very idea is the one the philosophers had in mind when they considered prophecy as something (intellectually) necessary for mankind. We have already shown the incorrectness of (their argumentation). One of its premises is that the restraining influence comes into being only through a sacred law from Allah, to which the mass submits as a matter of belief and religious creed. This premise is not acceptable. The restraining influence comes into being as the result of the impetus of power (mulk) and the forcefulness of the mighty, even if there is no sacred law. This was the case among the Magians and other nations who had no scriptures and had not been reached by a prophetic mission.
Or, we might say (against the alleged rational necessity of the caliphate): In order to remove disagreement, it is sufficient that every individual should know that injustice is forbidden him by the authority of the intellect. Then, their claim that the removal of disagreement takes place only through the existence of the sacred law in one case, and the position of the imam in another case, is not correct. (Disagreement) may (be removed) as well through the existence of powerful leaders, or through the people refraining from disagreement and mutual injustice, as through the position of the imam. Thus, the intellectual proof based upon that premise does not stand up. This shows that the necessity of (the position of imam) is indicated by the sacred law, that is, by general consensus (ijmā‘), as we have stated before.
Some people have taken the exceptional position of stating that the position of imam is not necessary at all, neither according to the intellect nor according to the sacred law. People who have held that opinion include the Mu‘tazili al-Asamm and some of the Khawarij, among others. They think that it is necessary only to observe the sacred laws. When Muslims agree upon (the practice of) justice and observance of the divine laws, no imam is needed, and the position of imam is not necessary. Those (who so argue) are refuted by the general consensus (ijmā‘). The reason why they adopted such an opinion was that they (attempted to) escape the authority (mulk) and its overbearing, domineering, and worldly ways. They had seen that the sacred law was full of censure and blame for such things and for the people who practiced them, and that it encouraged the desire to abolish them.
It should be known that the sacred law does not censure power (mulk) as such and does not forbid its exercise. It merely censures the evils resulting from it, such as tyranny, injustice, and pleasure-seeking. Here, no doubt, we have forbidden evils. They are the concomitants of power (mulk). (On the other hand,) the religious law praises justice, fairness, the fulfillment of religious duties, and the defense of the religion. It states that these things will of necessity find their reward (in the other world). Now, all these things are concomitants of power (mulk), too. Thus, censure attaches to power (mulk) only on account of some of its qualities and conditions, not others. (The sacred law) does not censure power (mulk) as such, nor does it seek to suppress it entirely. It also censures concupiscence and wrathfulness in responsible persons, but it does not want to see either of these qualities relinquished altogether, because necessity calls for their existence. It merely wants to see that proper use is made of them. Dawud and Sulayman (صلوات الله وسلامه عليهما) possessed power (mulk) such as no one else ever possessed, yet they were divine prophets and belonged among the noblest human beings (that ever existed).
Furthermore, we say to them: The (attempt to) dispense with power by (assuming) that the institution (of the caliphate) is not necessary, does not help you at all. You agree that observance of the sacred laws is a necessary thing. Now, that is achieved only through group feeling and power (mulk), and group feeling, by its very nature, requires (the existence of) authority (mulk). Thus, there will be authority (mulk), even if no imam is set up. Now, that is just what you (wanted to) dispense with.
If it has been established that the institution (of the caliphate) is necessary by general consensus (ijmā‘), (it must be added that the institution of the caliphate) is a community duty (farḍ al-kifāyah) and is left to the discretion of all competent Muslims (ahl al-hall wa al-‘aqd). It is their obligation to see to it that (the imamate) is set up, and everybody has to obey (the imam) in accordance with the verse of the Qur’an, “Obey Allah, and obey the Messenger and the people in authority among you.” [4:59]
[Abdur Rahman ibn Khaldun, Muqaddimah, Chapter 3, Section 26]
The Caliphate 
So how do you go about getting there? I think the concept of Caliphate is a moot point – it is Muslims remembering and wishing for the good times. We now live in an age where a new political dispensation needs to come about the is based within the Islamic tradition but has certain aspects of the Western democratic traditio such as public particiaption in election, rule of law (by and for all), equal rights for minorities, etc. Interesting the Shia have reached that stage where Ay. Khomeni managed to amalgamate western republicanism within shii political understanding. One of the things he did was to put the interest of the state BEFORE the impmentation of shariah – something that is lacking within the sunni tradition. Muslims have 3 choices – 1. Ay. khomenis position as outlined in his work islamic Republic based on the works of Plato and Farabi, 2 – copying the western countries – i.e. liberal democracy 3 OR a new understanding of politics that is based within sunni islamic tradition but with aspects of western understanding of representative politics. However no. is highly unlikely – afterall Shiis are viewed as rafidis, 2. muslims are a religious people and will not tolearate half naked women on their beaches as evident in Germany, France etc. And with no.3 – someone of Ay.Khomeni stature needs to appear from amongst Sunni ulema who can do it.
For Sunni Muslims, the establishment of the caliphate is an obligation which is mentioned by many scholars. New is not always best – just look at the economic mess which many Western countries are facing.
The fact is that poverty in the Muslim world can be solved just by fully implementing the rulings of Zakat.
You mention Western democracy – I would argue that the caliphate has more similarities with the idea of a constitutional republic as envisioned by the Founding Fathers of America. They viewed democracy as mob rule. James Madison wrote: “…democracies have ever been spectacles of turbulence and contention; have ever been found incompatible with personal security or the rights of property…” Please have a look at the 11 minute Youtube video which discusses democracy and republics. In a constitutional republic, there is a body of law which is supreme and cannot be undermined by the majority. Having said that, both democracies and constitutional republics consider the people as sovereign, which is not compatible with the Islamic concept of ruling, as sovereignty has to be with the Shari‘ah.
A number of Islamic scholars have codified laws on aspects of the Islamic system based on the Qur’an, Sunnah, Ijma‘ and Qiyas, but they are perhaps not very well known. Some scholars have also discussed how the Prophetic Manhaj (methodology) of setting up the Madinan state can be adapted as a model for us today, but this requires ijtihad.