Moving around the continent and talking to different Africans, three broad levels of identity and belonging emerge. The first level is ethnic or language identity; the second is the national identity, which invariably means the boundaries established by the Berlin Conference of 1884, a legacy of European colonialism; and finally, there is the pan-African continental identity that emerged in the diaspora (from Sylvester Williams and Marcus Garvey, before being articulated by Nkrumah and others) and which is still being constructed by the AU.
Different Africans process these three levels in different ways. Black Africans, with an ethnic and language identity which evolved solely in the context of the continent and is therefore exclusively rooted on the continent, are generally very comfortable with their ethnic/language identity (Igbo, Zulu, Wolof), alongside the Pan African continental identity, but sometimes have a profound problem with the national Berlin-state identity--hence the many civil and secessionist wars that have impacted Africa (Nigeria, Congo, Sudan etc).
For others without this historic ethnic or language identity (i.e. white. Asian and Arab Africans), what emerges is ease with their ethnic and language identity, as well as the national identity (South African, Kenyan etc), while having a general problem with the continental identity, which is sometimes viewed as a racialised identity, synonymous with black people.
Ergo I am not black so I cannot be African--a point raised by a North African Arab brother at a recent diaspora conference on African Identity, who challenged those who spoke interchangeably of blackness and Africanness, while refusing to accept that there was also some ambiguity about Africanness on the north side of the Sahara, despite Nkrumah's attempts at creating an African identity that included both sides of the desert.
It is often referred to as the great issue few people want to discuss. There is obviously work to be done in getting all these levels of identity discussed above to align for all Africans.
However the North African identity is not the only taboo. While people are comfortable having discourses about the 'Berlin 1884' national state and pan-African identity, having a discussion about ethnic and language identities in Africa is always difficult--and people believe that dwelling on it opens them up to charges of 'tribalism'...