Dana is of Yemenite descent, and it's quite possible that the language her parents used among themselves while she grew up was Arabic. However, as the focus in the first years after the Israeli state was on assimilating the immigrants coming to the country, people were encouraged not to bring up their children with other languages than Hebrew. A lot of Sefaradim (in general Jews from the Middle East, Balkan and Turkey) today feel this was especially expected of them, as their culture (coming from the Arab world, i.e. «backwards», «Eastern», etc) was considered inferior by the Ashkenazi elite. So it's not very likely that Dana's parents spoke much Arabic to their children. On the other hand, as they probably used this between themselves, in the extended family and with other Yemenites, she must have been exposed to it while growing up. (These days, there have been signs that the attitudes towards Arabic among Israeli Jews are changing read the Ha'aretz article Chattering away in Arabic if you want more information.)
So Dana sings in Yemeni Arabic, then? Nope. She uses a mix of Egyptian and Levantine Arabic quite different from the dialect used in Yemen. Not only the Yemenites, but also the other big immigrant communities in Israel from Arab countries Iraqis, Moroccans and Tunisians use dialects quite different from the Levantine dialects used in Syria, Lebanon, Jordan and among Palestinians (including Israeli Arabs, Druze and Circassians). Her Arabic is mostly based on Egyptian and Lebanese pop songs, the Friday night Arabic films (mostly Egyptian) showed on Israeli TV (and incidentally very popular in Israel) and Palestinian slang.
Is her Arabic good? Well yes and no. Her pronounciation is generally good, even though she sometimes mixes G and J in the same lyrics alternative pronounciation of the same consonant, G is Egyptian, J is used in all other Arabic dialects. (The worst example of this, however, was Ofra Haza (who was also of Yemenite descent) in the song Galbi (which, despite this, is one of my definite desert island discs). Here she used G for J (Egyptian), G for Q (Yemenite and Levantine countryside) as well as J for J (Arabic outside of Egypt). This is like mixing Cockney, Yorkshire and New York English at the same time not exactly an indication of fine-tuned linguistic abilities...)
Anyhow, Dana's pronounciation is fine probably reflecting her Yemenite background. Her grammar is another story sometimes it borders on pidgin-Arabic. It's especially clear in her haphazard conjugation of feminine and masculine forms in verbs, nouns and adjectives. Quite strange this, as this follows the exact same patterns in both Hebrew and Arabic dialects.
So what is so shocking about her songs in Arabic? Mostly her choice of words. In order not to be offensive (to the public and the censors alike), most Arabic pop songs use symbolism and euphemisms to talk about relationships between men and women. One example is that many Egyptian pop songs sung by men is usually sung as to a man, even though they are obviously love songs. This should not be interpreted as having anything to do with homosexuality (even though there are several popstars reputed to be gay), but rather a tactic to avoid saying things directly to a woman.
So when Dana is very explicit and even uses swearwords, it comes across as very shocking much like Je t'aime, moi non plus by Serge Gainsbourg and Jane Birkin in the 60s. When reading Dana's Arabic lyrics translated to English try to read them like people would have seen them in the 1950s or 60s to come close to how they are understood in Egypt, Jordan and Saudi Arabia today. Here lies the problem lyrics written by Dana in Arabic for an Israeli (and for our purposes «Western») audience in the 90s have become big risqué underground hits in the Arab world reading them quite differently from us. This has not only resulted in Dana becoming immensely popular in Israel's Arab neighbouring countries but also in a theory seeing Dana as an evil Israeli conspiracy to corrupt Arab youth! More of this on my Dana International Conspiracy page.
One peculiar point about her Arabic lyrics is that they are transcribed in latin letters in her lyric sheets. I would not have expected them to be written in Arabic (Chas ve-Chalila!) as very few Israeli Jews are able to read or write Arabic. (Even though a surprising number can speak good Arabic often a lot better than their English.) Arabic, though Israel's second official language (and English and Russian are not official languages), has only been compulsory for Jewish Israeli schools for a decade or two and is frankly not the most popular subject. But it seems a bit strange that they are not transcribed in Hebrew the two alphabets are very similar. Even though they look very different they have the same origin, structure and letters (with a few exceptions).