At the Algemeiner today, I address the just released Israeli-Palestinian School Book Project. Since posting I have gained further clarity and focus on problematic features of the project and the information about it released to the press.
About the number of books and items “analyzed,”
The official list of books included those approved by the Israeli and Palestinian Ministries of Education for 2011. The study examined school books used in the Israeli State secular and Religious tracts and from independent ultra-Orthodox schools. Palestinian books were the Ministry of Education’s textbooks used in the West Bank and Gaza Strip, and a small number of books from the few independent religious schools (Al-Shariah) when relevant to study themes. A total of 640 school books (492 Israeli books and 148 Palestinian books) were reviewed for relevancy to study themes, and content in the 74 Israeli books and 94 Palestinian books with most relevance was analyzed in detail. The researchers analyzed more than 3,100 text passages, poems, maps and illustrations from the books.
So, indeed, as I raise to question at the Algemeiner, why was there purposeful selection of textbooks from ” independent ultra-Orthodox schools,” but apparently – there is no reference – no comparable selection from Hamas-controlled schools? Why was the original selection of books weighted 3 to 1 toward Israeli books? What were the specific terms of the basis, determined by whom, of “most relevance” what constituted “relevance” that determined the choice of the final 74 Israeli and 94 Palestinian books?
The analysis examined 2,188 literary pieces from Israeli books and 960 from Palestinian books.
Why is there a more than 2 to 1 preponderance of Israeli literary pieces? Would this not provide a more than double opportunity for the detection of passages that might be “analyzed” as “negative”? What rationale is there for not working from equal databases?
Why, apparently, were no Arabic textbooks from Israeli Arabic schools included in the study? (What might it reflect on Israeli society and education if these books were notably free of “negative” depictions of the “other,” however the “other” might complexly be conceived in this circumstance?
A total of 670 literary pieces were analyzed independently by two different research assistants. Statistical analysis demonstrated high inter-rater reliability, meaning that two different raters independently evaluated the same poem, passage of map in highly similar ways.
How were these 670 pieces selected from the 3148 noted above? What was the reason and basis for this further selection?
I have placed in quotation marks around my own use, referencing the report’s use, of the word “analyze” or “analysis.” The report makes significant claims to scientific rigor. However, the analysis of a chemical compound is not the same as the analysis of a text, even if one attempts to subtract human subjectivity from the text by disregarding its truth value. (And was it a stipulated criterion to disregard truth value in determinations of negativity? As, I argue at the Algemeiner, this is indefensible and produces unavoidable and potentially dramatic distortion of the results.) And we are told above that “two” – only two – different research assistants analyzed the 670 pieces. Two analysts of negativity unrelated to truth. Did the study provide them with a list of specific verbs, adjectives, figures of speech and idioms the use of which were automatically to be designated negative? Was there no subjective, critical allowance for judgment beyond such a list? From what environment did the research assistants come? Were they already employed by, students or teaching assistants of the lead researchers who shared, perhaps, their predisposition toward the study’s outcome?
The press release states,
The study engaged a Scientific Advisory Panel that resulted in the worldwide collaboration of 19 experts, including textbook scholars, social scientists and educators from across the political spectrum of both Israeli and Palestinian communities. The advisory panel includes textbook researchers from Germany who led Germany’s self-examination of their textbooks in the decades after World War II, and U.S. scholars who have themselves analyzed school books in Israel, the Arab world, and the former Yugoslavia. The advisory panel reviewed every aspect of the study and agreed on the findings.
However, departing from this account, Eetta Prince-Gibson at Tablet reports,
Several Israeli members of the SAP dissented. According to a memo provided by the Education Ministry spokeswoman, Professor Elihu Richter of the Hebrew University said that “questions remain concerning definitions of the variables, how they are classified and measured and counted and what materials are included and excluded.” Richter warned that some of the comparisons may be “sliding down the slippery slope to moral equivalence.” SAP member Dr. Arnon Groiss, author of a separate study on Middle Eastern textbooks, wrote that he has severe reservations about the methodology and that some 40 significant items, which attest to incitement on the part of Palestinians, were not included.
Further, Groiss has now released this lengthy and instructive analysis and commentary on the report. He states,
Again, we, the SAP members, were not involved in the research activity.
Moreover, it was only a few days before the February 4 release of the report that I was first given the 522 Palestinian quotes for perusal. Having compared them to the quotations appearing in other research projects, I realized that some forty meaningful quotations, which other researchers in former projects, including myself, incorporated in the material and used them in forming their conclusions, were missing. [Emphasis in the original]
I have found deficiencies on both levels of definition and actual use. On the first level, categorization was restricted to very general themes, leaving out important issues such as open advocacy of peace/war with the “other,” legitimacy of the “other,” etc.
There is no attempt to study the quotes more deeply and draw conclusions. All items were treated equally, with no one being evaluated and given a more significant status that the other. It seems that they were simply lumped together, counted and then the numbers spoke. It might be statistically correct, but, as we all know, statistics not always reveal the actual complex picture. This kind of analysis has produced a “flat” survey of the quotes, without any reference to their deeper significance (for example, looking at a demonizing text with no specific enemy as if it were a “neutral” literary piece). Also, all quotes were treated as separate items with no attempt to make a connection between two quotes or more in order to reveal an accumulated message (for example, concluding from the connected recurrent mentioning of the need to liberate Palestine, and the similarly recurring theme that Israel in its pre-1967 borders is “occupied Palestine”, that the liberation of Palestine actually means the liquidation of Israel).
A full reading of Groiss will be instructive for the non-specialist. Its education is two-fold and contrary. First, one recognizes how complex is the activity of attempting to bring something approaching objective scientific rigor to the non-literary analysis of texts. The kinds and range of issues to consider is impressive in variety and complexity. But a mirror principle automatically arises from that condition – that all this complexity in conceiving and formulating the field and terms of analysis bespeaks just that subjectivity of which Groiss offers so many dissenting views, a subjectivity that should give pause on the level of a foot-pedal brake before one reaches with too grasping hands for the label of science.