The Effect of Directionality on Cognition in Right-to-Left Languages

Abstract

What we know
Much of graph perception relies on conventions that seem standard and intuitive. One such convention is that numbers on a graph increase from the bottom up and from left to right. This left-to-right directionality agrees with the left-to-right writing systems used by most of the world's population. However, around 8% of the world uses right-to-left (RTL) scripts such as Arabic and Hebrew. Previous research has shown that RTL people perceive graphical information differently from their left-to-right (LTR) counterparts.

Nevertheless, no empirical study tests how this perception differs when an RTL reader sees a graph. Here, we present a study performed with English, Hebrew, and Arabic speakers in which we measured the RTL effect on visual perception. We also provide some guidelines to mitigate the challenges that we discovered.

What we did
We constructed a series of visual stimuli to test the differences in the perception of LTR and RTL audiences. The interpretation of some of these stimuli does not depend on horizontal direction (e.g., which circle is larger). Other images require the observer to determine whether a sloped line increases or decreases. The participant needs to decide where on the graph the line begins to do so.

We constructed a website that allowed us to expose volunteers to the stimuli while asking them to select one interpretation out of three. We recorded the volunteers' demographic data, responses, and response times. We hypothesized that if RTL people are "confused" by their primary reading direction, we will see a larger discrepancy and slower response times than their LTR counterparts.

What we found
We collected responses from 45 English-speaking, 86 Hebrew-speaking, and 11 Arabic-speaking volunteers. We couldn't find any difference between LTR and RTL volunteers in direction-insensitive stimuli. However, the responses of RTL participants were less coherent than those of the LTR responders. Due to the lack of control over the participants' environments, we couldn't reliably measure the reading direction effect on the response speed.

What we conclude
In this empirical study, we demonstrated and quantified the difference in perception of horizontal direction in populations whose primary reading direction is right to left. Our results suggest that when other factors are equal, RTL readers (about 8% of the global population) will need more effort to decide the direction of a horizontal graph. We suggest mitigating this phenomenon by adding multiple visual clues of the horizontal axis directions. These clues include adding an arrow to the axis and providing numbered axis ticks.

Keywords: RTL, data visualization, pre-attentive attributes, diversion

Introduction

Data visualization is important in technical and academic fields and in day-to-day interfaces such as machinery and computers. It surrounds us, from automobile dashboards to financial documents and news sites. It is also essential in studying and communicating data-intense topics. Due to limited attention spans and perceptive capacities, charts must be precise and easily understandable. Otherwise, the author risks being misunderstood or ignored.

It is widely accepted that significant parts of how we perceive visual information are due to visual processing that happens without our conscious knowledge, called "preattentive attributes" (for example, see Albustin, Andreas, et al. "Pre-attentive processing." Course material. Graz University of Technology (2010)).

While most of the world's population reads and writes text from left to right, a significant portion reads and writes from right to left (RTL). Since reading is a central part of information consumption, the difference in reading direction may lead to variations in visual perception. As we will show below, these differences are very prominent in how a person analyzes graphs with temporal information. This study aims to find empirical evidence of the effect of reading direction on the perception of temporal information in graphs.

We will begin our discussion with a short overview of data visualization and what is already known about the unique characteristics of people who use RTL writing systems. We will then outline our study, present our results and discuss their implications.

On the Importance of Data Visualizations

Scholars have been using visual elements to summarize complex topics since the dawn of the scientific age. For example, figure 2 shows a graphical depiction of planetary observation data recorded as early as the tenth century BC. As time passed, attempts have been made to standardize data visualization and explore the theory behind it. Willian Playfair made one of the earliest attempts to formalize data visualization in his 1798 work Lineal Arithmetic (Playfair. Lineal Arithmetic Applied to..., 1798.).

Figure 2 Planetary movements shown as cyclic inclinations over time, by an unknown astronomer, appearing in a 10th century appendix to commentaries by A. T. Macrobius on Cicero’s In Somnium Scripionus. Source: Funkhouser (1936, p. 261).

When we consider the goal of data visualization, we may divide this field in two, partially overlapping regions: exploratory and explanatory visualization tasks. Exploratory data visualization aims to aid the researcher in understanding the problem they are trying to solve. Explanatory data visualization aims to help a person explain or convince the reader of a particular idea. Vanessa Echeverria and co-authors extensively discussed the difference between the two areas in a paper published in 2018 (Echeverria, Vanessa, et al. "Exploratory versus explanatory visual learning analytics..." Journal of Learning Analytics 5.3 (2018): 72-97.).

There is a great body of research exploring how we, humans, pay attention to visual information and deal with informational overload. Kahneman and Triesman suggested (Wolfe. "Preattentive object files: Shapeless bundles of basic features." Vision research 37.1 (1997): 25-43.) the existence of preattentive information attributes. According to this theory, which was later corroborated several studies, preattentive attributes are loose collections of basic features that allow us to make decisions about the information we perceive in a shorter period than the minimum required to pay attention.

For example, when we look at Figure 3A, we notice that it is a black-and-white image composed mainly of circles. We also immediately see the bright circle in the middle of the figure. We do so since the contrasting bright color is a preattentive attribute of Figure 3A. Similarly, we immediately notice the bright triangle in Figure 3B. However, most readers will need some effort to spot the hexagon on that same picture.

Figure 3: Examples of preattentive attributes. Note how the hexagon in figure 3B is hard to spot.

Graph composition doesn't only affect our ability to spot elements. It also affects what we think of the information presented in a picture. For example, most people will perceive the leftmost square in Figure 4 as brighter than the rightmost one, even though all the squares in this figure have the same shade of gray.

Figure 4 Effect of contrast

The importance of preattentive attributes comes into play in a shortage of time. Given enough time, guidance, and determination, most people will be able to correctly identify all the relevant features in the examples mentioned above. However, in the age of attention economy and informational overload, graph creators are competing with distractions. This limit of attention span can lead to misunderstandings or misguiding interpretations of data.

Temporal data and the time direction

An essential form of data visualization is the type of visualization that depicts temporal data, which are concepts that depend on time or a time-like variable. Most commonly, these visualization types use the horizontal axis to represent the progress of time. The direction of time here becomes crucial for the correct interpretation of a graph. For example, the axes in Figure 5 lack numbers or arrows. Determining whether the depicted line increases or decreases requires first determining where the line starts.

Figure 5 Is this line increasing or decreasing?

Written text is a form of graphical representation of information. This information has an explicit temporal component, as it defines the order in which it needs to be read and written. If the horizontal dimension represents a form of progress or time, the task is to decide which direction the time progresses.

The majority of human populations use left-to-right (LTR) writing systems. Therefore, it is not surprising that most temporal graphs choose the same direction for time. This choice isn’t that trivial for people who write and read from right to left.

Perception differences between RTL and LRT people

Some languages (such as Arabic, Farsi, and Hebrew) are written exclusively from right to left, while other languages (such as Hindustani, Swahili, and Kurdish) use both LTR and RTL writing systems, depending on usage circumstances. Overall, around 600,000,000 people, or 8% of the human population, live in countries that officially use a right-to-left language. Should graphs designed for the RTL audience encode the temporal dimension in the LTR or RTL direction? What about other graphical depictions of time and progress, such as logos and illustrations? Due to the preattentive processing, this decision may affect the accuracy and speed of consuming such information.

Since reading and writing are so central to everyday activity, researchers have been studying the differences in visual perception between people who read from left to right and those who read from right to left. For example, in a 1987 study, Vaid and Singh (Vaid. "Asymmetries in the perception of facial affect..." Neuropsychologia 27.10 (1989): 1277-1287) examined the perception of asymmetric composite faces and noted that LTR but not RTL readers demonstrated a significant preference for the left hemifield. In another study, Chokron and De Agostini [ref https://www.sciencedirect.com/science/article/abs/pii/S0926641000000215] presented Hebrew (RTL) and French (LTR) readers pairs of images and their mirror reflections. The authors noted a strong connection between the subject reading direction and the preferred image directionality. In a different study, Chokron and De Agostini presented 120 adults and children who read Hebrew and French with a line bisection task (Chokron. "Reading habits and line bisection: A developmental approach." Cognitive Brain Research 3.1 (1995): 51-58.). Interestingly, this study demonstrated a significant difference in bisecting the line, even before formal reading learning.

Another experiment by Afsari et al. (Afsari, Zaeinab, Jose P. Ossandón, and Peter König. "The dynamic effect of reading direction habit on spatial asymmetry of image perception." Journal of vision 16.11 (2016): 8-8.) exposed participants to various visual stimuli and used eye-tracking equipment to measure the visual fixation point relative to the image center. The researchers examined two groups of participants: native Arabic, Urdu, or Farsi (RTL) speakers who are also fluent in German (LTR), and a group of native German speakers. According to that study, when LTR participants start examining an image, the average fixation point lies approximately 10% to the left of the image center. The initial fixation point of RTL participants lies 5% to the right of the image. It took about three seconds before the average fixation point of the two groups overlapped. In other words, the study has demonstrated that RTL and LTR readers look differently at graphs and images for as long as three seconds, which is a significant period of time in cognitive terms.

Current state of data visualization in the RTL environments

There is no standard approach toward the time direction in data visualization in RTL languages. The images below show several examples of graphs in Arabic and Hebrew. These graphs come from textbooks, formal reports, and websites. Note the lack of uniformity. The example in Figure 6 from the Jordanian Edraak platform is especially interesting. Note the logo at the upper right corner of the figure. This logo forms an arrow, which we assume represents progress and thus points “forward” and up. Since the primary language of this organization is Arabic, the arrow points from right to left. However, the progress that the graph tries to show happens from left to right, thus causing a contradiction.

Figure 6 Ad example from Jordan, with X-axis representing years running from left to right, while the logo in the upper-right corner indicates progress in the RTL direction.

Figures 7 and 8 show examples of charts in Arabic and Hebrew respectively, where the horizontal X-axis shows time progressing from left to right. Figure 9, on the other hand, demonstrates an RTL language chart with time progressing from left to right.

Figure 7 Graph example in Arabic from Israel. The line presented here increases. The time on the X-axis runs from right to left.

Figure 8 A screenshot of a Hebrew graph that depicts the number of diagnosed and recovered COVID-19 patients in the Israeli city of Carmiel. Note how the X-axis is oriented from right to left.

Figure 9 Screenshot of a graph from a Hebrew website (link). The time flies from left to right.

In several cases, the graphs are tightly coupled with the text they contain. For example, Figure 10 shows a Sankey diagram with Hebrew text. Sankey diagrams have a time-related variable encoded as an X-axis. Since the text within the diagram uses an RTL script, the direction choice of the graph in that example looks more natural.

Figure 10 Screenshot of a Sankey diagram with Hebrew text. The overall direction of this graph corresponds the overall direction of the text, namely RTL. Origin (link)

Methods

In this section we describe the experiment designed to measure differences in visual cognition in RTL languages.

1.5 Stimuli Types

For the purpose of our experiment we classify visual stimuli according to their sensitivity to spatial organization. Namely, we classify images as being direction-free (F) or direction-sensitive (S). Examples of such stimuli are presented in Figure 11.

Figure 11: Stimuli examples. Nine stimuli examples. The first two rows are direction-free (F) and the last row is direction-sensitive (S).

Direction-free (F) stimuli are ones whose interpretation does not depend on horizontal reading direction. For example, visually assessing the height (row 1 of Figure 11) or size (row 2 of Figure 11) of a shape does not depend on whether a viewer scans the image from right to left or left to right. Similarly, detecting a horizontal line (row 3, left of Figure 11) doesn’t depend on reading direction either. However, determining whether a non-horizontal line is ascending, or descending is not as simple and depends on what a person considers to be the line beginning. Note that the charts in Figure 11 don’t have any markings on their X-axis. This was done intentionally, forcing the viewer not to rely on additional information.

The vertical position stimuli contain two circles (row 1 of Figure 11). The left-side circle was filled (“black”) and right one was empty. The only difference was the relative vertical position of the circles. We asked the participants whether the black circle was above, below, or at the same height as the white one. Thus, this family of images can be divided into three categories: "up", "down", and "equal".

The relative size stimuli contain two circles (row 2 of Figure 11). As with the height stimuli type, we placed a filled circle at the left side of the image, an empty circle at the right side of the image, and we asked whether the black circle was bigger, larger, or of the same size as the white one. Thus, the size family of stimuli can also be categorized into three categories "up" (i.e. bigger), down (i.e. smaller), and "equal".

The last family of stimuli, the one where the participant is asked to determine the overall direction of a line, contained only a black line. Since we were only interested in the way the participants perceive the slope of a line, we wanted to remove any clues that will affect the answer. We omitted a vertical axis as we assumed that the location of the vertical axis will provide a clue of where the horizontal axis begins. If we adopt the left-to-right perspective, the lines in our stimuli (row 3 of Figure 11) were either increasing (up), decreasing (down) or kept their vertical position.

When generating the images, we made sure that the differences were obvious to the observer by setting minimal slope angles, as well as height and area ratios.

Study Outline and Stimulus Sequence

Figure 12 shows a schematic representation of the study outline. To perform the study, we constructed a web site that is capable of presenting a user with stimuli and questions framed in one of several languages. We used English, Arabic, and Hebrew, and jsPsych to present the questions and record user responses and the time needed to answer the question. The first screen a user sees is a three-language message that asks them to select the language they are most comfortable reading. After the user selects a language, all the subsequent screens use only the language the user has selected. We did so in order to prime the user with the language they have selected.

Figure 12 Study outline.

Next, we ask the participants to provide some basic information about themselves, such as the gender, age, country of residence, languages they know, etc. We did not collect any personally identifying information such as names or email addresses.

Next, we present the participant with two series of direction-free stimuli. First, we asked the users to evaluate the relative size of two circles. Next, we asked them to evaluate the relative vertical position of two equally sized circles. Each such phase consisted of nine stimuli in a random order. Finally, we show the participant a series of black lines, and ask them to decide whether the line is increasing, decreasing, or level. We expose the users to the three types of questions (area, vertical position, slope) to make sure they get used to the interface, and that the subsequent decisions are mostly influenced by the visual perception, and not by the interaction with our site. The information obtained during this experiment phase is not used for making any direct inferences. However, we use this information as a prior distribution in the subsequent modelling process, which is described in the following section. We refer to this phase as the build-up or calibration phase.

After completing the built-up phase, we show the user a random series of twelve stimuli that are a combination of all three stimuli types. We made sure each type of stimulus appears an equal amount of times in this sequence of images. We refer to this last experiment segment as the collection phase.

Every time we expose the participant to a stimulus, we record the time needed to provide an answer, the image which we showed to the person, the question, and the answer.

Metrics

Perception accuracy

We created each stimulus to have one of the three “directions” – up, equal, and down. Correspondingly, study participant can provide one of the three answers: up, equal, and down. We used the Bayesian approach to estimate response accuracy of the different study groups (Salvatier J, Wiecki TV, Fonnesbeck C. 2016. Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2:e55).

Dirichlet distribution (DD) is a multivariate generalization of Beta distribution and is well suited to model choice probabilities. Our job is to estimate posterior distribution of P(R_r│D_d ), where R and D are the response and the true direction respectively; and r,d∈{up,equal,down} represent the response and the stimulus directions. Since each true direction is represented by a single DD vector, P(R_up│D_d )+P(R_equal│D_d )+P(R_down│D_d )=1 for each d.

The first step in Bayesian estimation is to select the prior distributions. We started with the naïve uniform distribution (i.e. all the parameters in the Dirichlet distribution equal to 1.0), where each choice is equally likely. Next, we collected the responses to direction-insensitive stimuli that we collected from all the participants during the calibration phase and used it to update the DD parameters. We used the updated parametrized DD as the prior distribution of our subsequent computations.

Finally, we used the information that we collected during the collection phase to estimate the distribution of P(R_r│D_d ) in the different populations that participated in the study, namely the participants who selected English, Hebrew or Arabic languages. We further split our analysis into two orthogonal groups: one for the direction-sensitive stimuli, and one for direction-insensitive ones. We use the resulting distributions to make inference on how likely the participants who are used to different reading direction to get confused while estimating whether a diagonal line represents an increasing or decreasing trend.

Response Time

In addition to collecting answers to our questions, we also recorded the time need by the participants to provide the response. Our initial intention was to check whether the RTL participants need more time to analyzed direction-sensitive stimuli. However, the participants filled in our questionnaires in their own time, in different places, and on different devices, including tablet computers and mobile phones. Thus, we found the analysis of the response time unfeasible. Nevertheless, we provide the response times as auxiliary data to this document, and we call the scientific community to study this aspect in a more controlled environment.

Results

Overall, 142 volunteers completed our survey. Responder distribution, as well as some basic demographic metrics, are summarized in Table 1.

Group	N	♂/♀/Other[%]	Age [Years ± Std]
English	45	51.1/46.7/2.2	34.2±13.9
Hebrew	86	30.2/58.1/11.6	37.1±18.1
Arabic	11	54.5/27.3/18.2	22.2±12.9
All	142	38.7/52.1/9.2	35.1±17.0

Table 1. Participant summary. Dot color in the Age Distribution column denotes responder’s gender: blue for male, red for female and black for other or unreported gender.

Response accuracy

We show the probability distribution of the response confusion matrix on Figure 13 (for direction-free stimuli) and 14 (for direction-sensitive stimuli).

We designed our stimuli in such a way that any stimulus that is not supposed to belong to the “equal” category is prominently seen as such. We even added a proper clarification to the explanation screens. Therefore, we assume that marking an “equal” stimuli as either “up” or “down” probably results from a random mistake. Similarly, marking an “up” or “down” stimulus as “equal” also corresponds to a random error. Therefore, we designate the corresponding cells in the confusion matrix with a gray background. Response cells that are most relevant to our study are the “up-up” and “down-down” for correct responses; and “up-down” and “down-up” for errors that might stem from a systematic difference in direction perception. Thus, we mark the correct cells with a blue background and systematic errors with a red one.

Figure 13. Similar probabilities of correct answers in direction-insensitive stimuli. Odds ratios (log scale) for correct stimulus classification. Each row represents the actual stimulus direction. Each column represents participants' response. The dots mark the maximum posterior estimation; the horizontal lines show the interquartile range of the rate estimation.

Figure 14. Less accurate responses by Hebrew and Arabic participants. Odds ratios (log scale) for correct stimulus classification. Each row represents the actual stimulus direction. Each column represents participants' response. The dots mark the maximum posterior estimation; the horizontal lines show the inter-quartile range of the rate estimation. Note the differences between the English vs. Hebrew and Arabic estimates in the upper left and lower right cells.

The graph matrices in Figures 13 and 14 represent "confusion matrices". To emphasize the effect of the reading direction on graph perception, we combined the responses located in the "mistake" corners of the confusion matrices (i.e upper left and lower right cells). These corners represent "genuine mistake", where a participant completely misclassifies the stimulus presented to them. If we combine the responses in these cells, group the Hebrew and Arabic responders in a single category, we can clearly see that the RTL participants have much higher chances to misunderstand the direction of a sloped line (Figure 15).

Figure 15. RTL people make more mistakes when reading direction matters. Error odds ratio (log scale) for answers that rely on reading direction. In this graph, we combined the Hebrew and Arabic participants to a single group named "RTL".

Conclusions

The results presented here demonstrate that the LTR and RTL audiences are similar in their perception of visual stimuli whose interpretation does not depend on spatial direction (Figure 13). However, our results show (Figures 14 and 15) that an RTL person is more likely to misinterpret the direction of a sloped line; they are more likely to classify an increasing line as decreasing and vice versa. These findings emphasize the need to account for graphical perception differences when designing graphs. With clear labels and axis markings, regardless of the direction in which text is written, the viewer can gather information with much less ambiguity.

However, we postulate that any added labels to a chart may not be dominant enough and to constitute a pre-attentive attribute. If we make that assumption the question remains: What graph directionality is most appropriate for RTL languages? We are not able to come to a conclusive recommendation based on this experiment. Instead, we present two recommendations, and leave it up to the reader and further research to come to a more concrete conclusion.

Recommendation 1: In Favor of Consistency with Global Standards

The X-axis sometimes inverted in RTL data visualizations, as shown in Figures 7 and 8. However, consider a viewer who receives a document written in a language they don't understand. If such a document contains graphs, the viewer may rely on the universally accepted graphical metaphors to gain some information regarding the document's content. An example of such a document appears in Figure 16. The person examining this document who does not understand the language might not be aware of the possibility of inverting the X-axis. If such a graph is inverted, their pre-attentive processing will lead them to the wrong and opposite conclusions.

Figure 16. Illustration of a document that is set in a non-comprehendible language. Without sticking to a standard, globally accepted convention, the viewer can't know the actual direction of the sloped line graph.

Instead of inverting the axis, we propose addressing the difference in visual perception among the RTL audience by adding a small arrow to each X-axis. We claim that the arrow, a globally accepted sign, can serve as a pre-attentive hint to people who need it and does not interfere with the graph perception of people who don't.

Figure 17. Arrow indicates the direction. Example from Figure 16 with an added error serving as a pre-attentive hint about X-axis direction.

Whether such an error helps to reduce the perception gap between RTL and LTR audience that we demonstrated in this study (Figure 15) remains to be determined in a dedicated experiment.

Recommendation 2: In Favor of Consistency with the Native Language

From the examples shown in this paper, it is clear that RTL languages deal with inconsistent graph directionality much more often than LTR languages. There are likely many reasons for this difference; the dominance of LTR languages in modern science and education, the globalization of media and communication, personal preferences, software with poor internationalization support, and the fact that many native RTL speakers are also familiar with at least one LTR language.

However, despite these factors, we continue to observe instances of data visualization with an RTL orientation. This can be viewed as an indication that left-to-right is NOT a preattentive attribute in data visualization, and that the preattentive attribute of graph directionality is likely more correlated with the directionality of the underlying language. That is, if you are reading from right to left, it is more natural to read a graph that flows in the same direction rather than the opposite.

As a final example, Figure 18 provides an interesting illustration of this relationship. On the left side, we have a graph of per-month values where the X-axis is labeled with numeric months. Many people would argue that this graph feels natural regardless of the direction of the surrounding text. However, if the X-axis labels are replaced with text-based months in an RTL language, the LTR orientation of the graph starts to feel abnormal. The introduction of RTL text directly impacts the perception of directionality in the graph.

Figure 18. Numbers vs. text on the X-axis affects graph perception. The chart on the left with months as numbers seems acceptable, but the same chart with text-based months feels unintuitive.

Based on this observation, one could conclude that adapting to the natural direction of a language is preferable. And of course, clearly labeling the direction of a graph with arrows remains the best way to eliminate confusion, regardless of the directionality of the data visualization.