Micro Visualisations

What is communication?

When one tries to define what communication really is, he quickly comes to the conclusion that everything we do is communication. »Activity or inactivity, words or silence all have message value: they influence others […].« (Watzlawick et al., 1967) As designers, we shape many aspects of this wide field and help other people to communicate the way they wish to. Viewed with an even broader definition of design, humankind as designer has developed a wide variety of communication channels, ranging from body language and spoken language over to written and printed type, and finally to electronically processed communication. Each of these steps marked a milestone in mankind’s history, but none could completely replace previous mediums. Even though some of the utilised technology for each communication channel might have been replaced over time, the basic concept of communication stayed preserved.

There are two reasons for that: First, every form of communication has its own qualities, advantages, and applications at different settings, and second, each development rests upon the former one and therefore relies on the common understanding of the previous one.

To emphasise this relation even further, Paul Watzlawick’s famous quote »One cannot not communicate« (Watzlawick et al., 1967) expands the definition of what communication is. He argues that there is no opposite of to communicate and thus makes »communication synonymous, or nearly synonymous, with behaviour.« (Motley, 1990) This implicates that at least one of the communication channels available to human is always active. It also facilitates the theory of multi-layered communication and thus that a single form of expression is seldom and a single form of impression is impossible. The following example illustrates this: When one reads a text, one is not just reading. He is also seeing how the text is presented. The written, printed or displayed words are only one part of the information that is received. The material, the layout and the many aspects of typography all communicate on their own and thus influence how and what information is perceived from the actual text. Marketing knows that and defines CIs consisting of corporate design, language, sound, smell, and many more aspects that all strengthen the stringent branding of the company.

What can be taken from this is the broad understanding of what communication is. To use this methods not to make money but to impart knowledge to others through a wide variety of means of communication.

What is Typography?

The importance of the appearance of the text is well known and generations of type designers and typographers developed a variety of techniques to design and enhance the »text-seeing« and reading process. It is »the art, or skill, of designing communication by means of the printed word« (Childers et al., 2002) But even before books were printed, a highly elaborated discipline had formed around type design. (Drucker, 2014) Today typography is an inherent and essential part in design education. The variety of type design principles and typographic variables need to be thoughtfully applied to the text to comply with the content but also to be understood by the reader. Some of these principles are conventions, which evolved over time (italic and regular fonts), others are based on fundamental decisions such as reading from one side in one direction (justified, flush left, or centered text), while others base on perceptional reasoning (font size and weight).

Carried by this unique craft, written and later – with Gutenberg’s invention – printed books came to their prominent status they hold today. But the importance of typography is not justified by the decoration of the text, but rather the readability of the text and more fundamentally by the supportive character the design plays for the content of the text. Robert Bringhurst exemplifies this for the typeface: »When the type is poorly chosen, what the words say linguistically and what the letters imply visually are disharmonious, dishonest, out of tune.« (Bringhurst, 2012) To clarify, honour or disguise the content is seen as the most important role typography plays. But with the leverage its application has, it can also be misused: »In a world rife with unsolicited messages, typography must often draw attention to itself before it will be read.« (Bringhurst, 2012) This implies the consequences that typographic decisions have on the visual appearance and hence on the very first impression the reader has of the text. With other words: »Type is persuasive – and thus so is typography […]« (Santa Maria, 2014)

Besides the used material, the selection of an appropriate typeface is the first and most fundamental decision when approaching a text. With the selection of one typeface the designer takes over many decisions that were previously made by the type designer. It may also demand a certain typographic application to suit its appearance. The explanation of the complete process of letter and type design would go beyond the scope of this study, but what is relevant for this thesis is the great impact the selection of a typeface has for the perception of the text. Stronger than any other typographic decision, the typeface affects many aspects of the reading experience. First the readability consisting of reader-friendliness (Lesefreundlichkeit), distinguishability (Unterscheidbarkeit), recognizability (Erkennbarkeit), and legibility (Leserlichkeit) (Herrmann, 2010) and secondly — even more prominent — the connotations that come with the typeface. From a consumer psychology view »typefaces convey meanings that have the potential to significantly influence important marketing constructs. These associations influence how consumers perceive brands, as well as, what they remember about brands.« (Childers & Jass, 2002) Taken the book as a product that represents the text, it becomes clear that we perceive and remember the content differently depending on the chosen typeface.

With this most obvious decision in typography as introduction, further »tools« that are available for typographers are for example margin, line-height, letter-spacing or letter case. A full exploration of these variables is discussed in relation to »visualisation« further down.

What is Visualisation?

The power of photographs has been used for a long time. The alleged objectivity induces publishers to show »the single truth«. The pretended evidence of the image reduces complex matters to single statements. (Bolz, 2012) As mentioned above, nearly every newspaper nowadays relies on the impact a photograph can have for a story. But with this practice problems arise concerning the photographability of certain subjects. (Bolz, 2012) Intangible and abstract problems cannot draw much attention to itself, because they cannot be captured in a photograph or an illustration. This becomes even more apparent in the context of social media, where posts with images reach higher click rates. Articles that do not show their content with attractive images cannot arise much attention. One answer to these undepictable subjects are visualisations, which encode abstract data into tangible graphics. Previously »invisible« and highly complex processes gain shape through abstract graphical elements. Thus visualisations are used for information explanation.

Visualisation is not a new field, but experienced an upturn in recent time. With abstract data becoming ubiquitous the challenge to make sense of this new resource became apparent in all industries. The term big data arised in the hope to find previously hidden insights from the analysis of great amounts of data. Besides classical statistical methods, visualisation is another promising and proven approach. For this purpose data visualisations are used for exploration. The statistician John Tukey already predicted 1965 »a greater and greater role for graphical techniques as aids to exploration and incisiveness« for his subject. (Tukey, 1965)

While the textual analysis of such an amount of data is not efficient, humans are very good at spotting pattern, abnormalities or trends on a visual level. (Few, 2014) Ware goes even further and says »There is no other way of presenting information so that structures, groups, and trends can be discovered among hundreds of data values«. (Ware, 2004)

Norbert Bolz speaks in this context about the concision (Prägnanz) of an image. This quality enables us to instantly see the content. (Bolz, 2012) In order to understand this process, the human sensory system needs to be examined. »The term sensory system includes all phenomena that manifest themselves in the consciousness as result of stimulation of any sensory organ«. (Haverkamp, 2009) This rather broad definition describes (all) the phenomena as something which is actively manifesting itself. This seems to be opposed to the widely established and accepted concept of constructivism, which describes how we perceive our surroundings on the basis of a theoretical model. In philosophy, anthropology and psychology — in contrast to constructivism in art — it describes the following process: When we observe objects, we reconstruct these as mental objects. These ideas are constantly matched to previous experiences and influences. These »images« of objects work as prototypes and can be recalled even with different appearance or after perspective transformation. (Ware, 2004) This implies that the human brain is no longer passive in its perception, but instead active in terms of selection, projection, interpretation and sense-giving: it constructs its world individually. (Maletzke, 1998)

This obviously raises questions regarding the »real reality«, which is a rather philosophical discussion that goes beyond the scope of this work. For designers, this model is interesting in regard of communication. Concepts of mass communications are no longer consistent with this theory. Every human needs to be viewed as an individual, from whom consistent reactions can no longer be expected. What is relevant for the purpose of this work and for the perception of visualisation is the active selection and interpretation of the presented image. Despite this, the passive perception of visual information is equally important and has been scientifically proven as well. The answer to this seeming contradiction lays in the way our visual sensory system works. Simplified it consists of three consecutive stages:

The last two stages involve active search, selection and interpretation of the given visual information where it is important to note that this »active« process is still processed mainly unconsciously. »Consider the eyeball as an information-gathering searchlight, sweeping the visual world under the guidance of the cognitive centers that control our attention. Information is acquired in bursts, a snapshot for each fixation. From an image buffer, the massively parallel machinery of early visual processing finds objects based on salient features of images.« (Ware, 2004)

In order to help spotting salient features, information designer should encode this information to »pop out«. Various studies analysed which encoding works best for which type of data. The first and most cited taxonomy of such kind was formulated by Jacques Bertin. In his book Semiology of Graphics (Bertin, Semiologie Graphique, 1983), the cartographer constrains his studies with some factors like printable, standard sized graphics on white paper. His work was later picked up by many scientists and expanded by unnoticed variables and the ones that were left out intentionally by Bertin, as he is rooted in cartography.

Bertin distinguishes seven forms of encoding. He describes these as »visual primitives« or as they are now commonly called »visual variables«, from which the designer can form the image to suggest perspective, reality, relationships or space. (Ware, 2004) These variables are: position, size, value (transition from light to dark), texture, colour hue, orientation and shape. I propose the following modifications for a better structure:

• subordination of colour hue, value, and texture under appearance and
• replacement of value with opacity, since the transition from light to dark can be treated as change of colour.

This results in the following taxonomy:

Bertin’s visual variables have often been extended, for example by MacEachren (2004) by crispness, resolution, and transparency. Chen and Floridi (2012) extended the list to 30 variables and subordinated them into four categories: Geometric, optical, topological and relational, and semantic channel. While this further definition is helpful it loses the idea of basic visual encodings.

Debatably all variables from the »Topological and Relational Channel« are rather results of the spacial positioning and the variables regarding the motion are also Bertin’s visual primitives combined with time. The concept of visual variables needs a second layer that covers such advanced cases and fill in the gap to the psychological Gestalt principles. While such an approach seems to be promising, it would go beyond the scope of thesis. Since Bertin’s approach still has a broader acceptance I will stick to the previously explained modified version of his taxonomy for this thesis.

Geometric Channels:

• Size / length / width / depth
• Orientation
• Shape
• Curvature
• Smoothness

Optical Channels:

• Intensity / brightness
• Colour / hue / saturation
• Opacity / transparency
• Texture (partly geometric)
• Line styles (partly geometric)
• Shape / blur
• Shading and lighting effects
• Shadow
• Depth (implicit / explicit cues)

One helpful aspect of these visual variables is the ranking in regard to their effectiveness. This can be viewed from different angles. First is the ability to pop-out. Visualisation can help elements to stand out from the rest, making it easier to spot when searching for a certain one. Studies (Borgo et. al, 2013) reveal this order:

The second ability of visual variables can be measured by human’s perception to most easily decode the information »behind« the encoded data point. It is the variables’ ability to be definite. And also the effortlessness of comparison between multiple points varies between the channels. This aspect was also considered by Bertin: For instance, the position of an object along the x- and y-axis (the positioning in 3D spaces is being discussed in the visualisation community controversial) was described as the most powerful for extraction and comparison. This idea of effectiveness was then continued by Jock D. Mackinlay. He provided the sorting in relation to the given data type. According to S. S. Stevens, these data types are »nominal, ordinal, interval and ratio«. (Stevens, 1946) Even though his definition can be criticised, his work set a standard in statistics. For a clearer separation I will distinguish »interval« and »ratio« under »quantitative« data types.

Typography as Visualisation?

»Letters were mainly regarded as a medium of communication; it is essential to recognise them as media of perception.«

 — Sybille Krämer, 2003

Writing as medium are fated in the same way every medium is considered: Aristotle already described that media of perception are indeed materialistic but at the same time need to be transparent. (Krämer, 2003) Fritz Heider continues this concept as he states that media make something visible which is not »of the ’nature‘ of the medium itself«. Media »aisthetisis« by »un-aisthetising« themselves (»Medien aisthetisieren, indem sie sich selbst an-aisthetisieren«). (Heider in Krämer, 2006) The media philosopher Sybille Krämer examines this relation between visibility and invisibility. In her ten theses about »notational iconicity« (Schriftbildlichkeit) she postulates the presentative instead of the representative perspective on letters. (Krämer, 2003)

The blurring of boundaries between presentation and means of presentation is also subject to the concrete poetry. Mostly through typographical arrangement of words »poems of figures« emerge. (Bauer et al., 2010) These figures raise questions on the similarity of the meaning and the optical appearance of the words. The distinction between figure and text can no longer be applied to this art form. In Eugen Gomringer’s poem the word »schweigen« (to be silent) has no similarity to the idea it represents, but illustrates its meaning through the arrangement.

In a similar way, my thesis highlights typography as a mean of communication for the text’s content and not only to serve as a tool for readability. To illustrate this problem in application: Journalists publish their articles in a newspaper. However, the typographic decisions were made long before the article was written by someone uninvolved in this particular or in every other article. The appearance of the text represents the newspaper, but not the text. Regulations on the font, the columns, use of emphases or layout may not be beneficial for the text.

In order to examine typography for visualising information, I discuss typography from a data visualisation’s perspective. The description of visual attributes like the ones by Bertin, Cleveland, Ware or Meirelles summarise typography as one visual variable. My hypothesis is that typography is not a subcategory like »size«, but rather a complete new »object« that can be modified with the visual variables described by Bertin and others. When comparing the visual primitives with basic techniques used in typography, one can see that these match in most cases concerning their implementation and effect. For example, a word can – just like a square – be scaled in size or changed in its position. But some major points need to be considered:

• In typography there are even more visual variables available due to conventions like underlining or line-height.
• It depends on whether it is applied to the whole paragraph, the sentence, the word, or only the letter.
• Some of the »typographic variables« cannot be used without compromises on the legibility. In contrast, a rectangle in a regular visualisation may suffer in visibility but not in legibility.
• In fact this is a complete new relationship between entity and label. Normally the entity is modified, while the label stays unaffected. In my approach the label is also the entity and thus requires careful modification.
• Also, the initial situation is different since in regular visualisations each object represents one (or multiple aggregated) data point. There are no objects on the plane that look like data points, but hold no data. When applying visualisation to only some words or text elements, the other words remain in their original form and can act as »normal« for comparison.
• The modification of any of the typographic variables influences the grey value (the perceived blackness from a distance) of the text. Sometimes, this is not intended. The change of the typeface for instance might hence not only have a semantic alteration but also a visual effect.

The table illustrates the typographic instruments for each form of encoding.

Shape

• Typeface
• Case / Style
• Weight
• Width
  
  

Size

• Scale

Shape

The shape of the letters cannot be mapped directly to a single typographical variable. There are multiple options to vary the font’s shape. The most radical would be the actual typeface. This categorical alternation should be treated with caution. Two major concerns need to be taken into account:

• The distinction of different typefaces can be hard for the laymen. Serif, sans-serif, script, blackletter and decorative fonts might easily be distinguished, but within these categories non-specialists could hardly separate Renaissance, Baroque or Neoclassical.
• Another thing to be concerned of is the inherent cultural association of that font. Blackletter fonts are commonly (and falsely) associated with the Third Reich, serif fonts with the antique and script fonts with a humanly touch. As these associations are mainly personal and individual impressions instead of facts, one should be careful when expecting the reader to link the contrasty, vertical orientation to an 18th century modern Romanism.

Another way to alternate the shape of a font is either its case (lowercase or uppercase) or its style: italic (and rarely left oblique), small caps and roman. This has been common practice since the sixteenth century (Bringhurst, 2012). This feature can only be used categorically since there is commonly no option of more or less italic, for example.

The third option is to use different font-weights. Depending on the amount of fonts the typeface consists of, the font weight is a feasible variable to visualise information. The weight of the font also influences the grey value of the text accordingly and is therefore – just like the colour value – able to communicate its value from first sight and from a greater distance by »popping-out«. In contrast to the shapes from regular visualisation, different font weight do have an order. The examples given by Bertin (rectangle, star, triangle, …) an not be ordered.

Another option is the width of the fonts. Some typefaces consist of extra fonts like condensed, narrow/compressed or extended ones. These fonts could be equally used as different weights, but since they are rather seldom (especially for antiqua typefaces), real world applications would be rare. Fonts should not be streched out programmatically in order to preserve their aesthetic. This is also why the width is not considered as size, but as shape.

One great possibility would be Multiple Master Fonts. These fonts can continuously interpolate between two »masters«. Possible axes can be weight, width, optical size and even style. With this technique individual interstages can be generated. Unfortunately, MM fonts could not establish itself and are hence not supported on current systems. Recent developments on this topic are JavaScript-based solutions such as Plumin.js, which allows to generate/manipulate fonts with JavaScript code. Prototypo is another approach that allows font manipulation with type designer variables like x-height, capital height, ascender and descender height, slant, overshoot, crossbar position, and width. Opentype.js even allows to directly manipulate anchor points. All these tools could potentially be used for visualisation with continuous interstates to represent values when they are further developed for this kind of application. The disadvantage of all of these tools is the purely mathematical generation of the fonts. The perception of letters and thus their design is not linear. This often leads to unpleasant results. Either the generation of fonts becomes more advanced and takes the non-linearity into account or the possible unaesthetic and illegible appearance needs to be accepted. Further development will show if these JavaScript-based tools can manage what the original Multiple Master tools could not.

Position

The position of a letter can be altered in various ways. On the x-axis the designer can change the kerning/spacing (the space between letters), the indention/margin (the space – usually – from the left-hand side) and the general alignment. On the y-axis the user can change the letters to superior/inferior or the general line-spacing.

In regard to single letters the spacing is a good option to communicate both qualitative as well as quantitative values. Despite the fact that exact decoding of the data is not possible, this technique allows for continuous data mapping. Another advantage of this method is also independency from the typeface: The spacing is applicable to every font. Obviously extreme values can cause illegibility or the drifting apart of letters. Shifting single letters in their y-position is already common typographic practise. For example, chemical, physical or mathematical notations use a shift for _sub- and ^superscript to create a hierarchy between characters. What is important to note here is that because of the little possibilities of variation within the line height, the visualisation becomes categorical or discrete rather than continuous. In some cases the reader can only designate bottom, middle or top position instead of exact differences. The comparison is also difficult across multiple lines. As Cairo (2013) describes, objects with a common baseline are easier to collate. This also limits the modified amount of characters to keep the baseline within a line row. Another problem that could cause a lack of comparison on the y-axis is the different size of the letters. The ascenders and descenders suggest different starting points. It is therefore recommended to use capital letters to work with the same letter height.

In general, the repositioning can cause issues on the readability. Each letter, word, sentence and paragraph has an inherent position that is crucial for the content. The variance of the position is therefore limited and needs to be applied carefully. Letters and words are especially bound to their position.

The shift on the x-axis of sentences or paragraphs is already used. Lists in a table of contents for example use different positions to indicate its hierarchy. Again, a modest execution should be applied to display both categorical or quantitative data. One advantage of the continuous mapping is that people can instantly understand the technique.

An unique variable is the text alignment like flush left, justified or centered text. Spreadsheet software take advantage of this by aligning text on the left side, while numbers are aligned on the right side of the cell. Here, the cultural conventions need to be considered.

Appearance

As previously mentioned, colour has the strongest pop-out effect. In the commonly black and white surroundings of normal text, elements stressed with colour draw even more attention to themselves. On one side this can be used for advantage by using it to emphasise certain values, but on the other side it can also distract users from reading. Because colour can communicate information from a greater distance, it is often used in combination with text for way finding systems. While it allows to convey categorical information quickly, quantitative information is even more difficult to decode because of the small surface area. For this is advisable to use multi-hue diverging colour palettes since the size of the font might restrain the conception of diverse colours. When using the same colour on extensive objects and text, it should be applied to the latter with extra saturation or darkness to counteract the small area.

The use of opacity may cause problems for the legibility. Just like objects that do not stand out from the background are hard to be spotted, letters that are too similar to the background cannot be read anymore. Additionally, all changes of opacity (similar to colour) should be stressed especially on lightweight fonts to counteract the small surface area. A change of opacity is often more practicable when combined with a change of colour.

The application of a texture highly depends on the type of texture. Normally, the surface of the font is smaller than typical textured objects like for example countries on a map. This can result in an illegibility of the word and also an unidentification of the actual texture. Especially for small fonts this typographical variable is not recommended. It is worth to mention that from a distance the whole text appears as one image and hence gains a texture.

Size

The use of size to indicate value is an already common practise. Headlines, sub-headlines, body text and foot notes often use different font-sizes to indicate hierarchy. Cartography also makes great use of scaling in order to visualise importance of objects. (Imhof, 2007) But rather than scaling continuously for example the height of mountains or the size of cities, they are often categorised. Since the size of irregular forms like letters is more difficult to compare than the size of two regular squares, the discrete values are rather difficult to extract and should rather be treated as ordinal data.

Just like with the positioning of letters, words and sentences have an inherent value: Words with more letters are longer, capital letters are usually higher than lowercase letters. In the design of maps, inherent sizes often casuse biases. As countries with a greater landmass appear to be more important, »bigger« words also seem to be more important. This problem can be often seen with word clouds, which scale words for example in regard to their appearance. When encoding with size, these points should be taken care of:

• Proportion As mentioned above, the width and the height of a font should not be individually scaled to preserver their legibility and aesthetic.
• Word’s length Since the number of letters is the main influence on the word’s length and hence its initial size, it is advisable to use words with the same letter length whenever possible. For this, either representative abbreviations, shortcuts or codes can be used. For example, the ISO 3166 country codes can be used when dealing with countries, words can simply be shortened with an ellipsis or replaced with an acronym. Monospaced fonts also help to compensate varying letter width.
• Letter’s height Capital letters can also help when encoding with size. Since capital letters often have the same height it also makes them easier to compare as they have an even initial size. They also have a common starting level as they have no ascender or descenders.
• Minimum size Obviously, the scaling should have a minimum value in order to maintain the readability of words. But this might also be a problem: When the zero-point is at some minimum text size, the comparison can become difficult, as this minimum size needs to be subtracted from the total size.

Summarised, one should use the size when

• comparison does not need to be exact and
• when none of the underlying data falls below a value that would result in unreadable text size or
• when the given words vary too much in their initial size.

To maintain legibility, it is also recommended to always scale width and height accordingly and not individually. When only the width is changed designated fonts like compressed, condensed or extended should be used.

Besides the scaling of the words, the size could be applied to the width or height of the »paragraph box«. When the text is long enough, the text boxes can imitate the usage of bar charts, where the line-length corresponds to the value.

Orientation

Different orientations can hardly be used without compromises on the text flow. While complete sentences or paragraphs can be read with varying slopes, letters or words should only be carefully rotated within sentences. Correct applications can often be found in cartography, where letters follow the path of a river or a word adheres to the coastline. In the same way angles are difficult to extract from objects, the slope of a word cannot be decoded without a doubt. Also ascenders and descenders can increase this difficulty.

Additional typographical variables

Over the course of hundreds of years, typographic conventions were established that are unique in graphical distinction. These are often additional graphical elements that can distinguish words. For example, the underlining of a word is clearly highlighting that word, while underlining a rectangle would not be directly understood as highlighted. With the Web’s growing size, significance and level of access the underlining of text became a semantic meaning: being a hyperlink. Especially in combination with a blue or purple text colour the association is widespread among Web users.

The highlighting of text passages also works with characters added at the beginning and end. This way markup languages *highlight* words or declare <div>boxes</div> of content. (But also in regular usage parentheses indicate a devaluation.)

Conclusion

Summarised, typographic variables contain parallels to visualisation variables and can be used as such within limits. Always in consideration of the legibility, typographic distinctions need to be applied more carefully than in regular visualisations. In the same way visual variables are also not equally suitable for every type of data, some typographic variables can used for qualitative values only.

While the position is widely regarded as the best way to encode in »normal« visualisations, it needs special treatment with text in order to keep the reading order and flow. »Weaker« visualisation variables like colour appear to be more useful because firstly, they do not displace the text corpus and secondly allow for comparison across multiple text lines, which is difficult with a change of x-position for example.

Typographic conventions can be beneficial, but need to be treated with respect for cultural inherent meanings. In general the proposed typographic visualisations are not »as strong« as regular visualisation. The extraction of exact quantitive data is never as good as with graphical means. This is mainly due to the lack of a coordinate system that would give the user an orientation and to the distribution across multiple lines that hinders comparison.

In conclusion, the visualisation with typography is mainly suitable for ordinal and nominal data. Quantitive visualisations should only be used in a supportive role.

Why adhere to text?

With visualisation as the seemingly solution to all modern problems, the question arises why one should stick to text. The answer to this is based on the cultural imprint by the book. As previously stated it had a tremendous effect on not only how knowledge is distributed but also in the way we think. Habits formed conventions and today most people are used to obtain information from written text with its discursiveness, linearity and abstraction. Mendelssohn bewails how this has made us literati:

»We teach and instruct one another only through writings; we learn to know nature and man only from writings.
We work and relax, edify and amuse ourselves through scribbling. The preacher doesn’t talk to his congregation; he reads or recites to it something he has written.
The teacher reads his written lectures from the podium. Everything is dead letter; the spirit of living conversation has vanished. […]
This has brought it about that man has almost lost his value for his fellow-man. We don’t try to meet with the wise man because we find his wisdom in his writings. […]
In short, we are literati, men of letters.«

 — Mendelssohn, 2011

»Wir lehren und unterrichten einander nur in Schriften; lernen die Natur und die Menschen kennen nur aus Schriften; arbeiten und erholen, erbauen und ergötzen uns durch Schreiberei; der Prediger unterhält sich nicht mit seiner Gemeinde, er liest oder deklamiert ihr eine aufgeschriebene Abhandlung vor.
Der Lehrer auf dem Katheder liest seine geschriebenen Hefte ab. Alles ist toter Buchstabe, nirgends Geist der lebendigen Unterhaltung. […]
Daher ist es gekommen, dass der Mensch für den Menschen fast seinen Wert verloren hat. Der Umgang des Weisen wird nicht gesucht; denn wir finden seine Weisheit in Schriften. […]
Mit einem Worte, wir sind litterati, Buchstabenmenschen.«

 — Mendelssohn, 1989

I do not to want people to turn away from books as Mendelssohn’s text might suggest, but rather stress how many fields are framed by the book. Even today where the production of images is effortless as never before, text does not disappear. Instead, new forms of text »types« evolve: Comic books, Instant Messaging and Twitter. In contrast to the pessimistic view of some writers, we should utilise these tools for they are still forms of expression based on textual communication. While some describe the printed book as »the perfect reading-machine« (Die perfekte Lesemaschine, Reuß, 2014) I personally see much potential for new forms of printed books, but even more for interactive, responsive, illustrative ebooks or any form of textual expression.

But the triumphant advance of text has not only to do with habituation. The linearity, the discourse and the abstraction might bring restriction but it also brings advantages over the visual communication. While images allow the viewer to freely explore the content, text guides the reader through the information. The linearity brings order, the discourse allows to follow argumentation. And with the ability to communicate abstractly, text often allows to communicate intangible ideas.

For this thesis I use the prominent status and the qualities of the text as starting point and enhance its comprehension through the combination with visualisations.

Why combine both?

These two disciplines – typography and visualisation – are too often treated as two separate things. Both work with visual variables that are applied according to the underlaying data. Where typography adds this data level to the text, visualisation works with graphical elements to communicate the data.

In the book »Understanding Comics«, Scott McCloud illustrates that words and images are two ends on a scale from concrete to abstract. (McCloud, 2001) He also shows how these two ends can interact with one another, each fulfilling its own purpose. He exemplifies this with »The Adventures of Tintin«: The simply, abstract drawn persons help the reader to identify with them, because their appearance is not so determining. On the other side, the detailed, concrete background shows the fantasy world the story takes place in.

But when typography is applied to the abstract text, it becomes more graphical and hence more concrete. On the other side, visualisations make use of graphical, iconic forms and thus make the images more abstract. Expanding McCloud’s model, the new order is: Images — visualisations — visualising typography — text. While most publications make only use of the extremes of the spectrum, my approach invites the designer to use all »tools« of this range to tell a story.

But not only results the combination in a greater variety of storytelling-elements for the sake of cross-mediality, but also helps the reader to understand the content. Albrecht Dürer wrote in his textbook for the adolescent youth already that he wants to combine image and letter to enhance memorability. (»Deshalb will ich das Wort und das Werk zammentan, auf daß mans dest baß merken müg.«) (Wenzel, 2006) Similarly, the brain researcher Wolf Singer states that when our perception is confirmed by two stimulus modalities, the perceived is evaluated as to be the truth. For example when the eye sees an object that is also recognised by another sensory perception, the assumption of what is »out there« is more reasonable. It passes the test of consensus. (Wenzel, 2006) Even though this does not directly justify the use of different encodings within one sensory medium, it paves the way to this research: And indeed the combination of images and words has been scientifically studied by a number of researchers. Most of the results support the theory of »dual coding« by Mayer et al. (1999 in Ware, 2004). »They suggest that if active processing or related material takes place in both visual and verbal cognitive subsystems, learning will be better.« (Ware, 2004) Summarised, this shows how an increased exploitation can be achieved through the combination of text and image.

One study that followed this thesis’ approach from the other side was conducted by Chandler and Sweller (1991 in Ware, 2004). It examined the integration of text directly inside graphics. Their test graphic — a diagram with operating instructions — had text snippets directly placed at the relevant point. Their successful experiment was justified by the »limited-capacity working memory«. »They argue that when the information is integrated, there is a reduced need to store information temporarily while switching back and forth between locations.« (Ware, 2004) This is often justified by the cognitive load theory. As the theory says, our working memory is limited. Hence object and key should be close together to reduce information that needs to be stored temporally in our memory. It may be expected that this concept performs in the same way with images directly integrated into the text.

Other psychological studies on reading comprehension can also be interpreted to support the thesis’ approach. Perrig and Kintsch (1985 in Müsseler et. al, 2002) described a fictitious city first from a bird’s perspective (»north of the highway and east of the river is a gas station«) and secondly from a driving perspective (»When crossing the river, you see a gas station to your left«). Candidates who received the bird’s perspective were less able to repeat the text, but instead could easily draw a map. (Müsseler et. al, 2002) It can be expected to support memorability when representing the information for example in driving perspective as text and from a bird’s view on a map. It needs to be evaluated in further research if this »dual coding« mentioned before also helps when text and image hold different perspectives.

Another reason, why the combination of these two provide additional insight, lays in the frequently given reasoning for visualisation. For this, the Anscombe’s quartet is often used. After presenting tables with numbers, visualizing the same numbers in diagrams seems obvious and convenient. And indeed it is much easier to compare and spot trends and patterns with the diagrams. But one thing is often neglected: The table »expresses […] values precisely and it provides an efficient means to look up values for a particular […]« dimension. (Few, 2014) For some graphics, the underlaying data might even not be extractable at all. For most applications that is absolutely valid: The more data points you have, the less important the single one become. Visualisations’ purpose is to serve for impression, rather than exact numbers. As John Tukey states, it is better to be »approximately right rather than exactly wrong«. (Tukey, 1965) What should be taken from this is that both »visualisations«  — table and graphic — have an unique quality, which are advisable for different settings. And the same holds true for text. Through the combination of these different forms the reader can decide which one suit his interest or ability to extract the data best.

These qualities can be exemplified by uncertainty and diversity. While visualising uncertainty is gaining more attention in recent time (Potter et al., 2012), text can often more easily express this uncertainty. And in other cases, where the verbal expressing generalises and labels information (Drucker, 2014), visualisation can demonstrate diversity.

Each form can compensate the difficulties of the other and can even help to overcome the lack of expression in text in general. As Marc Davis wrote in regard to emojis: »they are best used as an ‘adjunct’ to text — especially in social media — helping to make up for the the lack of gestures, facial expressions and intonation found in speech.« (Stern, 2015) And while this might seem only applicable for emojis, graphic novels show us how anger, fear or gentleness can be expressed through type. Hence, the mixture of multiple sources of information creates »a more complete image«, resulting in a greater exploitation for readers.

The impression an user is given by visualisations can also be used as benefit: Willett et al. describe the concept of Scented Widgets which act as »embedded visualizations that facilitate navigation in information spaces«. (Willett et al., 2007) The idea behind this approach is to give users a preview (a scent) of what they are about to see, when they click on the equipped widget. Because of the simplistic appearance users are able to compare multiple elements of information (or in this case choices) at once.

What needs to be considered is that not all text especially literature »wants« to be equipped with additional graphics. As these visualisations can add the appearance of an analysis and statistical effort, one needs to accept that especially lyrical texts are often not meant to be analysed by mathematical means.

Summarised, departments too often adhere to their medium: Journalists to text, financiers to tables, designers to graphics. Each discipline should more often use the adequate presentation for the message. Or as this thesis shows: A combination.

Why Micro Visualisation?

The combination of text and visualisation can be approached from many directions. Besides the aim to create a balance between the two, the focus could be either on the verbal or on the graphical side. For this thesis I will focus on text with visualisations embedded. There are six reasons for that:

• Based on the previously discussed considerations on the history of communication I have come to believe that text is still human’s main medium of knowledge impartation. And even without the aspect of additional graphical visualisations that is discussed in this thesis, the research to enhance this standard text presentation through typographical visualisations is valuable for this field.
• In regard to the first point I believe that an adduction to visualisation for the general public could be best achieved with simple graphics embedded right into the familiar medium. The simplification is the first step to make them accessible to a broader audience. With changing reading conventions and the acceptance of this novel sources of information, more complex visualisations can be used in mass media.
• The use of both text and images is claimed to have a higher efficiency than single-channel encoding. One aspect that is crucial for this dual-coding and also for the concept of this thesis is the connection between these two levels. That supports the decision to directly integrate the visualisation inside the text to provide this kind of link.
• Obviously this link needs to be explicit in order to connect corresponding elements. (Ware, 2004) Experiments by Zellweger et al. (2000 in Goffin et al. 2014) evaluated the positioning of meta-information related to hyperlinks in margins, footnotes or inline. »They found a large variability in participants’ preferences but observed that, in general, it was desirable to keep meta-information close to the original link. Where information was placed also had a significant impact on reading speed with information close to the links being read significantly faster.« (Goffin et al., 2014) Even though dealing with different information these results are a confirmation for the usage of inline visualisations for better and quicker comprehension.
• Previous research in the field of visualisation has been mostly towards complex, data-rich graphics. I see a general need for smaller, simpler visualisations. As the media consumption tends towards mobile, content needs to be designed for the smaller display sizes. (Noirhomme-Fraiture et al., 2005) And with screens becoming cheaper and with OLED technology also more energy efficient so that they can be used for even more and also smaller devices like smartwatches, photography lenses or electronic toothbrushes, visualisation need to be specifically designed for this size. The »Internet of Things« is becoming inevitably a reality and all the devices hold opportunities for visualisations of the objects’ data. With their limited output capacities, the visualisation of their status is a crucial aspect. But not only for explicit small displays, but also for varying output devices small visualisation can be easier adopted to the available space.
• But also in time crucial environments like newspapers Micro Visualisations can be helpful. Smaller graphics are often easier to produce. Especially with the proposed typographic visualisations, this type of visualisation can also be created using amateur software like Microsoft Word. Instead of aiming for complex and costly graphics, the options possible with the available means are often not taken into account.
• As we can see from the three stages of perception: We see before we read. Using visual and typographic variables that convey the content visually allows the reader to comprehend the structure and possibly also parts of the content before reading. Especially colour that pops out from the rest can guide the reader to relevant parts.

Relevance

With the derivation of my approach in the last chapters, I will now further explain its relevance in a cultural and technological perspective and elaborate its relevance and implications for design. In the last part I will explain my own motivation for the approach.

Cultural and technological relevance

Technology and culture follow each other, facilitate and demand each other, depend on each other. The history of communication is a history of technology. Nowadays, as technology changes so rapidly, our behaviour highly depends on our technology usage. Our life is more than ever defined by devices and their inherent functioning. As these devices become more complicated to use we need to learn how to use them, learn to understand their output. The data generated by our computers and its amount is often beyond our understanding. But not only abstract data comes in an overwhelming amount. Also news, information of relevance or daily impressions inundate our life. The demand for sorting, contextualisation and filtering becomes stronger every day. In the following sections. I show how my thesis relates to these problems.

Rising quantities

McLuhan’s »implosion« (McLuhan, 2011) of the world brings people together in quantities that have never been possible before. Electronically processed information allows higher rates and greater amounts of data. People nowadays take a thousand pictures in their holidays, companies trade with goods in quantities of millions, countries have GDPs in billions of dollars. But besides higher dimensions we also measure smaller and smaller dimensions. Charles and Ray Eames’s short film »Powers of Ten« (Eames, 1977) illustrates this impressively: starting from above a couple having breakfast, the camera zooms out until it reaches the limit of our vision 100 million lightyears away and then zooms in to a single proton with a size of 0.0001 ångströms. And also Gavin Schmidt describes our capabilities to already measure 14 orders of magnitude of weather data: »from the small microscopic particles that seed clouds to the size of the planet itself«. (Schmidt, 2014) The recently populated term »big data« (which seems to have become a buzzword especially in business) is used for the rising quantities of data that are being captured through the electronical processing of scientific calculations and measurements, financial transactions, electronic communication and internet traffic in general. Companies like Google, Amazon, or Facebook, but also scientific institutions like the CERN or national agencies like the NSA use these data and the rising computing power to find previously undetectable patterns and insights. To emphasize this importance, the World Economic Forum rises the status of data to be the new oil. (World Economic Forum, 2011)

Computers allow us to process this amount of quantities, but it seems that our language lacks behind the technological progress: The difference between two pieces and a million pieces is tremendous and still the English and German languages for example have only two modes to describe quantities: singular and plural. Some other languages have more forms of expression quantities. Gravel G. Corbett summarises these in his book »Number« (2000): Some languages have dual, trial and (debatabally) quadrals. For example the Upper Sorbian language can differentiate between »you« (»ty«), »you two« (»wój«), and »you (all)« (»wy«). Accordingly, the trial and quadral expresses the amount of three respectively four. Other languages allow for a distinction of singular, paucal and plural. The paucal can be described as »small plural«, often used for quantities between two or to six. It can be found for example in Lihir, an Oceanic language spoken on a group of islands next to Papua New Guinea. They use five forms: Singular, dual, trial, paucal and plural. What can also be found is a split in the plural, to differentiate amounts of many. This serves for various functions: In the Arabic language for example it discerns between a general form (»debbānten« : flies) or the expression of abundance (»dababīn« : many flies). In Kaytetye (an Arandic language spoken in Central Australia) the two forms of plural discern between many and »all the x in the universe of discourse«. Two last examples come from the Celtic language Breton that describes »children« (»bugal-e«) and »groups of children« (»bugal-e-où«) and the Warekena language (an Arawakan language spoken in Brazil and Venezuela) that can express »pigs« (»abida-pe«), »very many pigs« (»abida-nawi«) and »very many pigs indeed, too many to count« (»abida-pe-nawi«) by composing different forms in one word. (Corbett, 2000)

What can be extracted from this excursus is that in fact very few languages have forms to differentiate between multiple quantities and it may be expected to cause translation and understanding problems in other languages that lack these forms. What visualisation can contribute here is the probably universally understandable form of expressing such quantities. Amounts of »two«, »tree«, »all«, »groups of multiple« and »too many to count« can be easily communicated in an image.

Images of visualised quantities gain importance for two reasons:

• First, many languages do not provide methods to express amounts accentuated. Obviously missing grammatical forms can be compensated by further textual expressions, but these supplements often fail in conciseness and universality.
• The second aspect is based on the rising amounts of data. Because nowadays the importance of an object relies so much on its underlaying quantities it needs to be directly integrated and connected.

In general, the data of an object becomes so important that this information should be placed together with the object.

Technological change

Processing in technology demands redesign. Every time a new medium is invented it is treated in the way the old technology was used. It takes time to adapt the usage to new input methods, faster transmission or greater range. I believe we live in a time where the application of technology is lacking behind the possibilities technology would allow.

Scott Klein showed in his talk »The forgotten History of News Graphics« (2014) that newspapers that published visualisations in the 1840s needed to build these graphics out of letters or other symbols. For rotated elements they needed to build special constructions in the composing galley. With the invention of offset print, this effort has been dramatically reduced. Elements can be freely composited digitally but still the design of the book has not changed much in the last 500 years. Now with the emergence of new digital technology the text layout is even less restricted to the classical design. But often the layout of books, posters or websites still keeps the form the letterpress required. Digital printing and displaying methods would allow absolute freedom over the book’s layout, but is »restrained by conventions of design and reading«. (Drucker, 2014) McLuhan spoke about this ever-repeating attitude in an interview in 1969:

»Theirs is the customary human reaction when confronted with innovation: to flounder about attempting to adapt old responses to new situations or to simply condemn or ignore the harbingers of change — a practice refined by the Chinese emperors, who used to execute messengers bringing bad news.
The new technological environments generate the most pain among those least prepared to alter their old value structures.The literati find the new electronic environment far more threatening than do those less committed to literacy as a way of life. When an individual or social group feels that its whole identity is jeopardized by social or psychic change, its natural reaction is to lash out in defensive fury. But for all their lamentations, the revolution has already taken place.«

— McLuhan, 1969

As he states, the »literati« are balking at the change, and I believe that non-specialists from other disciplines can help the future book to find its form But this needs to happen not only for the sake of change, but there is also a demand: Digitally published books cannot be designed in the same way analog letterpress books were designed, but need to consider varying reading environments, materials and different reading behaviours. Myers et al. state that we also need to »create user interfaces that work with vastly different sizes«. (Myers et al., 2000) Their article form March 2000 shows that difference of displays »are factors of over 1000 in number of pixels and 100 for size«, where it is noteworthy to mention that the highest resolution wall-sized display they describe was used by the Stanford University and had a similar pixel number to the laptop this thesis is written on 15 years later. In recent time, responsive web design evolved as an answer to this problem. Layouts from websites can now be specifically defined for every resolution. Since every display size leads to different reading distance as well, Oliver Reichenstein of iA discusses the concept of responsive typography. (Reichenstein, 2012) He distinguishes between the metric and the perceived size of the text. Since the metric size normally stays fixed on various devices the perceived size varies. He demands a fixed perceived size and hence an adaption of the metric size for each device’s reading distance.

But not only should the book’s design adapt to the user’s needs, but also explore newly formed options. Some avant-garde analog books already explore these possibilities. Jonathan Safran Foer’s book »Extremely loud and incredibly close« is an example of experimental typography. Converging and drifting apart text, marked and crossed out words, handwriting and photographs are used throughout the book. The book uses the possibilities of modern typography to support its content, the protagonist’s thoughts and activity and the overall mood of the book.

Drucker states that the current »new writing modes are shaped by social media, by email, blogs, Twitter, and wikis. In these changing conventions the surface of interface often conceals the back-end technical and conceptual processes by which they are produced.« (Drucker, 2014) As with all electronical devices, the appearance is separated from the function. Digitally published text for example is often designed for interaction. Facebook users can »like« everything published without needing to know how the servers store the »Like«. Only the metaphoric thumbs up icon brings the process back to the perceptible reality of the user. Other examples for these interactive texts are the platform Medium.com which allows for commenting single paragraphs, the browser plugin Dictionary of Numbers, which wants to »put numbers in human terms« by translating numbers like 86 million on websites into familiar terms, and Google Docs enables simultaneous work on a single document. Thus one might say that the display of the text becomes the interface of the text.

Andries van Dam demands interfaces to »enabl[e] and empower[…] people to be creative, productive, and less constrained by the limitations currently imposed by the technology.« (van Dam, 2001) My concern is that currently only few take advantage of what technology already enables us to do. The given examples are by leading companies or independent designers. The media consumed by the centre of the society is still dominated by classic text formats. We need more experiments and implementations of behaviour that would not be possible with letterpress technology and show how communication can be altered for good or also for bad. A change many industries who depended on the old technology seek for. Journalism for example is struggling with an identity crisis as its former distribution and speed is not compatible with people’s usage of the internet. A new role and a different form of communication could help overcome its problems. As companies search for information within their big data, newspapers as expounders of the world might seek for answers in the same data pools. As explained above, the use of visualisations helps to explore as well as to communicate these findings.

Summarised, digitally produced and published books or other essay forms allow for a form that combines multiple mediums within one. »Digital media have become the meta-medium par excellence, able to absorb and re-mediate all previous forms in a fluid environment in which remixing and culture jamming are the common currency.« (Burdick et al., 2012, p. 15)

Information overflow

We live in a time where »information gently but relentlessly drizzles down on us in an invisible, impalpable electric rain.« (von Baeyer, 2003) To find sense in these data is our daily challenge. Companies like Google specialise in this sense making by making available websites searchable. But even when we quickly find resources, the challenge to quickly comprehend and evaluate the given content remains.

One solution to this analysis of this content is the categorisation of objects. This has been a long practise for libraries or dictionaries in order to sort and later find information quicker. For this, objects are sorted into categories and subcategories, which can be easily navigated top-down. Cross-references also allow for links between objects. This way they stay in their place while still being connected. The whole internet is built upon the same structure: websites contain links that bring the users to the next website. When websites are organised in this way it can help users to quicker find relevant content. But not always is this structure visible. The UTS Library for example illustrates its catalogue with a stacked bar. Each colour represents one subject and its width represents the amount of books in this field. Each element also acts as navigation and splits out to more colours that represent subcategories. This Micro Visualisation illustrates the complete catalogue consisting of thousand of books in a memorisable, navigable, and compact way.

But also after the correct website is found on Google and on this website the relevant page is found, the presented document may be hard to comprehend. And again Micro Visualisation can help. Visualisations can for example act as an overview of the structure, they can help to find relevant points by »popping-out«, or they can show the information in its context.

When there is so much information from different sources the context becomes more important. To provide the user with this information but not overwhelm him is crucial. For this it is helpful to provide the reader with further information he can access if necessary. This concept matches Shneiderman’s »Overview first, […] then details-on-demand« mantra in its basic concept. While Shneiderman’s starting point is also a visualisation, my approach starts with the text in which the user can »zoom« to further information on demand through a visualisation.

To provide the user with orientation, navigation, and comprehension aids does not only help the reader, but also makes the content findable and accessible in the great amount of information given. Therefore the author should strongly support this process by not only presenting raw text or the »ten blue links«.