Web Documents Using Graphic Resources

Customizable Search Engine for Web Documents Using Graphic Resources.

Graphinder.

Illustration 1. Graphinder looks for Dalí. Manuel Viñas Limonchi

The proposed interface, called graphinder, acquires the characteristics of the traditional toponymic search, i.e. searching via text, through the well-known browsers; but, in this case, the element to be introduced – the original referent of the search – is a graphic fragment or a complete image that becomes the core of an exploration among an infinity of web images linked to the first one according to their distinctive graphic qualities. Thus, this is a new and accurate procedure for selecting information, which specifically manipulates the “graphic goods” – image formats included in the generic bit map – that ramble through the net, starting with the formal and – above all – color attributes of image as the vehicle and final purpose of the exploration. This way, according to the selective search philosophy favored by the Internet, once the image has been located our screen will automatically display the page/s that has/have any graphic nexus with the said image; after selecting one of these web addresses, the pre-established volume of information – text, hyperlinks, video, sound, etc. – is appropriately filtered.

Intro

If analyzed as a whole, the study proposed in this document employs methodological aspects of the image, understood as a visual referent which drives graphic recognition functions in a set of components belonging to the same family; furthermore, it deals with image as an element with the necessary physical attributes to regulate human-machine interaction processes. This practice will be based on the field of operation that uses image as one of its main bastions: the Internet.

The name of this engine for searching graphic signs in the web’s numerical space will make the reader think of stories already lived, completely comparable with the experimental archetype being determined. In 1996, the National Aeronautics and Space Administration sent the Mars Pathfinder probe to Mars – still a virtual world – with the purpose of using a small jeep called Sojourner to collect samples that could clarify certain dilemmas about the composition of the red planet. graphinder is not just a pseudonym out of an English word game; it is also a project committed to the same mission of informative inquiry as the device placed in the Martian orbit by the NASA. As in this superb aerospace project, in our digital iconographic essay image becomes an instrument for the transmission of information references; it is the origin, the means and the end of a binary operation that aims at new approaches to the methodology of communication.

The proposed interface, called graphinder, acquires the characteristics of the traditional toponymic search, i.e. searching via text, through the well-known browsers; but, in this case, the element to be introduced – the original referent of the search – is a graphic fragment or a complete image that becomes the core of an exploration among an infinity of web images linked to the first one according to their distinctive graphic qualities. Thus, this is a new and accurate procedure for selecting information, which specifically manipulates the “graphic goods” – image formats included in the generic bit map – that ramble through the net, starting with the formal and -above all- color attributes of image as the vehicle and final purpose of the exploration. This way, according to the selective search philosophy favored by the Internet, once the image has been located our screen will automatically display the page/s that has/have any graphic nexus with the said image; after selecting one of these web addresses, the pre-established volume of information – text, hyperlinks, video, sound, etc. – is appropriately filtered.

This kind of study requires a prior survey of the potential group of persons and, specially, of the professional body which will receive the results of the programmed experience. Thus, among the foreseeable users of this new system, we could distinguish two different groups. On the one hand, experts on the various professional areas: plastic artists, architects, engineers, 3D graphic designers,… They handle, on a daily basis, visual information related to the different creative manifestations – advertisement, computer assisted design, virtual reality, multimedia applications… – which require the participation of iconographic resources in the regular development of their internal processes. On the other hand, those web users in general who need that information typology and decide to choose this method as a natural complement to the one which now prevails. The latter, as we already know, is based on the recognition of chains made of typographical characters using Boolean formulae and other procedures, such as searching by the subject fields that could contain the term proposed.

Any reader with a certain knowledge of the matter being discussed will understand the high complexity of the central idea behind our search system. In order to process the data flow required for a graphic file, it is essential to initiate mechanisms of substantial experimental cost. In addition, we should bear in mind that this methodological exercise reunites different aspects that pertain to the distinctive features of the generic image – color, size and shape – with different concepts closely related to the computational side of the study – interface, devices, binary graphic typologies, etc. Nevertheless, the initial objective of writing this project is to discuss the new systems of communication, such as the Internet, in which the graphic category plays an active role in achieving the goals that make, as we said at the beginning, easier to further develop human-machine interaction. Leaving aside the meta-technological aspects responsible for constructing a new being called cyborg (or, in other words, “a being stuffed with digital devices”), we will establish a sequence that, above all, defends the participation of the design method – and all its train of elements and instruments – as the requirement or paradigm to be achieved.

It should be noted beforehand that, even though the proposal for the graphic search engine could be undertaken from different practical approaches, in this case we will focus on the technical formulations that are helpful for the constructive part of the graphic work, whatever its genre. The following items explain actual elements that stem from the project.

Description of the method

The project’s framework that includes graphinder’s operation determines a fluent bilateral circulation of graphic data. This circulation compares two image models which, in order to achieve the goals proposed for this research, will ultimately maintain certain similarity in most of their component physical attributes. Or, in other words, the said development shows a similarity of iconographic patterns obtained as a result of comparing the graphic information, and this confirms that the data exchange found a series of values related to both images.

Therefore, we will start by briefly defining these two iconographic typologies, in order to get used to their personal mission in the devised interface.

1. INPUT Image (or Pattern)

A complete image or part of it that, in its digital state, will be employed as an initial search standard in the extensive catalogue of graphic data supplied by the Internet. Explicit preliminary reference that articulates the exploration.

2. STABLE Image (or Exposed Image)

This name refers to images with any graphical format – included in the bitmap catalogue: GIF, JPG, BMP, TGA, TIFF, etc. – and from many different genders (painting, photography, illustration, etc.) previously inserted in the web pages that are part of the web’s volume of information.

…

When we carry out a search using text, we resort to values determined by the chains of characters that make up the linguistic term or terms; in the iconographic search, we will set the attributes for each of the pixels concentrated in a graphic area of the image as the cataloging clause or archetype. Thus, the qualities attributed to color, which can be quantified using numbers, will be the main initial parameter needed to specify accurately the numerical code that equals or differentiates one dot matrix from another. This deduction will be analyzed in the following chapter.

Physical characteristics of the pixel matrix

Size of the matrix

Albrecht Durer, in his famous treatise on measurement Underweysung der Messung, divided the space where the drawing was to be made using a proportional grid (illustration 2). The result of the experiment was a gridded surface in the shape of a window that contained imaginary coordinate axes, which would make easier to locate the most interesting points projected on the model’s features while he/she posed behind the “window”. The pixel matrix extension that we will have in our graphic search interface is very similar to this Renaissance tool; the definitive practice stipulated is based on the particular distribution and attributes of each one of these tiny pixels taken as a whole.

Illustration 2. Artist painting a lying model. Albrecht Durer

Therefore, one of the first tasks to be performed is defining the size (height and width) of the matrix that will receive the input images. Considering that this standard size will be equally appropriate for any iconography used, and that it will be sufficient to solve the calculations generated by the operation of searching its homonymous image or images, we will agree on a number of pixels specifying a minimum and a maximum. Below the minimum, it is understood that the image would be very difficult to read by the search systems established, or it could even be mistaken with other stable images with a similar chromatic range.

Another possibility is using a TESSELATION method, i.e. segmenting the real graphic area of the image in a number of subdivisions. This way, we will cover and analyze an important part of the image, obtaining a greater reliability of results when comparing the data. Each one of these image fractions will be first independently investigated, and then globally analyzed with the rest of the fragments provided.

As shown in illustration 3, if we insert -as input- a fragment of the image to be located, this fragment should be sufficiently significant when compared with the image as a whole to permit the best search possible through the web’s conduits without suffering the aforementioned color mismatching. For example, if we have an image with large surfaces in different shades of plain colors, it is convenient to discard these areas as a search pattern, given the deficient quantity and quality of graphic information they supply because of their chromatic homogeneity. In the same illustration by Raphael, the matrix that contains the shape of the image isolated as input has an area of 55 x 63 pixels or dots (height/width). This would be and adequate graphic fragment to start an efficient web search, because it has the appropriate size and because it has been extracted from an area with suitable descriptive characteristics. It could be part of a greater variety of tiles adjacent to the first one or scattered around the orography of the image.

Illustration 3. Pixel matrix (Virgin at the Rose, by Raphael). Manuel Viñas Limonchi

Chromatic composition of the pixels

As we already know, in the digital domain the treatment of color follows a series of codes perfectly organized into ranges or models, which create groups with some of these colors. Regulated by the CIE (Commission Internationale de l’Eclairage), there are models – sometimes organized as color palettes – such as RGB (which will be explained later on), CMYK (especially used for color printing, and composed of cyan, magenta, yellow and black) or HSB (hue –H, saturation –S, and brightness –B, which determine, in this order and using numerical characters, the color, its depth and intensity). One more could be added; the PANTONE (PANTONE© Formula Guide, a product of Pantone, Inc.) is used -as the CMYK- in printing applications for the graphic arts industry. Its colors are described in a schematic guide or table by means of alphanumerical codes, in order to help choosing the right ink. For example, PANTONE© 7487 C would have a chromatic correspondence with light green.

Out of the models mentioned above – and some more -, we prefer the RGB, as it is the one employed by digitalization devices such as the scanner, by display devices such as the screen and, beyond technological considerations, we could consider this model as the one that regulates our binocular vision. For this reason, given the importance of the digitalization peripherals as mediating instruments between the original image – possibly with an analogue or traditional manufacture – and the image that we will use as input, the said RGB model becomes the most suitable language for interpreting or translating the graphic data, thanks to its color qualities.

Deciphered into numbers, each one of the three channels that make up this model of additive colors (red, green and blue) has been assigned a value from 0 to 255. For example, the numeric expression “0,0,255” would correspond to a deep blue with no mixture of red or green. The numbers “0,0,0” would give black, and “255,255,255”, would give white. Following this schematic procedure, we obtain a matrix in which each one of its cells provides three values that refer to that particular pixel.

Illustration 4. Chromatic values of the pixel according to HSB and RGB models. Manuel Viñas Limonchi

Project Verification

To backup the development and utility of the project, it will be essential to create a graphic user interface that supports the management of the data to be introduced, and an intranet that simulates Internet’s properties, as the Internet is the final and actual defender of the chosen method. The insides of this intranet would be equipped with the catalogue of stable images that cover the search for homonymous images from different iconographic stages.

The registration process of stable images will be very similar to the one used by the traditional search engines, i.e. hierarchizing the information with criteria that examine thoroughly its natural content. Thus, we obtain a base of graphic data with an important number of classified representations incorporated in definite fields such as image-photography, image-drawing, image-painting… Additionally, each field will have subdivisions containing the set of works identified by an author within a certain category.

In short, the sequence of the process that generates the search would be as follows (see illustration 7 for a more descriptive diagram):

1. Introduction of the input image through the interface (illustration 5). The best method would be importing (“pasting”) the image -transformed in a bitmap- in the maximum area of the matrix, adapting -never deforming- its dimensions to the said gridded extension. Previously, we must copied this image in the clipboard using a paint program.

Illustration 5. Graphic interface prototype. Manuel Viñas Limonchi

2. Once the appropriate mechanisms have been activated, the search begins. The collection or catalogue of images previously created is explored, and the color qualities of the input image -transcribed to numbers- are mechanically compared with the parameters that belong to each one of the stable images.

3. Once all the stable images with physical attributes within the graphic synthesis requested have been verified, we decide to receive directly from them the exact routes needed to locate the file with the requested information, i.e. the URLs (Uniform Resource Locators) indicated beforehand, when we created the catalogue. Upon finishing this operation, a list of web pages

Illustration 6. Traditional search method. Manuel Viñas Limonchi

linked to each graphic element will be displayed on our graphinder search engine. Then, we will only have to click on one of them to access the virtual space with the information.

Illustration 7 graphinder. Manuel Viñas Limonchi

Illustration 8. Search hypothesis (Bash, by E. Paolozzi). Manuel Viñas Limonchi

Advantages and Disadvantages of Graphinder

Having analyzed the different internal aspects of the project, we will comment on the capacities and limitations of its operation. These are the advantages and disadvantages of a pilot experience in the use and systematics of the Internet, a field yet to be fully discovered. The initial purpose is creating a beta version that, in the neat future, could become a tool for a wide group of people, to be used at work or during their spare time.

Firstly, we will mention the EFFICIENT QUALITIES that we – hypothetically – find in graphinder. The different aspects related to the methodology of the image, to its construction, will be considered at all times. Thus, this methodology becomes the mediator of exclusive procedures based on a wide set of areas.

ADVANTAGES

General Level:
Selective search of information based on graphic data

On several occasions, when working with iconographic documents, we do not have data on the manipulated images. Therefore, we need one or more reliable references to provide us of a starting point from which to access all the data available in the Internet on the field of the illustration, drawing or photograph. This is the original, main and virtually sole purpose of inventing graphinder. To this essential and strictly informative function created to aid the cyberspace community involved in certain professional activities, we add a second one. This second function could be described from a more technical point of view, as the necessity of including new archetypes of the visual alphabet in the data processing that occurs inside the channels in charge of regulating the information that gets to our terminals.

graphinder completes and refines the systematics of the search

graphinder would provide a tool that covers most of the data searching spectrum of the world wide web, at least until the invention of a search engine that recognizes sound entries following the philosophy described in this paper. Chains of characters and images or, in other words, typographical expressions and iconographic representations, will fulfill all their needs of information trasmission from the internal channels of the Internet, where they live.

Technical Level:
Verification of images already included in the net (STABLE images)
for their comparison with the new ones (INPUT images)

It is not necessary to verify each one of the images in the Internet -which would be nearly impossible- because their original or new digital structure bears their genetic ID in numeric values. These values refer to the color attributes that characterize each one of the pixels that make up its appearance.

… and, in ascending order according to the ranking that classifies the obstacles encountered, these are the DISADVANTAGES.

DISADVANTAGES

General / Technical Level:
Less spontaneous introduction of data in the initial phase.
Availability of essential devices

The idea of substituting the elemental – or spontaneous – action of introducing a certain quantity of text related to an inquiry in the search engine or browser, with the action suggested for this new system, in which the data come from graphic formats (bitmaps), reveals a first significant load or hindrance that we must overcome to achieve our goals.

On the other hand, acquiring the said graphinder interface is not enough. We must also have access to a series of hardware devices (digital camera or scanner) and image processing programs, which will permit the manipulation of the document to best adapt it to the search method. Nevertheless, we assume that the graphinder user has an unquestionable practice in operating with this type of visual entities, and owns the abovementioned physical tools as part of his/her work equipment. In any case, it should be mentioned that the initial course of this search engine follows the same steps as other experimentations, such as Internet 2, created in and for the academic research bodies. This circumstance reduces the original number of people interested in the engine and increases its capacity for this kind of applications.

Technical Level:
Physical difficulties in the recognition of the data to be verified

The main obstacle for matching the models is that, in 99% of the cases, the graphic information we have will be physically distorted or imbalanced when compared with the image that is uploaded in the Internet. An example that corroborates this fact is explained below (see illustration 8 for more information):

We receive a printed photograph with a fragment of a painting by Eduardo Paolozzi called Bash. Although I am giving the name of the author and the work, we do not know these data upon receiving the photograph and, eager for obtaining information about this image we find so attractive, we begin our research based on what we have, a small color piece probably torn from an art handbook. Following the steps described above, we scan the image using a 1:1 scale to transform it into a digital document. After repeating several digitalization operations on the same piece, we obtain the same number of graphic files. These, when compared among them, show slight alterations of color (see the numeric equivalences in illustration 8, detail “A” and “B”).

We could perform an infinite number of verifications on this argument, and the above fact would still be the same: two digital documents that refer to the same iconographic representation do not have to show the same chromatic values, as the level of manipulation they suffered when converted to the binary format could be – and most probably was – different in each case. Therefore, it is necessary to solve the problem of identifying both images as the same document (input image = stable image), overcoming the numeric eventualities or mismatches that affect each image.

The solution is creating a method of graphic recognition based on the numerical comparison of the pixel values. This application must establish a general average of the values for all the pixels that correspond to both images (input and stable), and then generate a table of figures which will be proportional to the set of points that shape both matrixes (algorithm for the interpolation of pixels).

After our investigation, whose introduction was the appearance of that small piece of paper, we obtain a valuable list of references about the author of the previously unknown work. The artist has Italian name and ancestors, but is of Scottish nationality; he was one of the first Pop Art painters in the 50s. This information has been extracted from the web after processing a tiny graphic excerpt.

Illustration 9. Comparison of pixeled areas from two reproductions of the same image. Manuel Viñas Limonchi

Up to this point, we have analyzed the difficulties that could arise due to a possible difference in the color attributes of two equal graphic documents. However, as a rule, the various copies of the same image that we can find in books, newspapers, CD-ROMs, … appear in different levels of graphic resolution or size. This indicates that the resolution of the fragment or image introduced will probably be different from the resolution of the web images. In other words,

INPUT IMAGE Size STABLE IMAGE Size

This is the case with images such as drawings or pictorial representations made using traditional instruments and then digitalized by means of the known devices. These are not usually “uploaded” to the Internet servers with their actual size, as they would take too much space and therefore give problems for storage and user’s manipulation (slow screen display and downloading). These inconveniences are magnified in countries such as Spain where, due to technical problems, the transmission of data through the network protocols does not reach the speed we would all like.

On the other hand, it is more usual – although we cannot generalize – to find originally digital works (synthetic images) in the web with their actual sizes. This circumstance would make our search process much easier.

After all these incidents, the same image, processed and then examined using different media, acquires different graphic imprints from the personal marks added by each digital mechanism. Thus, the quantity of dots per inch (dpi) of its total area and its chromatic values – as mentioned above – vary. This is the main dilemma that will hinder the ideal projection of graphinder. A possible solution could be developing a relational algorithm that compares -interpolating color pixels- this discrepancy on the quantity and quality (attributes) of the pixels that shape both copies (illustration 9).

… in short

We are talking about a new intelligent agent, graphinder, inserted in the methodological stage that regulates the information search systematic in the vast cosmos of the Internet. An innovative network knowledge management system, whose last purpose is offering services for transmitting communication habits to any “cyber-consumer”, using the semantic faculties of the graphic element.

Fortunately, the digital technology allows us to appear in the Internet with first and last names, photos, videos, electronic identities before and after the @, etc. But, above all, we are just numbers. This is the most outstanding characteristic of an electronic environment as peculiar as the Internet and, oddly enough, one of the less considered, even though it has a devastating impact on all its users.
(Luis A. Fernández Hermana)