README

XOgastan 1.0
Programmed by the student Gianluca Masone
At University Of Sannio in Benevento
October 2001

 

XOgastan a software for C developers.

XOgastan is an application that  includes a set of program's tools for translation and analysis of the AST (abstract syntax tree) dumped by gcc compiler (gnu compiler collection). The programs are written in PERL and C++ language, and they can run under a LINUX operating system on a iX86 machine.

 
----------------------------------------------------------------------------
Introduction:

In this document we describe the software XOgastan. The objectives we want to reach are three: why we have written XOgastan, how XOgastan works, and who would use XOgastan.

To reach these three objectives we answer the followings nine questions:

 

What does XOgastan mean?

Why is XOgastan XML oriented?

What?s the requirements analysis of XOgastan?

What is the habitat of XOgastan?

How does XOgastan read the AST dumped by gcc?

Does XOgastan use an internal representation of the AST?

What data can XOgastan get?

What is the output of XOgastan?

When XOgastan be useful?

 

Note, this document is not a FAQ (frequently asked questions). It is a semi-formal document about XOgastan. For a full documentation about how XOgastan works you can see the on- line documentation included with the software package.

Now, we begin with the questions !


----------------------------------------------------------------------------
What does XOgastan mean?

 

XOgastan means: XML-Oriented Gcc Abstract Syntax Tree ANalyzer. Wahoo, it's a very long name ! Yes, this is  a very long name but the interpretation is very simple. 

The name is composed of three parts:

 

ü       XML Oriented

#-       Gcc Abstract Syntax Tree

#-       ANalyzer 

 

The first part, XML Oriented, gives us information about the technology XOgastan uses: the "new" XML technology. 

The second part, Gcc Abstract Syntax Tree, gives us information about the fact that XOgastan interacts with the output file of gcc. More precisely, it uses the file produced by gcc that contains the ast of a C program. This dumped file, before the use by XOgastan, is translated into an intermediate XML representation. 

The third part, ANalyzer, gives us information about the purpose of XOgastan: it analyzes its input file and gets some data. 

We can tell:

 

?XOgastan is a program that takes in input the file containing the ast generated by gcc, and analyzes it. The results are a set of information about the C program represented by the AST. The analysis and the results are format and presented by using the XML technology.?

----------------------------------------------------------------------------
Why XOgastan is XML oriented?

 

XOgastan is our latest program so we decided to use the nearest technology: c++ stl, XML, linux? This is just our observation!

In fact, we decided to use the XML technology for several serious reasons:

 

1.    XML can be used to represent simple or complex data structures.

2.    XML can be used to represent data that can be shared among applications.

3.    XML will be one of the key technologies of future Web applications.

4.    XML is very flexible.

5.    XML has reached a good level of maturity.

6.    Currently there are a lot of api, produced by different vendors or groups, that implement the DOM. We choose the exerces-c api of the apache project.

 

Now, we list the purposes of XOgastan:

 

a.    XOgastan must take an input file with the ast.

b.    XOgastan could produce data that can be shared with other applications.

c.    XOgastan should use standard technologies!

 

So, if we analyze the previous three points we can deduce:

 

#-       The ast dumped by gcc doesn?t use a standard format (it is a format designed  by gcc designers) ? We translate this format in a more standardized one: gxl, which is in fact an XML dtd (document type definition).

#-       If XOgastan shares data with other applications, then it will be a good idea to use a standard language. We define some dtd to represent the structure of the output of XOgastan, and we produce output in XML oriented way. 

----------------------------------------------------------------------------
What is the requirements analysis of XOgastan?

 

When application is designed, the first thing to do is requirements analysis. 

We analyze the requirements of XOgastan by using two different points of views.  Each point of view is an actor of the system where XOgastan will work. We have:

 

#-       The user?s view point: what the human user will do with XOgastan 

#-       The external agent?s view point: what an XML oriented application will do with XOgastan.

 


Good ! XOgastan must satisfy the users and the external agents? requirements ! In other words:

 

1.    The application must translate a foo.c.tu file into a foo.c.tu.gxl file. 

2.    The application must build a new AST into the main memory. 

3.    The application must analyze the new AST and must pick up some data about the C program and the new AST itself. 

4.    The application must dump the data using a human friendly language. 

5.    The application must build some DOM documents associate to the data picked up. 

6.    The application must produce, using the DOM, some XML files.

 

 

The first point is not achieved by XOgastan itself, but the translation is performed by a perl tool, called gcc2gxl, supplied with the XOgastan package. In this way, XOgastan performs only the loading of the ast, its analysis and the output generation. 

 

 
----------------------------------------------------------------------------
What is the habitat of XOgastan?

 

By habitat of a program P we mean the set of programs that are ancestry of P or has the same purpose (functionality) of P.

This is the first version of XOgastan and then we can tell that XOgastan is the base program of its genealogy. However, there are many projects that work as XOgastan, including following: gasta, cppx, DATRIX.

 

There are a lot of differences between XOgastan and the previous projects:

 

#-       XOgastan versus DATRIX. Te input file, the representation used, the resulting data are different. Datrix is designed to analyze the C/C++ source code and not the intermediate representation of an ast generated by a compiler. Datrix does not analyze an ast, but an asg that is an evolution of the ast. Moreover, Datrix is also oriented to the analysis of C++ programs.

#-       XOgastan versus cppx. Both programs analyze the ast produced by gcc, but there are differences: the languages used and the purpose. Cppx is written in C. The purpose of cppx is transforming the ast of gcc into a Datrix asg, and the output is a graph represented by using the gxl language. The only point in common between XOgastan and cppx is the XML representation of the graph.

#-       XOgastan versus gasta. Gasta is software for the analysis of gcc dumped by gcc. The differences between XOgastan and gasta are: the purpose is the same, but the techniques and the technologies used to achieve the purpose are different. Gasta is written in C, XOgastan is written in C++. Both software are intended to be used by developers. XOgastan is XML oriented and gasta is not.

 

At the end we can tell:

 

?XOgastan uses the same technologies used by cppx and its purpose is the same of gasta: obtaining information about a C program !?

 
----------------------------------------------------------------------------
How does XOgastan read the AST dumped by gcc?

 

XOgastan is an XML oriented application because its input and its output are xml files. So, XOgastan take in input an XML representation of the ast dumped by gcc. The point is: the ast dumped by gcc is not represented using an XML format! How do we translate the format dumped by gcc into a XML format? Which XML format do we choose? 

After a search on the web, and some advices of an expert in this field we choose the gxl dtd. Now, when we have the initial format, the one produced by gcc, and the final format, the gxl dtd, and we can describe how the translation is made. We wrote a perl tool called gcc2gxl. It includes some perl scripts and a translation table: oneFORline.pl, gcc2gxl.pl and g2x.map. 

The script oneFORline.pl takes in input the file foo.c.tu dumped by gcc and produces an output file foo.c.tu.ONE with one token of foo.c.tu for line. In this context a token is a set of characters without space, tab ? 

The script gcc2gxl.pl takes two inputs file: foo.c.tu.ONE (the output of oneFORline.pl) and the translation map g2x.map. The output file is a file with the ast represented in gxl format.

We can think that oneFORline.pl is the left hand, gcc2gxl.pl is the right hand and g2x.map is the mind! Yes, the most important file is g2x.map. The ast of gcc is made up of many nodes, and each node has: its code (or in other words its purpose), its list of attributes, and its list of possible linked nodes. The file g2x.map lists all the nodes of the ast (also the C++, pascal ? nodes) and for each node specifies a set of translation rules. A translation rule transforms information of foo.c.tu into a gxl element. This is an example:

 

1.    Gcc has a node to represent the function declaration, its code is function_decl. This node contains information about the status of the function declaration: static or extern memory class, the name of the source file where is declared, the number of line in the source file where the declaration is ? Moreover, this node has different links: link to the node with the function name (an identifier_node), link to the first node of the body (compound_stmt), link to the next declaration in the same scope (can be any declaration node), ?

 

2.    The following lines  are an example of the information dumped gcc about the function_decl:

 

@15     function_decl    

name: @29     mngl: @30     

type: @31     srcp: div.c:101

chan: @32     args: @33     

static        body: @34  

 

where @15 is the index (unique in the dumped unit file) of the function_decl, @29 is the index of the node with the name, and so on ?

 

3.    Some of translation rules for the function_decl are:

 
case FUNCTION_DECL:
name:*%<edge from="index" to="*"><type xlink:href="gccast.xml#name"/></edge>
type:*%<edge from="index" to="*"><type xlink:href="gccast.xml#type"/></edge>     
scpe:*%<edge from="index" to="*"><type xlink:href="gccast.xml#scope"/></edge>
srcf:*%<attr name="source_file"><string>*</string></attr>
srcl:*%<attr name="source_line"><int>*</int></attr>
artificial %<attr name="flag"><string>artificial</string></attr>  
chan:*%<edge from="index" to="*"><type xlink:href="gccast.xml#next_decl"/></edge>
 
args:*%<edge from="index" to="*"><type xlink:href="gccast.xml#arguments"/></edge>
undefined  %<attr name="flag"><string>undefined</string></attr>
extern %<attr name="flag"><string>extern</string></attr>
static %<attr name="flag"><string>static</string></attr>
body:* %<edge from="index" to="*"><type xlink:href="gccast.xml#body"/></edge>    
fn:*%<edge from="index" to="*"><type xlink:href="gccast.xml#body"/></edge>
 
Each rule has its format and we have two different formats. In this document we do not talk about these formats. For more info see the on-line doc of XOgastan.
 

4.    At the end, the final result of the translation will be:

 

<node id="15">

   <type xlink:href="gccast.xml#function_decl"/>

   <attr name="source_file"><string>div.c</string></attr>

   <attr name="source_line"><int>101</int></attr>

   <attr name="flag"><string>static</string></attr>

</node>

 

<edge from="15" to="29"><type xlink:href="gccast.xml#name"/></edge>

<edge from="15" to="31"><type xlink:href="gccast.xml#type"/></edge>

<edge from="15" to="32"><type xlink:href="gccast.xml#next_decl"/></edge>

<edge from="15" to="33"><type xlink:href="gccast.xml#arguments"/></edge>

<edge from="15" to="34"><type xlink:href="gccast.xml#body"/></edge>

 

The file g2x.map is the result of a long analysis of the ast gcc. We studied the source code, the documentation file and at the end we got a comprehensive  set of translation rules. The most important files of gcc project we studied are: tree.def, tree.h, c-common.def, cptree.def, cptree.h, dump.h, dump.c, cpdump.c, ?

It has been a very hard work !

----------------------------------------------------------------------------
Does XOgastan use an internal representation of the AST?

 

The answer is simple: yes, XOgastan uses an internal representation of the AST !

XOgastan is written in C++ and, if you do not know, this is an object-oriented language. So, XOgastan exploits all the power of the object-oriented paradigm.

We define a hierarchy of classes to represent an ast and we call, this structure, NAST (New Abstract Syntax Tree). In other words we create an object oriented AST. This ast is not equal to the ast of gcc: some parts of ast are similar to the nast (in the following we use ast to refer to gcc ast, and nast to refer to XOgastan ast), but some other parts of the ast are not present into nast. 

Let us have a short look ! Nast is a three level hierarchy of nodes:

 

#-       first level - the father.

#-       second level - the different set of concepts: declarations, types, constants, expressions, statements, other different concepts.

#-       third level - the elements of a C program: variable declaration, integer type, real constant, call expression, for statement, identifier, ? 

 

 



Figura 2 first/second level

 

 

When XOgastan reads a node from the gxl file then it parses the code of the node and it creates an object of a leaf class. For example:

 

#-       if XOgastan reads a var_decl node then it creates an object var_decl_n

#-       if XOgastan reads a integer_type node then it creates an object integer_type_n

#-       if XOgastan reads a real_cst node then it creates an object real_cst_n

#-       if XOgastan reads a call_expr node then it creates an object othr_expr_n

#-       if XOgastan reads a for_stmt node then it creates an object for_stmt_n

#-       if XOgastan reads a identifier_node node then it creates an object identifier_n

#-       if XOgastan reads a start_catch_stmt node then it creates an object othr_stmt_n

 

You should note three important points:

 

#-       The code of gcc node differs from the name of the nast classes only for the chars _n at the end of the word (see var_decl and var_decl_n).

#-       The correspondence between the ast nodes and the nast classes is not always one-to-one. A var_decl ast-node has only the correspondent var_decl_n nast-class, but the ast-nodes plus_expr, mult_expr, le_expr correspond to the class unary_expr_n.

#-       XOgastan can parse also the C++ ast-nodes like start_catch_stmt. When it reads an ast-nodes that do not belong to the C grammar then it creates an object of one of the special classes: othr_decl_n, othr_type_n, othr_cst_n, othr_stmt_n, othr_expr_n, special_n ! Is this mechanism very powerful and extendable?


----------------------------------------------------------------------------
What data can XOgastan get?

 

We can tell that the analysis of XOgastan is function-oriented ! In other  words, XOgastan can get a lot of information about the functions present into the nast (or the ast, that is the same).

This version of XOgastan analyzes the nast by searching the function declarations and for each one of these it performs same further analysis: 

 

#-       Gets the name, the type returned, the parameter list ... of the function.

#-       Gets some information about the statements into the function's body: pick up statistics of the statements used, build graph of the body tree produced by gcc, build a control flow graph, ... 

#-       Gets the list of declarations into the body of function: variable declarations, typedef declarations, ... 

#-       Gets statistic about the number of expressions and operators ... 

#-       Gets a list of the variables used into the expressions: variables with local scope, variables with global scope, parameters.

#-       Builds a call-graph of the function called. 

 

----------------------------------------------------------------------------
XOgastan picks up also statistical information about nast: total number of nodes, frequency of a node, ?

Now, you could ask us how XOgastan collects these data ! Simple, it uses an analysis-pattern called visitor-pattern (remember that XOgastan uses an  object-oriented abstract syntax tree, and remember that C++ is a very powerful language). You can understand by reading the following situation:

 

?A man, with a big drawer, walks inside a big palace which consist of a lots of coloured rooms (at least 2000 rooms). Each room has its colour, not all  adjacent rooms have the same colour, and we have only one hundred  colours ! The man searches some objects, ... and each typology of object is always present into the rooms with the same colours. During the visit the objects are taken by the man and put into the drawer ! To visit the palace the man reads some instructions written by a friend !?

 

 



Figura 3 visitor and drawer

 

 

Maybe it is a very stupid example but, if you think ...

 

#-       The NAST is like a big palace.

#-       A node is like a room.

#-       The code of the node is like the colour of a room.

#-       The visitor is like a man.

#-       The single data are the objects collected by the man.

#-       The structure used to hold the collected data is like the drawer. 

#-       The strategy of the visit is the instruction's list used by the man. 

#-       The programmer is the friend !! 

 

Ok, this is the visitor-pattern used by XOgastan !


----------------------------------------------------------------------------
What is the output of XOgastan?

 

XOgastan has two main outputs: the html pages and the xml files. The html pages can be read by an human user, and the xml files can be read by an other application XML oriented or can be browsed by using an XML browser. At the moment, the xml dtd designed are thought also for using the data into a relational data base. 

XOgastan produces also other output! A classification of the output produced is:

 

#-       File foo.dot, in the directory dot/, for a graph: body graph, cfg or call graph.

#-       File foo.gif, in the directory img/. Each gif file has a correspondent foo.dot file.

#-       File foo.txt, in the directory txt/, describing a cfg in a text mode.

#-       File foo.htm, in the directory html/. There are html pages for the functions, html pages for the different kind of index, and the hmtl with statistics about the nast.

#-       File foo.xml, in the directory xml/. Each xml file reflects a well defined xml dtd.

 

----------------------------------------------------------------------------
When XOgastan be useful?

 

People who can use XOgastan are the developers and not generic computer users. More precisely, XOgastan can be useful to the C programmers. 

Let us suppose, for example, that Donald Duck, a friend of us, is a C programmer. He writes a big application like hello.c and he compiles it. The application runs, but it s very complex; there are: a lot of functions, a lot of typedef, struct, and so on ? Donald Duck can?t remember all by heart, and he will have problems when, after some time, he wants to make some changes in its code. Now, Donald Duck has not written down any documentation about hello.c. What can he do? He can use XOgastan ! He must follow these steps:

 

1.    compile hello.c by using gcc options -fdump-translation-unit:

 

[lucas75it@home]# gcc -fdump-translation-unit hello.c

 

 

2.    translate the output file using the perl tool gcc2gxl:

 

 

[lucas75it@home]# oneFORline --ifile hello.c.tu  --ofile hello.c.tu.ONE

 

[lucas75it@home]# gcc2gxl --ifile hello.c.tu.ONE --ofile hello.c.tu.gxl --table g2x.map

 

 

Where g2x.map is the translation map presented in the perl tool package.

 

3.    analyze the ast, and in this way the C source code, using XOgastan:

 

[lucas75it@home]#  XOgastan hello.c.tu.gxl

 

4.    open a beer and read the html output of XOgastan !

 

Therefore, XOgastan can help all C programmers who want to get well organized information about their C programs. Also the programmers that write document for their projects can use XOgastan.

Good work with XOgastan ! (For the Donald Duck fans: I love it !) 

 



Figura 4 basic steps

 

Miscellaneous

 

Xogastan installation cook book: now, we suppose that you use the bash shell (for the other shells you can think ...). The basic steps are:

 

1.    Get one of the package:

XOgastanFull.tar.gz - full version with all the documentation 
XOgastanHack.tar.gz - full version with the documentation, but without the full testing documentation 
XOgastanUser.tar.gz - only the sources

2.    Extract the files by typing:

 

[lucas75it@home]# tar xvzf XOgastanHack.tar.gz

 

3.    In your configure file .bashrc set environment variable XOgastanROOT:

 

export XOgastanROOT=/home/lucas75it/XOgastan

 

Where in my home computer /home/lucas75it/XOgastan is the base path of XOgastan. 

 

4.    Compile the sources:

 

[lucas75it@home]# cd $XOgastanROOT/src

 

[lucas75it@home]# gmake

 

5.    If you have installed the Doxygen tool on your computer, then you can produce the online documentation of the sources by typing:

 

[lucas75it@home]# gmake doc

 

XOgastan dependencies: XOgastan has two dependencies:

 

#-       Compile Dependency: to compile XOgastan you require the xerces-c library (part of linux-apache project). You must get this library and install it. 

#-       Run time Dependency: to get the graphs you require the tool graphviz (developed by the AT&T co.). The package must be installed, and the path of binary file dot must be inserted into the PATH environment variable. 

 

Also, you require that your system is POSIX compliant.

 

XOgastan programmer: The programmer of XOgastan is Gianluca Masone. He is student at the University of Sannio in Benevento (Italy) at the Faculty of Engineering (computer science). 

This software has been developed under the supervision of the professors G. Antoniol, U. Villano and G. Canfora.

 

Bibliography:

 




Appel


Modern Compiler Implementation in C


 




 

H.Maruyama, k.Tamura, N.Uramoto


 

XML and Java, developing web applications


 

Addison Wesley




 

E.Gamma


 

Design patterns: element of reusable Object Oriented Software


 

Addison Wesley




 

Bjarne Stroustrup


 

The C++ language


 

Addison Wesley




 

S.St.Laurent


 

XML, le basi


 

Tecniche Nuove




 

Document Object Model (DOM) Level 1 Specification (Second Edition)


 

Version 1.0


 

W3C Working Draft 29 September, 2000

 




 

Xerces-C Documentation v1.5.1


 

API Docs for SAX and DOM


 

online documentation




 

Gnu.gcc.org


 

Using and Porting the GNU Compiler Collection (GCC)


 

online documentation




 

Looking for a Graph eXchange Language


 

Presentation at the APPLIGRAPH-Subgroup Meeting on Exchange Formats for Graph Transformation September 5-6, 2000, Paderborn University, Germany


 

online documentation

 

For contact, send mail to lucas75it@yahoo.it.