Open Access. © 2019 Azanzi Jiomekong et al., published by De Gruyter. This work is licensed under the Creative Commons

Attribution alone 4.0 License.

Open Computer Science 2019; 9:181–199

Research Article Open Access

Azanzi Jiomekong*, Gaoussou Camara, and Maurice Tchuente

Extracting ontological knowledge from Java

source code using Hidden Markov Models

https://doi.org/10.1515/comp-2019-0013

Received April 29, 2019; accepted July 25, 2019

Therefore, there is a lot of value in using ontology learn-

ing which consists in automatic or semi-automatic extrac-

tion of ontological knowledge from structured or unstruc-

tured knowledge sources such as texts, databases, etc.

Many techniques have been used but they all are limited

in concepts, properties and terminology extraction leav-

ing behind axioms and rules. Source code which naturally

embed domain knowledge is rarely used. In this paper,

we propose an approach based on Hidden Markov Models

(HMMs) for concepts, properties, axioms and rules learn-

evaluation of the relevance of the terms extracted to the

medical domain by aligning them with ontologies hosted

on Bioportal platform through the Ontology Recommender

tool. The results were interesting since the terms extracted

were covered at 82.9% by many biomedical ontologies

such as NCIT, SNOWMEDCT and ONTOPARON.

Keywords: Knowledge Extraction, Ontology Learning,

Hidden Markov Models, Java Source Code, Viterbi

*Corresponding Author: Azanzi Jiomekong: University of

Yaounde I, Faculty of Science, Yaounde, Cameroon; IRD, Sorbonne

Studer et al. [1] dened an ontology as "A formal, explicit

specication of a shared conceptualization". In the con-

text of domain ontologies, conceptualization refers to the

abstract model of the domain which is machine readable,

and where all the elements are explicitly dened and ac-

cepted by the members of a group. Several domain ontolo-

gies dene and organize relevant knowledge about activi-

ties, processes, organizations and strategies, in order to fa-

cilitate information exchange between machines and, be-

tween a human and a machine [2, 3]. Building domain on-

the experts is likely to be out of date. Therefore, there is a

lot of added value in creating domain ontologies from ex-

isting knowledge sources such as structured and unstruc-

tured documents of the domain: texts [5–8], databases [9–

12], XML les [13], existing ontologies [14–16], UML/Meta-

model diagrams [17–19], and source code [12, 20–24]. Al-

though source code is often used to extract concepts and

relations, its full potential is not exploited to extract, for

example, axioms and rules [21, 22]. Indeed, source code

is any fully executable description of a software designed

main are captured and integrated in the source code.

The extraction of knowledge from structured (rela-

tional databases, XML) and unstructured (text, docu-

182 | Azanzi Jiomekong, Gaoussou Camara, and Maurice Tchuente

ments, images) sources is also known as ontology learn-

ing [25–27] that consists in applying statistical techniques,

symbolic techniques or both to (semi-)automatically ex-

tract the ontological knowledge from knowledge sources.

Several authors have proposed the use of symbolic tech-

niques [12, 20, 28] and statistical techniques [23, 29] to ex-

aims at building an integrated platform for epidemiologi-

cal surveillance of tuberculosis in Cameroon. The project

started in 2012 and involves partners from the dierent

area: academy (University of Yaounde 1 in Cameroon),

clinic (fty hospitals in Cameroon), epidemiology (Epi-

demiology and Public Health department of the Centre

Pasteur of Cameroon, and the National Tuberculosis Con-

trol Program), and industry (MEDES in France).

The rest of this paper is organized as follows. In sec-

tion 2, we present an overview of ontology learning. Our

Acquiring knowledge for building an ontology from

scratch, or for rening an existing ontology is costly in

time and resources. Ontology learning techniques are used

to reduce this cost during the knowledge acquisition pro-

cess. Ontology learning refers to the extraction of onto-

logical knowledge from unstructured, semi-structured or

fully structured knowledge sources in order to build an

ontology from them with little human intervention [3, 25,

26, 30]. In this section, we present the basic ontological

knowledge, knowledge sources generally used for ontol-

, this means

Knowledge extraction from source code | 183

knowledge loss or even worse, introduce errors because

misunderstandings that arises frequently in human com-

munication.

2.2.2 Unstructured knowledge sources

Unstructured knowledge sources contain knowledge that

do not have a pre-dened organization. These are all kinds

– XML (Extensible Markup Language): The aim of XML

data conversion to ontologies is the indexing, inte-

gration and enrichment of existing ontologies with

knowledge acquired from XML documents [13];

– UML/meta-model: To learn an ontology from UML

or/and meta-model, one approach is to extract OWL

– Entity-relation diagram: They can be used to learn on-

tologies because they are used to describe the infor-

mation managed by the databases [35];

– Source code [12, 21–23, 28]: Generally, in source code,

the names of data structures, variables, functions are

close to the terms of the domain.

A lot of work has been done on the extraction of ontologi-

cal knowledge from texts, databases, XML les, vocabular-

ies, and the use of ontologies to build or enrich other on-

code whereas axioms and rules are also key elements of

ontologies.

2.3 Ontology learning techniques

To extract knowledge from knowledge sources, many tech-

where each rule consists of a condition and an action

[30].

184 | Azanzi Jiomekong, Gaoussou Camara, and Maurice Tchuente

– Logical rules may be used to discover new

knowledge by deduction (deduce new knowledge

from existing ones) or induction (synthesize new

knowledge from experience). For example, induc-

tive logic programming can be used to learn new

concepts from knowledge sources [5, 25, 26, 37];

ceptual denitions [5, 26, 38].

2. Linguistic approaches (syntactic analysis, morpho-

syntactic analysis, lexico-syntactic pattern parsing,

semantic processing and text understanding) are used

to derive knowledge from text corpus [25, 26]. This

technique can be used to derive an intentional de-

scription of concepts in the form of natural language

description [38].

3. Pattern/Template-driven approach allows to search for

predened keywords, templates or patterns. Indeed,

techniques are inexible because of their strong depen-

dency on the structure of the data. Symbolic techniques

are precise and robust, but can be complex to implement,

and dicult to generalize [26].

2.3.2 Statistic-based techniques

learning concepts by considering clusters of related

terms as concepts and learning taxonomies by orga-

Knowledge extraction from source code | 185

language, the term "class" is used to dene a class, the

terms "if", "else", "switch", "case" are used to dene the

business rules (candidate to become rules). Other terms

dened by the programmer such as "PatientTuberculeux"

are used to represent the names of classes (candidate to be

concept); the term "examenATB" is used to dene the re-

and use Hidden Markov Models (HMMs) for knowledge ex-

traction from Java source code.

3.1 Hidden Markov Models

P(q

) (2)

Where q

and q

are limited to q

and q

is an end of word. The observable output of the system

is the sequence of symbols emitted by the states, but the

is searched.

For example, in automatic translation, one may want

the most probable string sequence that corresponds to the

string to be translated. In this case, instead of taking the

sum of the probabilities, the maximum must be chosen

(equation 3).

P(O | M) =

P(q

) (3)

Before using the model, its parameters (transition

probabilities, emission probabilities and initial probabil-

ities) must be calculated using statistical learning, Baum-

Welch algorithm or Viterbi training [41].

3.2 Source code versus HMM

During software development, it is recommended to write

the source code according to good programming practices,

including naming conventions [50]. These practices in-

form programmers on how to name variables, organize

and present the source code. This organization can be used

to model source code using HMMs (see Fig. 1). For exam-

ple, from Java source code, we can say that at a time t, the

programmer enters a word (e.g. "public" at the beginning

of a Java source le). Thus, the keyword "public" at time t

conditions the next word at time t+1 which in this case can

186 | Azanzi Jiomekong, Gaoussou Camara, and Maurice Tchuente

scribing data, les processing data, user interface les and

conguration les.

3.2.1 Files describing data

These les describe the data to be manipulated and

equally, some constraints on this data (e.g., data types).

In Java EE for example, there are entities whose names are

close to the terms of the domain that will be transformed

into tables in the database. These les often contain cer-

tain rules to verify the reliability of the data. Thus, from

these les, we can retrieve concepts, properties, axioms

certain users (e.g., only the attending physician has

the right to access the data), checking the validity of

a eld (checking whether the data entered in an "age"

eld is of type integer);

– Calculation: For example, converting a date of birth

into an age, determining the date of the next appoint-

ment of a patient, calculating the body mass index of

a patient based on his/her weight and height.

These are the algorithms implementing the business rules

to be applied to the data. They are thus good candidates

Knowledge extraction from source code | 187

preprocessing, entity labeling, formal language transla-

tion, and knowledge validation.

3.3.1 Data collection

dataset necessary for the next steps. In Java les, state-

ments for importing third-party libraries and comments

all the words of the source code and extracting the words

– POST: Any information that follows the knowledge

sought. In some cases, POST is a punctuation charac-

– OTHER: Any other word in the source code that nei-

ther precedes nor follows the knowledge sought.

An example of HMM annotated with labels is given by Fig.

1. Concepts, properties, axioms, and rules are usually ar-

ranged dierently in the source code. We propose the def-

learning on data, specialized algorithms such as Baum-

Welch or Viterbi training [41, 42]. In this paper, we have

chosen statistical learning on data to train the HMMs mod-

eled in the previous paragraphs. Thus, we assumed that

we have access to T source code les labeled f

knowing

that f

words pairs with the word and its label (see Fig. 1) mod-

eled by the equation 4. To train the model, we assume that

we can dene the order in which the dierent words are en-

tered by the programmer. We assume that before entering

rent word depends only on the current label, the following

label depends on the previous label, and so on. The pro-

cess continues until the end of the le.

= [(w

, e

) , ..., (w

, e

] ,

(4)

In the equation 4, w

les respectively. In practice, w

are words contained in

the source code (observations) and e

are the labels of w

used as hidden states.

is the word

les.

P(Q

= a, f

)

(equation 6). The conditional probability that the k

word is

, knowing that the label is

corresponds

188 | Azanzi Jiomekong, Gaoussou Camara, and Maurice Tchuente

the label

normalized with

the fact that the label

word in f

a concept, an attribute, but cannot be a rule.

= w | q

freq((w, b), f

)

to identify concepts and attributes would be: [("public",

PRE), ("class", TARGET), ("Patient", TARGET), ("extends",

TARGET), ("ImogEntityImpl", TARGET), ("{", OTHER), (...),

("int", TARGET), ("age", TARGET), ...]. Tab. 1 presents the

initial vector, which is the probability that the rst label is

PRE, TARGET, POST, or OTHER; Tab. 2 presents the transi-

tion vector containing the frequencies that a state follows

to nd from the les f

determine the most plausible string sequence. From this

string, the hidden states will be identied and the targets

(words that are labeled TARGET) will be extracted. In our

approach, we used Viterbi algorithm which provides an ef-

cient way of nding the most plausible string sequence of

hidden states [51, 52]. The algorithm 1 gives an overview of

the Viterbi Algorithm. More details can be found in [41].

Any source code can then be submitted to the HMM

Knowledge extraction from source code | 189

– ObjectProperty: If two classes ’A’ and ’B’ are candi-

dates to be concepts and ’b’ of type B is declared as

attribute of class ’A’, then classes ’A’ and ’B’ are re-

lated. The attribute ’b’ is an ObjectProperty having ’A’

as domain and ’B’ as range.

– DatatypeProperty: If a class ’A’ is a candidate to be a

the class ’A’ (in Java, the keyword "extends" is used),

then, one can dene a taxonomic relation between the

classes ’B’ and ’A’.

3.3.4 Translation in a formal language

Once all relevant knowledge are identied in the previ-

ous phase, they are automatically translated to a machine

readable language. We use OWL for concepts, properties

and axioms, and SWRL for rules.

3.3.5 Knowledge evaluation

After the extraction process, the evaluation phase permits

or more domain experts who have to judge to what extent

these knowledge are correct; (2) The comparison of the

knowledge extracted (alignment) to gold standards which