Background: Often, claims in Component-Based Development (CBD) are only sup-
ported by qualitative expert opinion, rather than by quantitative data. This contrasts
with the normal practice in other sciences, where a sound experimental validation of
claims is standard practice. Experimental Software Engineering (ESE) aims to bridge
this gap. Unfortunately, it is common to find experimental validation efforts that are
hard to replicate and compare, to build up the body of knowledge in CBD.
Objectives: In this dissertation our goals are (i) to contribute to evolution of ESE, in
what concerns the replicability and comparability of experimental work, and (ii) to ap-
ply our proposals to CBD, thus contributing to its deeper and sounder understanding.
Techniques: We propose a process model for ESE, aligned with current experimen-
tal best practices, and combine this model with a measurement technique called
Ontology-Driven Measurement (ODM). ODM is aimed at improving the state of prac-
tice in metrics definition and collection, by making metrics definitions formal and ex-
ecutable, without sacrificing their usability. ODM uses standard technologies that can
be well adapted to current integrated development environments.
Results: Our contributions include the definition and preliminary validation of a pro-
cess model for ESE and the proposal of ODM for supporting metrics definition and
collection in the context of CBD. We use both the process model and ODM to perform
a series experimental works in CBD, including the cross-validation of a component
metrics set for JavaBeans, a case study on the influence of practitioners expertise in
a sub-process of component development (component code inspections), and an ob-
servational study on reusability patterns of pluggable components (Eclipse plug-ins).
These experimental works implied proposing, adapting, or selecting adequate ontolo-
gies, as well as the formal definition of metrics upon each of those ontologies.
Limitations: Although our experimental work covers a variety of component models
and, orthogonally, both process and product, the plethora of opportunities for using
our quantitative approach to CBD is far from exhausted.
Conclusions: The main contribution of this dissertation is the illustration, through
practical examples, of how we can combine our experimental process model with
ODM to support the experimental validation of claims in the context of CBD, in a re-
peatable and comparableway. In addition, the techniques proposed in this dissertation
are generic and can be applied to other software development paradigms.
] 