Preguiça, N., R. Rodrigues, C. Honorato, and J. M. Lourenço,
"Byzantium: Byzantine-fault-tolerant database replication providing snapshot isolation",
Proceedings of the Fourth conference on Hot topics in system dependability, Berkeley, CA, USA, USENIX Association, pp. 9–9, 2008.
AbstractDatabase systems are a key component behind many of today's computer systems. As a consequence, it is crucial that database systems provide correct and continuous service despite unpredictable circumstances, such as software bugs or attacks. This paper presents the design of Byzantium, a Byzantine fault-tolerant database replication middleware that provides snapshot isolation (SI) semantics. SI is very popular because it allows increased concurrency when compared to serializability, while providing similar behavior for typical workloads. Thus, Byzantium improves on existing proposals by allowing increased concurrency and not relying on any centralized component. Our middleware can be used with off-the-shelf database systems and it is built on top of an existing BFT library.
Cunha, J. C., P. D. Medeiros, J. M. Lourenço, V. Duarte, J. Vieira, B. Moscão, D. Pereira, and R. Vaz,
"The DOTPAR Project: Towards a Framework Supporting Domain Oriented Tools for Parallel and Distributed Processing",
Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking (HPCN'98), London, UK, Springer-Verlag, pp. 952–954, 1998.
AbstractWe discuss the problem of building domain oriented environments by a composition of heterogeneous application components and tools. We describe several individual tools that support such environments, namely a distributed monitoring and control tool (DAMS), a process-based distributed debugger (PDBG) and a heterogeneous interconnection model (PHIS). We discuss our experience with the development of a Problem Oriented Environment in the domain of genetic algorithms, obtained by a composition of heterogeneous tools and application components.
Dias, R. J., J. M. Lourenço, and N. Preguiça,
"Efficient and Correct Transactional Memory Programs Combining Snapshot Isolation and Static Analysis",
Proceedings of the 3rd USENIX Conference on Hot Topics in Parallelism (HotPar'11), Berkeley, USA, Usenix Association, May, 2011.
AbstractConcurrent programs may suffer from concurrency anomalies that may lead to erroneous and unpredictable program behaviors. To ensure program correctness, these anomalies must be diagnosed and corrected. This paper addresses the detection of both low- and high-level anomalies in the Transactional Memory setting. We propose a static analysis procedure and a framework to address Transactional Memory anomalies. We start by dealing with the classic case of low-level dataraces, identifying concurrent accesses to shared memory cells that are not protected within the scope of a memory transaction. Then, we address the case of high-level dataraces, bringing the programmer's attention to pairs of memory transactions that were misspecified and should have been combined into a single transaction. Our framework was applied to a set of programs, collected form different sources, containing well known low- and high-level anomalies. The framework demonstrated to be accurate, confirming the effectiveness of using static analysis techniques to precisely identify concurrency anomalies in Transactional Memory programs.
Cunha, J. C., J. M. Lourenço, J. Vieira, B. Moscão, and D. Pereira,
"A Framework to Support Parallel and Distributed Debugging",
Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking (HPCN'98), London, UK, Springer-Verlag, pp. 708–717, 1998.
AbstractWe discuss debugging prototypes that can easily support new functionalities, depending on the requirements of high-level computational models, and allowing a coherent integration with other tools in a software engineering environment. Concerning the first aspect, we propose a framework that identifies two distinct levels of functionalities that should be supported by a parallel and distributed debugger using: a process and thread-level, and a coordination level concerning sets of processes or threads. An incremental approach is used to effectively develop prototypes that support both functionalities. Concerning the second aspect, we discuss how the interfacing with other tools has influenced the design of a process-level debugging interface (PDBG) and a distributed monitoring and control layer called (DAMS).
Duro, N., R. Santos, J. M. Lourenço, H. Paulino, and J. Martins,
"Open Virtualization Framework for Testing Ground Systems",
Proceedings of the 8th Workshop on Parallel and Distributed Systems (PADTAD'10), New York, NY, USA, ACM, pp. 67–73, 2010.
AbstractThe recent developments in virtualization change completely the panorama of the Hardware/OS deployment. New bottlenecks arise in the deployment of application stacks, where IT industry will spend most of the time to assure automation. VIRTU tool aims at managing, configuring and testing distributed ground applications of space systems on a virtualized environment, based on open tools and cross virtualization support. This tool is a spin-off of previous activities performed by the European Space Operations Center (ESOC) and thus it covers the original needs from the ground data systems infrastructure division of the European Space Agency. VIRTU is a testing oriented solution. Its ability to group several virtual machines in an assembly provides the means to easily deploy a full testing infrastructure, including the client/server relationships. The possibility of making on-demand request of the testing infrastructure will provide some infrastructure optimizations, specially having in mind that ESA maintains Ground Control software of various missions, and each mission cam potentially have a different set of System baselines and last up to 15 years. The matrix array of supported system combinations is therefore enormous and any improvement on the process provides substantial benefits to ESA, by reducing the effort and schedule of each maintenance activity. The ESOC's case study focuses on the development and validation activities of infrastructure or mission Ground Systems solutions. The Ground Systems solutions are typically composed of distributed systems that could take advantage of virtualized environments for testing purposes. Virtualization is used as way to optimize maintenance for tasks such as testing new releases and patches, test different system's configurations and replicate tests. The main benefits identified are related to deployment test environment and the possibility to have on-demand infrastructure.
Pessanha, V., R. J. Dias, J. M. Lourenço, E. Farchi, and D. Sousa,
"Practical verification of high-level dataraces in transactional memory programs",
Proceedings of 9th the Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging, New York, NY, USA, ACM, pp. 26–34, July, 2011.
AbstractIn this paper we present MoTh, a tool that uses static analysis to enable the automatic verification of concurrency anomalies in Transactional Memory Java programs. Currently MoTh detects high-level dataraces and stale-value errors, but it is extendable by plugging-in sensors, each sensor implementing an anomaly detecting algorithm. We validate and benchmark MoTh by applying it to a set of well known concurrent buggy programs and by close comparison of the results with other similar tools. The results achieved so far are very promising, yielding good accuracy while triggering only a very limited number of false warnings.
Dias, R. J., V. Pessanha, and J. M. Lourenço,
"Precise Detection of Atomicity Violations",
Haifa Verification Conference, Haifa, Israel, Springer Berlin / Heidelberg, Nov 2012.
AbstractConcurrent programs that are free of unsynchronized ac- cesses to shared data may still exhibit unpredictable concurrency errors called atomicity violations, which include both high-level dataraces and stale-value errors. Atomicity violations occur when programmers make wrong assumptions about the atomicity scope of a code block, incorrectly splitting it in two or more atomic blocks and allow them to be interleaved with other atomic blocks. In this paper we propose a novel static analysis algorithm that works on a dependency graph of program variables and detects both high-level dataraces and stale-value errors. The algorithm was implemented for a Java Bytecode analyzer and its effectiveness was evaluated with some well known faulty programs. The results obtained show that our algorithm performs better than previous approaches, achieving higher precision for small and medium sized programs, making it a good basis for a practical tool.
Martins, H. R. L., J. Soares, J. M. Lourenço, and N. Preguiça,
"Replicação Multi-nível de Bases de Dados em Memória",
Proceedings of INForum Simpósio de Informática, Lisbon, Portugal, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa, pp. 190–201, sep, 2013.
AbstractOs serviços Web são frequentemente suportados por sistemas com uma arquitetura em camadas, sendo utilizadas bases de dados relacionais para armazenamento dos dados. A replicação dos diversos componentes tem sido uma das formas utilizadas para obter melhorarias de escalabilidade destes serviços. Adicionalmente, a utilização de bases de dados em memória permite alcançar um desempenho mais elevado. No entanto é conhecida a fraca escalabilidade das bases de dados com o número de núcleos em máquinas multi-núcleo. Neste artigo propomos uma nova abordagem para lidar com este problema, intitulada MacroDDB. Utilizando uma solução de replicação hierárquica, a nossa proposta, replica a base da dados em vários nós, sendo que cada nó, por sua vez, executa um conjunto de réplicas da base de dados. Esta abordagem permite assim lidar com a falta de escalabilidade das bases de dados relacionais em máquinas multi-núcleo, o que por sua vez melhora a escalabilidade geral dos serviços.
Silva, J. A., T. M. Vale, J. M. Lourenço, and H. Paulino,
"Replicação Parcial com Memória Transacional Distribuída",
Proceedings of INForum Simpósio de Informática, Lisbon, Portugal, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa, pp. 310–321, 2013.
AbstractOs sistemas de memória transacional distribuída atuais recorrem essencialmente à distribuição ou à replicação total para distribuir os seus dados pelos múltiplos nós do sistema. No entanto, estas estratégias de replicação de dados apresentam limitações. A distribuição não oferece tolerância a falhas e a replicação total limita a capacidade de armazenamento do sistema. Nesse contexto, a replicação parcial de dados surge como uma solução intermédia, que combina o melhor das duas anteriores com o intuito de mitigar as suas desvantagens. Esta estratégia tem sido explorada no contexto das bases de dados distribuídas, mas tem sido pouco abordada no contexto da memória transacional e, tanto quanto sabemos, nunca antes tinha sido incorporada num sistema de memória transacional distribuída para uma linguagem de propósito geral. Assim, neste artigo propomos e avaliamos uma infraestrutura para replicação parcial de dados para programas Java bytecode, que foi desenvolvida com base num sistema já existente de memória transacional distribuída. A modularidade da infraestrutura que apresentamos permite a implementação de múltiplos algoritmos e, por conseguinte, avaliar em que contextos de utilização (workloads, número de nós, etc.) a replicação parcial se apresenta como uma alternativa viável a outras estratégias de replicação de dados.
Silva, J. A., T. M. Vale, R. J. Dias, H. Paulino, and J. M. Lourenço,
"Supporting Multiple Data Replication Models in Distributed Transactional Memory",
Proceedings of the 2015 International Conference on Distributed Computing and Networking, Goa, India, ACM, pp. 11:1–11:10, 2015.
AbstractDistributed transactional memory (DTM) presents itself as a highly expressive and programmer friendly model for concurrency control in distributed programming. Current DTM systems make use of both data distribution and replication as a way of providing scalability and fault tolerance, but both techniques have advantages and drawbacks. As such, each one is suitable for different target applications, and deployment environments. In this paper we address the support of different data replication models in DTM. To that end we propose ReDstm, a modular and non-intrusive framework for DTM, that supports multiple data replication models in a general purpose programming language (Java). We show its application in the implementation of distributed software transactional memories with different replication models, and evaluate the framework via a set of well-known benchmarks, analysing the impact of the different replication models on memory usage and transaction throughput.
Silva, J. A., T. M. Vale, R. J. Dias, H. Paulino, and J. M. Lourenço,
"Supporting Partial Data Replication in Distributed Transactional Memory",
Proceedings of Joint Euro-TM/MEDIAN Workshop on Dependable Multicore and Transactional Memory Systems, Vienna, Austria, jan, 2014.
Abstract
Monteiro, R., J. M. Lourenço, and H. Paulino,
"Um Armazenamento Distribuído para uma Rede de Dispositivos Móveis",
Proceedings of INForum Simpósio de Informática, Covilhã, Portugal, sep, 2015.
AbstractOs dispositivos móveis em proximidade geográfica representam um conjunto de recursos inexplorados, tanto em termos de capacidade de processamento como de rmazenamento, o que abre caminho para novas aplicações com oportunidades e desafios únicos. Os sistemas atuais de partilha de dados (e. g., fotos, música, vídeos) para dispositivos móveis exigem que exista conectividade com a Internet para funcionarem. No entanto, em ambientes onde a conectividade com a Internet não é constante ou de boa qualidade (e. g., eventos desportivos e concertos), ou em locais remotos onde as infraestruturas de rede não existem, é difícil (ou mesmo impossível) partilhar dados entre vários dispositivos móveis. Para resolver este problema, os dispositivos móveis podem formar uma rede ad hoc para compartilhar os seus dados e recursos. Neste artigo propomos um sistema de armazenamento distribuído para partilha de dados entre dispositivos móveis de uso diário, e. g., smartphones e tablets, usando um mecanismo de melhor esforço para garantir persistência e disponibilidade de dados suportando churn (entrada e saída inesperada de dispositivos).
Silva, J. A., J. M. Lourenço, and H. Paulino,
"Um Mecanismo de Caching para o Protocolo {SCORe}",
Proceedings of INForum Simpósio de Informática, Porto, Portugal, FEUP Edições, pp. 260–275, sep, 2014.
AbstractOs protocolos de replicação parcial de dados apresentam um grande potencial de escalabilidade. O SCORe é um protocolo para replicação parcial proposto recentemente que faz uso de controlo de concorrência multi-versão. Neste artigo abordamos um dos problemas principais que afeta o desempenho deste tipo de protocolos: a localidade dos dados, i.e., pode-se dar o caso do nó local não ter uma cópia dos dados a que pretende aceder, e nesse caso é necessário realizar uma ou mais operações de leitura remota. Assim, a não ser que se empreguem técnicas para melhorar a localidade no acesso aos dados, o número de operações de leitura remota aumenta com o tamanho do sistema, acabando por afetar o desempenho do mesmo. Nesse sentido, introduzimos um mecanismo de caching que permite replicar cópias de dados remotos de maneira a que seja poss{\'ı}vel servir localmente dados remotos enquanto que se mantém a consistência dos mesmos e a escalabilidade oferecida pelo protocolo. Avaliamos o mecanismo de caching com um benchmark conhecido da literatura e os resultados experimentais mostram resultados animadores com algum aumento no desempenho do sistema e uma redução considerável da quantidade de operações de leitura remota.
Lourenço, J. M., R. J. Dias, J. Luís, M. Rebelo, and V. Pessanha,
"Understanding the Behavior of Transactional Memory Applications",
Proceedings of the 7th Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging (PADTAD'09), New York, NY, USA, ACM, pp. 31–39, 2009.
AbstractTransactional memory is a new trend in concurrency control that was boosted by the advent of multi-core processors and the near to come many-core processors. It promises the performance of finer grain with the simplicity of coarse grain threading. However, there is a clear absence of software development tools oriented to the transactional memory programming model, which is confirmed by the very small number of related scientific works published until now. This paper describes ongoing work. We propose a very low overhead monitoring framework, developed specifically for monitoring TM computations, that collects the transactional events into a single log file, sorted in a global order. This framework is then used by a visualization tool to display different types of charts from two categories: statistical charts and thread-time space diagrams. These last diagrams are interactive, allowing to identify conflicting transactions. We use the visualization tool to analyse the behavior of two different, but similar, testing applications, illustrating how it can be used to better understand the behavior of these transactional memory applications.