Departamento de Informática (DI)
Permanent URI for this community
Browse
Browsing Departamento de Informática (DI) by Title
Now showing 1 - 10 of 212
Results Per Page
Sort Options
- Adaptive and Scalable Database Management with Machine Learning Integration: A PostgreSQL Case StudyPublication . Abbasi, Maryam; Bernardo, Marco V.; Vaz, Paulo; Silva, José; Martins, Pedro; ANTUNES VAZ, PAULO JOAQUIM; Silva, JoséThe increasing complexity of managing modern database systems, particularly in terms of optimizing query performance for large datasets, presents significant challenges that traditional methods often fail to address. This paper proposes a comprehensive framework for integrating advanced machine learning (ML) models within the architecture of a database management system (DBMS), with a specific focus on PostgreSQL. Our approach leverages a combination of supervised and unsupervised learning techniques to predict query execution times, optimize performance, and dynamically manage workloads. Unlike existing solutions that address specific optimization tasks in isolation, our framework provides a unified platform that supports real-time model inference and automatic database configuration adjustments based on workload patterns. A key contribution of our work is the integration of ML capabilities directly into the DBMS engine, enabling seamless interaction between the ML models and the query optimization process. This integration allows for the automatic retraining of models and dynamic workload management, resulting in substantial improvements in both query response times and overall system throughput. Our evaluations using the Transaction Processing Performance Council Decision Support (TPC-DS) benchmark dataset at scale factors of 100 GB, 1 TB, and 10 TB demonstrate a reduction of up to 42% in query execution times and a 74% improvement in throughput compared with traditional approaches. Additionally, we address challenges such as potential conflicts in tuning recommendations and the performance overhead associated with ML integration, providing insights for future research directions. This study is motivated by the need for autonomous tuning mechanisms to manage large-scale, hetero geneous workloads while answering key research questions, such as the following: (1) How can machine learning models be integrated into a DBMS to improve query optimization and workload management? (2) What performance improvements can be achieved through dynamic configuration tuning based on real-time workload patterns? Our results suggest that the proposed framework significantly reduces the need for manual database administration while effectively adapting to evolving workloads, offering a robust solution for modern large-scale data environments.
- Agile-based Requirements Engineering for Machine Learning: A Case Study on Personalized NutritionPublication . Cunha, Carlos; Oliveira, Rafael; Duarte, RuiRequirements engineering is crucial in developing machine learning systems, as it establishes the foundation for successful project execution. Nevertheless, incorporating requirements engineering approaches from traditional software engineering into machine learning projects presents new challenges. These challenges arise from replacing the software logic derived from static software specifications with dynamic software logic derived from data. This paper presents a case study exploring an agile requirement engineering approach popular in traditional software projects to specify requirements in machine learning software. These requirements allow reasoning about the correctness of software and design tests for validation. The absence of software specification in machine learning software is offset by employing data quality metrics, which are assessed using cutting-edge methods for model interpretability. A case study on personalized nutrition and physical activity demonstrated the adequacy of user stories and acceptance criteria format, popular in agile projects, for specifying requirements in the machine learning domain.
- Agregação de redes sociaisPublication . Fonseca, Vítor Manuel Seixas da; Tomé, Paulo Rogério PerfeitoOs últimos anos, em especial os anos de 2009 e 2010, foram claramente marcados pela exponencial divulgação e utilização das redes sociais na Internet como forma primária de comunicação. O Twitter, uma rede de micro blogging, foi dos que mais notoriedade ganhou desde a sua criação, em 2006, devido à simplicidade e rapidez com que se seguem e publicam as actualizações dos utilizadores, quase em tempo real. O Facebook, é o fenómeno do momento e teve desde a sua criação, em 2004, o crescimento mais abrupto da história de qualquer ferramenta de comunicação utilizada até à data, conta com cerca de 500 milhões de utilizadores activos de todo o mundo e é um caso de referência da sociedade moderna, independentemente da geração. Várias outras redes sociais, vocacionadas para vertentes específicas de utilizadores, continuam a nascer ou a sobreviver, contando com comunidades de utilizadores enormes que, com a massificação da Internet, vêem nestas comunidades, a forma mais eficaz e preferencial de obter informações sobre os mais variados assuntos, como exemplos, o MySpace, uma rede social que permite a criação de páginas pessoais, vulgarizada por músicos e bandas em ascensão para divulgar o seu trabalho, O LinkedIn, uma rede vocacionada para relações profissionais e de apresentação no mercado laboral, o Foursquare, que surgiu recentemente, e permite a divulgação da localização geográfica, pontos de interesse e comentários entre utilizadores. Cada uma no seu ramo, ou tentando competir entre elas quando têm conceitos semelhantes, o importante é que há dezenas de redes sociais, e que continuam a surgir cada vez mais, cada uma com o seu vasto leque de utilizadores e cada utilizador, com presença em várias delas. A única forma de não se promover apenas algumas das redes sociais em detrimento das demais, é a agregação! Tornar fácil a presença de um utilizador em várias redes, de uma só vez, comunicando com todos os seus contactos, independentemente da sua rede. Todas as redes sociais actuais disponibilizam acesso às suas plataformas de variadas formas, sendo a maioritária a utilização de browsers em computadores. Por vezes fornecem as suas próprias aplicações, mas o mais importante é a disponibilização de APIs que permitam a programadores desenvolver as suas próprias aplicações de interacção com essa rede social abrindo portas a que criativos, externos à organização, criem aplicações que popularizem a sua rede social. Este trabalho visa à concepção e desenvolvimento de uma plataforma de agregação de várias redes sociais, disponibilizando uma API própria, que permite a implementação de aplicações cliente unificadas, levando o utilizador a abstrair-se da proveniência das actualizações dos seus contactos e publicando as suas próprias para todos eles. Neste trabalho, para completar a solução informática, foram ainda implementadas algumas aplicações cliente, perfeitamente escaláveis e que tentam, utilizando a plataforma, colmatar os problemas das aplicações agregadoras que têm vindo, recentemente, a ser criadas, mas que acedem directamente às APIs das redes sociais, e são, assim, constantemente penalizadas pelas alterações, por vezes sem aviso prévio, dos métodos e interfaces de acesso às suas funcionalidades, provocando que estas deixem de funcionar e necessitem de constantes revisões e desenvolvimentos. Trata-se de um sistema em modelo cliente-servidor, que implementa o conceito de agregação feito no servidor, como que criando uma rede social paralela que agrega as demais, criando uma credencial de acesso única que permite ao utilizador aceder à plataforma em várias aplicações, sem que tenha que configurar consecutivamente as suas contas. No caso de estudo actual, a chave de agregação foi o MSISDN (número de telemóvel) de um operador de comunicações móveis e aplicações vocacionadas para dispositivos móveis.
- An Advertising Real-Time Intelligent and Scalable Framework for Profiling Customers EmotionsPublication . Alves, Leandro; Oliveira, Pedro; Henriques, João; Bernardo, Marco V.; Wanzeller, Cristina; Caldeira, FilipeThe advertising industry is continuously looking up for effective ways to communicate to customers to impact their purchasing. Usually, profiling them is a time-consuming offline activity. Therefore, it becomes necessary to reduce costs and time to address consumers’ needs. This work proposes a scalable framework enabled by a Machine Learning (ML) model to profile customers to identify their emotions to help to drive campaigns. A multi-platform mobile application continuously profiles consumers crossing the front stores. Profiling customers according to their age and hair color, the color of their eyes, and emotions (e.g. happiness, sadness, disgust, fear) will help companies to make the most suitable advertisement (e.g. to predict whether the advertising tones on the front store are the adequate ones). All that data are made available in web portal dashboards, wherein subscribers can take their analysis. Such results from the analysis data help them to identify tendencies regarding the culture and age, and drive companies to fit front stores accordingly (e.g. to discover the right tones for the season). This framework can help to develop new innovative cost-effective business models at scale by driving in real-time the advertisements to a huge number of consumers to maximize their impact and centralizing the data collected from a large number of stores to design future campaigns.
- An automated closed-loop framework to enforce security policies from anomaly detectionPublication . Henriques, João; Caldeira, Filipe; Cruz, Tiago; Simões, PauloDue to the growing complexity and scale of IT systems, there is an increasing need to automate and streamline routine maintenance and security management procedures, to reduce costs and improve productivity. In the case of security incidents, the implementation and application of response actions require significant efforts from operators and developers in translating policies to code. Even if Machine Learning (ML) models are used to find anomalies, they need to be regularly trained/updated to avoid becoming outdated. In an evolving environment, a ML model with outdated training might put at risk the organization it was supposed to defend. To overcome those issues, in this paper we propose an automated closed-loop process with three stages. The first stage focuses on obtaining the Decision Trees (DT) that classify anomalies. In the second stage, DTs are translated into security Policies as Code based on languages recognized by the Policy Engine (PE). In the last stage, the translated security policies feed the Policy Engines that enforce them by converting them into specific instruction sets. We also demonstrate the feasibility of the proposed framework, by presenting an example that encompasses the three stages of the closed-loop process. The proposed framework may integrate a broad spectrum of domains and use cases, being able for instance to support the decide and the act stages of the ETSI Zero-touch Network & Service Management (ZSM) framework.
- An Evaluation of How Big-Data and Data Warehouses Improve Business Intelligence Decision MakingPublication . Martins, Anthony; Martins, Pedro; Caldeira, Filipe; Sá, Filipe; Rocha, {\'AAnalyze and understand how to combine data warehouse with business intelligence tools, and other useful information or tools to visualize KPIs are critical factors in achieving the goal of raising competencies and business results of an organization. This article reviews data warehouse concepts and their appropriate use in business intelligence projects with a focus on large amounts of information. Nowadays, data volume is more significant and critical, and proper data analysis is essential for a successful project. From importing data to displaying results, there are crucial tasks such as extracting information, transforming it analyzing, and storing data for later querying. This work contributes with the proposition of a Big Data Business Intelligence architecture for an efficiently BI platform and the explanation of each step in creating a Data Warehouse and how data transformation is designed to provide useful and valuable information. To make valuable information useful, Business Intelligence tools are presented and evaluates, contributing to the continuous improvement of business results.
- An evolved security architecture for distributed industrial automation and control systemsPublication . Rosa, L.; Proença, J.; Henriques, João; Graveto, V.; Cruz, T.; Simões, P.; Caldeira, Filipe; Monteiro, E.Over the recent years, control and sensor systems used for IACS (Industrial Automation and Control Systems) have become more complex, due to the increasing number of interconnected distributed devices, sensors and actuators. Such components are often widely dispersed in the field – this is the case for microgeneration (wire-to-water generation, solar or wind), smart metering, oil and gas distribution or smart water management, among others. This IoT (Internet of Things)-centric IACS paradigm expands the infrastructure boundaries well beyond the single or aggregated-plant, mono-operator vision (mostly associated with geographically constrained systems topologies), being dispersed over a large geographic area, with increasingly small areas of coverage as we progress towards its periphery. This situation calls for a different approach to cyber threat detection, which is one of the most relevant contributions of the ATENA (Advanced Tools to assEss and mitigate the criticality of ICT components and their dependencies over critical infrAstructures) H2020 project (ATENA 2016). This paper presents and describes the ATENA cyber-security architecture, designed for the emerging generation of distributed IoT IACS, leveraging technologies such as Software Defined Networking/Network Function Virtualization and Big data event processing) within the scope of a cyber-detection architecture designed to deal with the inherent challenges of dispersed IACS, involved different operator domains.
- An Intelligent and Scalable IoT Monitoring Framework for Safety in Civil Construction WorkspacesPublication . Ferreira, Carolina; Correia, Luciano; Lopes, Manuel; Henriques, João; Martins, Pedro; Wanzeller, Cristina; Caldeira, FilipeKeeping civil construction workers safe is an important challenge due to working conditions and low technological support due to the inherent costs. This work surveys the literature and proposes a scalable framework for monitoring workers to minimize the response time with real-time warnings in hazardous situations or safety incidents. From the literature, it was possible to devise a gap in business addressing this problem. To address this problem, this work proposes an IoT scalable framework able to scale to a large number of civil construction companies with a large number of workers in order. The results from this work demonstrate the feasibility of the proposed framework and the low cost of the IoT solution and the scalability of the framework offers the opportunity to leverage new innovative business models capable to leverage their revenues.
- An Overview on Cloud Services for Human TrackingPublication . Martins, Manuel; Mota, David; Martins, Pedro; Abbasi, Maryam; Caldeira, FilipeThis paper reflects the intention to test the use of public cloud services to assess the presence of humans in a given space, more precisely, multiple stores, with the least effort and in the fastest way. It is also intended to demonstrate that the use of the public cloud can be an instrument of added value in business areas and research areas. In the specific case, the cloud was used to train and use artificial intelligence models.
- An overview on how to develop a low-code application using OutSystemsPublication . Martins, Ricardo; Caldeira, Filipe; Sá, Filipe; Abbasi, Maryam; Martins, PedroThe motivation for developing a self-service platform for employees arises precisely from the idea that in all organizations there are tasks that could be automated in order to redirect work resources to more important tasks. The proposed application consists of the development of a self-service platform, for personal information and scheduling tasks, aimed at the employees instead of all the solutions that are in the market that aim their platform to the Human Resources. We focus on the employers giving them more responsibility to make their own personal management like, change their personal info, book their vacations and other, giving to the Human Resources the tasks of managing all these actions made by the employers. At the end of the work, it is expected that the final solution to be considered as an example of success with regards to the theme of business automation and innovation, using the low-code application Outsystems to perform the full proposed application development.