Scheduling and processing decisions in multiplatform data systems can lead to efficient execution of workflow tasks across available platforms. In this work, we discuss how to create a cross-platform system for Data Science tasks that leverages modern data abstractions, such as visual programming and DataFrames. Our approach uses an extended graph to represent workflows, capturing various execution aspects, including dataset evolution, execution costs, and data migration costs across different platforms and environments. This graph can be further transformed based on predefined rules, optimization techniques, and the application of machine learning tools to guide decision-making during the scheduling of the blocks that compose the workflows in question. Empirically, we demonstrate that our prototype, which initially supports Spark and Pandas, can identify not only the best platform with 90.8% accuracy, or the best execution environment for an entire workflow (83.8%), but also achieve performance gains of up to 2.5 times by scheduling tasks on the best platform for each activity.

A enumeração dos ativos conectados à rede é uma etapa importante na análise de vulnerabilidades. Nesse contexto, a utilização de motores de busca, como o Shodan, vem se tornando popular para a identificação de serviços e dispositivos acessíveis pela Internet. No entanto, as informações inferidas por esses motores nem sempre são completas e, muitas vezes, não acompanham a velocidade com que novos serviços surgem. O presente trabalho apresenta uma solução para a enumeração eficiente de serviços a partir de fingerprints. Para validar nossa solução, comparamos as informações obtidas pelo nosso arcabouço com as fornecidas pelo Shodan. Por exemplo, nossa solução permite o aumento da identificação de serviços, como o sistema operacional, em 1,6 vezes e informações sobre o hardware em até 14 vezes. Apresentamos também dois casos de uso que mostram como nosso arcabouço pode auxiliar na análise de vulnerabilidades fornecendo informações mais precisas.

This text summarizes the key contributions of the dissertation entitled "Graph Pattern Mining: consolidating models, systems, and abstractions", approved in the Graduate Program in Computer Science of the Federal University of Minas Gerais (DCC/UFMG) Graph Pattern Mining (GPM) refers to a class of problems involving the processing of subgraphs extracted from larger graphs. Applications to GPM algorithms include querying subgraphs with given properties of interest, identifying motif structures in biological networks, among others. GPM algorithms are challenging to develop and thus, general-purpose GPM systems emerge as a solution to improve the user experience with such algorithms. In this dissertation we propose a primitive-based model for representing GPM algorithms, a distributed system implementing this model, and an extensive experimental study of popular algorithms used in GPM systems. We demonstrate empirically the effectiveness of our model by showing competitive performance without sacrificing the expressiveness of algorithms.

Motores de busca de dispositivos desempenham um papel importante no processo de rastreamento de vulnerabilidades. Entretanto, existem poucos estudos que analisam as capacidades desses sistemas. Nosso trabalho compara dois sistemas de busca populares, o Censys e o Shodan, no contexto da internet brasileira. Devido ao grande volume de dados gerados pelos motores de busca, implementamos uma abstração de dados única que simplifica consultas complexas e que permite a integração de dados externos complementares. Propomos um arcabouço para avaliar ambos os sistemas. Nossos resultados apontam diferenças significativas no modo de operação dos dois sistemas, sendo o Censys o sistema com maior cobertura de dispositivos no Brasil, enquanto o Shodan possui uma maior diversidade de serviços detectados e atualizações mais frequentes. A combinação dos dois motores aumenta a quantidade de serviços detectados e a taxa de varredura em até 1,8 vezes, ao mesmo tempo que obtemos mais detalhes sobre os serviços avaliados.

Datacenters consomem 1% de toda a energia mundial. Este trabalho estudou os fatores que compõem o consumo de energia ao processar big data sobre Spark. Caracterizou os fatores mais significativos a partir de um conjunto de testes em cargas big data, que foram processadas com a energia medida fim a fim agregada a métricas da infraestrutura. Um estudo multifatorial sobre os resultados, demonstrou que: (i) nem sempre o acréscimo de recursos acelera o processamento a ponto se economizar energia; (ii) a forma como o recurso é ofertado (tamanho de VM) também altera o desempenho e o consumo; (iii) uma monitoração integrada a uma abordagem multicamada é fundamental para elencar fatores que podem ser a chave na economia de energia.

Sistemas de busca como o Shodan desempenham um papel importante no processo de mapeamento de dispositivos e no rastreamento de vulnerabilidades. No entanto, a integração, a codificação e o processamento das análises pelos especialistas do domínio podem ser complexas devido ao grande volume de dados gerado por esses sistemas. Nosso trabalho apresenta uma nova abstração, de alto nível, para análises eficientes, e uma API de bases externas para fácil integração. Em nossa abstração, as complexidades do código são ocultadas a partir de operadores funcionais próximos ao contexto da área de domínio. Validamos a usabilidade da nossa biblioteca a partir de um estudo de caso envolvendo vulnerabilidades críticas para a LGPD. Nossos resultados identificam uma série de bancos de dados acessíveis na internet e sistemas sendo expostos por meses com várias falhas de alto risco de confidencialidade.

Graph Pattern Mining (GPM) refers to a class of problems involving the processing of sub graphs extracted from larger graphs. Applications to GPM algorithms include querying subgraphs, identifying motif structures in biological networks, characterizing social media, among others. G PM algorithms are challenging to develop due to subroutines that include non-trivial graph theory concepts and methods such as isomorphism. General-purpose GPM systems have emerged as a solution to improve the user experience with such algorithms. However, existing general-purpose GPM systems are heterogeneous in terms of implementation details, hardware environment and algorithmic paradigms for sub graph exploration and thus, observations taken from the experimental results alone may not clearly identify when a particular paradigm prevails over another. In this work we present an experimentation analysis of popular paradigms used in existing GPM systems. In order to provide a fair and comprehensive evaluation of various algorithmic paradigms we implement all of them within a single GPM framework. Our results show that no single paradigm is best for every application scenario, and we believe that our findings may guide practitioner towards more optimized GPM systems in the future.

Serviços de monitoramento como o Shodan são cada vez mais populares no rastreamento de aplicações e vulnerabilidades na Internet. Neste artigo caracterizamos e discutimos vulnerabilidades encontradas na Internet brasileira utilizando dados de monitoramento provenientes do Shodan. Além disso, discutimos métodos de Ciências dos Dados para melhorar a escalabilidade e a qualidade de análises; também combinamos metadados de fontes complementares sobre rede e vulnerabilidades para extrair resultados e conclusões mais assertivos. Nossa caracterização expõe diversas vulnerabilidades de alta severidade na rede brasileira, algumas catalogadas há mais de cinco anos e que continuam prevalentes até hoje. Esperamos que as análises apresentadas neste artigo incentivem organizações a implantarem atualizações e mecanismos de proteção para mitigar essas ameaças.

O tema deste artigo é o estudo e a aplicação dos princípios da abordagem de confiança zero (zero trust) para segurança cibernética a um ambiente de big data. A partir da definição de um ambiente computacional big data baseado em um ambiente real de produção, definimos uma arquitetura de segurança baseada na confiança zero e a instanciamos para o ambiente em questão, abordando alguns casos de autenticação e autorização. Para a instanciação da arquitetura, várias ferramentas de software são utilizadas, diversas delas já disponíveis no ambiente real. As soluções propostas são ilustradas por casos de uso. A conclusão é que essa a tarefa é complexa, porém factível, e que pode ser implementada com as ferramentas usuais de ambientes big data.

In recent years, the areas of High-Performance Computing (HPC) and massive data processing (also know as Big Data) have been in a convergence course, since they tend to be deployed on similar hardware. HPC systems have historically performed well in regular, matrix-based computations; on the other hand, Big Data problems have often excelled in fine-grained, data parallel workloads. While HPC programming is mostly task-based, like COMPSs, popular Big Data environments, like Spark, adopt the functional programming paradigm. A careful analysis shows that there are pros and cons to both approaches, and integrating them may yield interesting results. With that reasoning in mind, we have developed DDF, an API and library for COMPSs that allows developers to use Big Data techniques while using that HPC environment. DDF has a functional-based interface, similar to many Data Science tools, that allows us to use dynamic evaluation to adapt the task execution in run time. It brings some of the qualities of Big Data programming, making it easier for application domain experts to write Data Analysis jobs. In this article we discuss the API and evaluate the impact of the techniques used in its implementation that allow a more efficient COMPSs execution. In addition, we present a performance comparison with Spark in several application patterns. The results show that each technique significantly impacts the performance, allowing COMPSs to outperform Spark in many use cases.

Campanhas de phishing frequentemente utilizam páginas Web que imitam páginas legítimas para enganar as vítimas. Apesar dos esforços da comunidade científica em combater essa atividade, o phishing fica cada vez mais sofisticado e continua fazendo vítimas. Neste artigo apresentamos um novo arcabouço de monitoramento de páginas de phishing que combina técnicas que proveem escalabilidade e efetividade. Também estudamos os compromissos existentes na complexa tarefa de construir modelos para identificar páginas de phishing. Mostramos que bases de dados representativas e atributos do conteúdo das páginas são cruciais para construir modelos gerais. Nosso arcabouço de monitoramento e identificação de páginas de phishing foi aplicado a centenas de milhares de e-mails diários, identificando uma centena de páginas de phishing, uma redução de três ordens de magnitude, e serve também de ponto de partida para direcionar esforços futuros de combate ao phishing.

Software-defined networking (SDN) has provided a new paradigm for network management by allowing a central controller to program the underlying switches directly. However, OpenFlow, the de facto standard API for communicating with the switches, has limited visibility into the network headers, hindering innovations in the data plane and overloading the controller when a more sophisticated network application is needed. In this work, we leverage existing capabilities of modern switches to increase the abstraction power of OpenFlow and enrich the functionalities performed on the data plane of a network. We present an architecture that extends OpenFlow to support matching rules with domain names and provides data-plane operations that are only supported by the controller in existing approaches. Our architecture provides a better abstraction for programming the network and enables more concise policy specifications by requiring fewer rules in the switch flow table. To realize our architecture, we developed a prototype of a switch and a controller to handle the domain name extensions. We presented an application use case for blocking unwanted traffic required for Telecom companies. Our experimental results show that our solution reduces latency, number of rules in the switch, and number of packets sent to the controller. We also show that the new abstraction we provide can significantly reduce the code size of a network application.

Cloud computing is a general term that involves delivering hosted services over the Internet. With the accelerated growth of the volume of data used by applications, many organizations have moved their data into cloud servers to provide scalable, reliable and highly available services. A particularly challenging issue that arises in the context of cloud storage systems with geographically-distributed data replication is how to reach a consistent state for all replicas. This survey reviews major aspects related to consistency issues in cloud data storage systems, categorizing recently proposed methods into three categories: (1) fixed consistency methods, (2) configurable consistency methods and (3) consistency monitoring methods.

Recently, efforts have been made to bring together the areas of high-performance computing (HPC) and massive data processing (Big Data). Traditional HPC frameworks, like COMPSs, are mostly task-based, while popular big-data environments, like Spark, are based on functional programming principles. The earlier are know for their good performance for regular, matrix-based computations; on the other hand, for fine-grained, data-parallel workloads, the later has often been considered more successful. In this paper we present our experience with the integration of some dataflow techniques into COMPSs, a task-based framework, in an effort to bring together the best aspects of both worlds. We present our API, called DDF, which provides a new data abstraction that addresses the challenges of integrating Big Data application scenarios into COMPSs. DDF has a functional-based interface, similar to many Data Science tools, that allows us to use dynamic evaluation to adapt the task execution in runtime. Besides the performance optimization it provides, the API facilitates the development of applications by experts in the application domain. In this paper we evaluate DDF's effectiveness by comparing the resulting programs to their original versions in COMPSs and Spark. The results show that DDF can improve COMPSs execution time and even outperform Spark in many use cases.

Mudanças de caminho causadas por eventos como engenharia de tráfego, alteração de parcerias de troca de tráfego, ou falhas de enlace impactam vários caminhos na Internet. Plataformas de monitoramento topológico realizam medições periódicas usando traceroute para um grande número de destinos. Esta abordagem, porém, é inadequada para identificar precisamente a extensão do impacto de eventos de roteamento. Por exemplo, uma falha de enlace pode ser restaurada antes que todas as rotas sejam medidas. Neste trabalho apresentamos estratégias de medição que minimizam o custo de sondagem para identificar caminhos impactados por um evento de roteamento. Nossos resultados mostram que é possível identificar o conjunto de caminhos impactados por um evento de forma eficiente. Nossos resultados indicam ainda que, quando integradas a um sistema estado-da-arte de rastreamento de mudanças de caminhos, nossas estratégias mais que dobram o número de mudanças detectadas.

In this paper we propose Fractal, a high performance and high productivity system for supporting distributed graph pattern mining (GPM) applications. Fractal employs a dynamic (auto-tuned) load-balancing based on a hierarchical and locality-aware work stealing mechanism, allowing the system to adapt to different workload characteristics. Additionally, Fractal enumerates subgraphs by combining a depth-first strategy with a from scratch processing paradigm to avoid storing large amounts of intermediate state and, thus, improves memory efficiency. Regarding programmer productivity, Fractal presents an intuitive, expressive and modular API, allowing for rapid compositional expression of many GPM algorithms. Fractal-based implementations outperform both existing systemic solutions and specialized distributed solutions on many problems - from frequent graph mining to subgraph querying, over a range of datasets.

High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes ( i ) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; ( ii ) Lemonade, a data mining and analysis tool; and ( iii ) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

IoT devices are often a vector for assembling massive botnets, as a consequence of being broadly available, having limited security protections, and significant challenges in deploying software upgrades. Such botnets are usually controlled by centralized Command-and-Control (C&C) servers, which need to be identified and taken down to mitigate threats. In this paper we propose a framework to infer C&C server IP addresses using four heuristics. Our heuristics employ static and dynamic analysis to automatically extract information from malware binaries. We use active measurements to validate inferences, and demonstrate the efficacy of our framework by identifying and characterizing C&C servers for 62% of 1050 malware binaries collected using 47 honeypots.

Despite advances in prevention and mitigation mechanisms, phishing remains a threat. One reason for this is that phishers continuously improve their techniques. In this paper we study and characterize one of these improvements: phishers' use of redirection chains to evade identification mechanisms and avoid takedown of the infrastructure hosting the malicious content. We propose a method to group messages and URLs into phishing campaigns, and develop a framework to identify their hosting infrastructure. We apply our method and framework on a dataset of spam and phishing messages collected from lowinteractivity honeypots. We explore and characterize phishing campaigns as well as their hosting infrastructure. Our results indicate that phishing campaigns are usually hosted in cloud providers, but some are hosted on devices in access networks, possibly infected end-user devices. This indicates multiple approaches used by phishers, and motivates different fronts to combat this threat. Apesar de avanços em mecanismos de prevenção e mitigação, o phishing continua uma ameaça. Uma das razões disso é que o agente responsável pelo envio dessas mensagens, o phisher, aprimora suas técnicas continuamente. Neste trabalho estudamos e caracterizamos um destes aprimoramentos: o uso pelos phishers de cadeias de redirecionamentos para ludibriar mecanismos de identificação e evitar bloqueio da infraestrutura de hospedagem do conteúdo malicioso. Propomos um método para agrupamento de mensagens e URLs em campanhas de phishing, e desenvolvemos um arcabouço para identificação da infraestrutura de hospedagem. Aplicamos nosso método e arcabouço em um conjunto de mensagens de spam e phishing coletado por honeypots de baixa interatividade. Caracterizamos campanhas de phishing e a infraestrutura de hospedagem. Nossos resultados mostram que a infraestrutura de hospedagem das campanhas identificadas se concentra em provedores de computação em núvem, mas que algumas campanhas são hospedadas por dispositivos em provedores de rede, possivelmente em dispositivos infectados. Isso indica diferentes abordagens de phishers, sugerindo esforços em diferentes frentes de combate.

Analysis of public transportation data in large cities is a challenging problem. Managing data ingestion, data storage, data quality enhancement, modelling and analysis requires intensive computing and a non-trivial amount of resources. In EUBra-BIGSEA (Europe–Brazil Collaboration of Big Data Scientific Research Through Cloud-Centric Applications) we address such problems in a comprehensive and integrated way. EUBra-BIGSEA provides a platform for building up data analytic workflows on top of elastic cloud services without requiring skills related to either programming or cloud services. The approach combines cloud orchestration, Quality of Service and automatic parallelisation on a platform that includes a toolbox for implementing privacy guarantees and data quality enhancement as well as advanced services for sentiment analysis, traffic jam estimation and trip recommendation based on estimated crowdedness. All developments are available under Open Source licenses (http://github.org/eubr-bigsea, https://hub.docker.com/u/eubrabigsea/).