Real-time strategy (RTS) games have provided a fertile ground for AI research with notable recent successes based on deep reinforcement learning (RL). However, RL remains a data-hungry approach featuring a high sample complexity. In this paper, we focus on a sample complexity reduction technique called reinforcement learning as a rehearsal (RLaR) and on the RTS game of MicroRTS to formulate and evaluate it. RLaR has been formulated in the context of action-value function based RL before. Here, we formulate it for a different RL framework, called actor-critic RL. We show that on the one hand the actor-critic framework allows RLaR to be much simpler, but on the other hand, it leaves room for a key component of RLaR–a prediction function that relates a learner’s observations with that of its opponent. This function, when leveraged for exploration, accelerates RL as our experiments in MicroRTS show. Further experiments provide evidence that RLaR may reduce actor noise compared to a variant that does not utilize RLaR’s exploration. This study provides the first evaluation of RLaR’s efficacy in a domain with a large strategy space.

High utility itemsets mining (HUIM) is an important sub-field of frequent itemset mining (FIM). Recently, HUIM has received much attention in the field of data mining. High utility itemsets (HUIs) have proven to be quite useful in marketing, retail marketing, cross-marketing, and e-commerce. Traditional HUIM approaches suffer from a drawback as they need a user-defined minimum utility ($ min\_util $) threshold. It is not easy for the users to set the appropriate $ min\_util $ threshold to find actionable HUIs. To target this drawback, top-k HUIM has been introduced. top-k HUIM is more suitable for supermarket managers and retailers to prepare appropriate strategies to generate higher profit. In this paper, we provide an in-depth survey of the current status of top-k HUIM approaches. The paper presents the task of top-k HUIM and its relevant definitions. It reviews the top-k HUIM approaches and presents their advantages and disadvantages. The paper also discusses the important strategies of the top-k HUIM, their variations, and research opportunities. The paper provides a detailed summary, analysis, and future directions of the top-k HUIM field.

In the previous two decades, knowledge graphs (KGs) have evolved significantly, inspiring developers to build ever-more context-related KGs. Due to this development, artificial intelligence (AI) applications can now access open domain-specific information in a format that is both semantically rich and machine comprehensible. In this article, a framework that depicts functional design for indoor workspaces and urban adaptive design, in order to help architects, artists, and interior designers for the design and construction of an urban or indoor workspace, based on the emotions of human individuals, is introduced. For the creation of online adaptive environments, the framework may incorporate emotional, physiological, visual, and textual measures. Additionally, an information retrieval mechanism that extracts critical information from the framework in order to assist the architects, artists, and the interior designers is presented. The framework provides access to commonsense knowledge about the (re-)design of an urban area and an indoor workspace, by suggesting objects that need to be placed, and other modifications that can be applied to the location, in order to achieve positive emotions. The emotions referred reflect to the emotions experienced by an individual when being in the indoor or urban area, which are pointers for the functionality, the memorability, and the admiration of the location. The framework also performs semantic matching between entities from the web KG ConceptNet, using semantic knowledge from ConceptNet and WordNet, with the ones existing in the KG of the framework. The paper provides a set of predefined SPARQL templates that specifically handle the ontology upon which the knowledge retrieval system is based. The framework has an additional argumentation function that allows users to challenge the knowledge retrieval component findings. In the event that the user prevails in the reasoning, the framework will learn new knowledge.

We propose a logic-based framework to model a system whose aim is to help provide the user with those pieces of information that are useful with respect to his/her current information need, as well as relevant to his/her query. More precisely, we propose three measures of information usefulness which take into account the fact that the user can be represented as a cognitive agent endowed with some beliefs—a partial “picture” about what it already knows—and goals—a certain state of affairs in which the agent would like to be. This paper extends a previous version of the framework by considering a more realistic hypothesis, according to which there are several ways to achieve goals. We present three different approaches: the binary approach, the ordinal approach, and the numerical approach. We take information retrieval (IR) as a particular application domain, and we compare some existing measures with the usefulness measure we introduce here.

Emerging reinforcement learning algorithms that utilize human traits as part of their conceptual architecture have been demonstrated to encourage cooperation in social dilemmas when compared to their unaltered origins. In particular, the addition of a mood mechanism facilitates more cooperative behaviour in multi-agent iterated prisoner dilemma (IPD) games, for both static and dynamic network contexts. Mood-altered agents also exhibit humanlike behavioural trends when environmental aspects of the dilemma are altered, such as the structure of the payoff matrix used. It is possible that other environmental effects from both human and agent-based research will interact with moody structures in previously unstudied ways. As the literature on these interactions is currently small, we seek to expand on previous research by introducing two more environmental dimensions; voluntary interaction in dynamic networks, and stability of interaction through varied network restructuring. From an initial Erdos–Renyi random network, we manipulate the structure of a network IPD according to existing methodology in human-based research, to investigate possible replication of their findings. We also facilitated strategic selection of opponents through the introduction of two partner evaluation mechanisms and tested two selection thresholds for each. We found that even minimally strategic play termination in dynamic networks is enough to enhance cooperation above a static level, though the thresholds for these strategic decisions are critical to desired outcomes. More forgiving thresholds lead to better maintenance of cooperation between kinder strategies than stricter ones, despite overall cooperation levels being relatively low. Additionally, moody reinforcement learning combined with certain play termination decision strategies can mimic trends in human cooperation affected by structural changes to the IPD played on dynamic networks—as can kind and simplistic strategies such as Tit-For-Tat. Implications of this in comparison with human data is discussed, and suggestions for diversification of further testing are made.

In the previous two decades, Knowledge Graphs (KGs) have evolved, inspiring developers to build ever-more context-related KGs. Because of this development, Artificial Intelligence (AI) applications can now access open domain-specific information in a format that is both semantically rich and machine comprehensible. In this article, we introduce the XR4DRAMA framework. The KG of the XR4DRAMA framework can represent data for media preparation and disaster management. More specifically, the KG of the XR4DRAMA framework can represent information about: (a) Observations and Events (e.g., data collection of biometric sensors, information in photos and text messages), (b) Spatio-temporal (e.g., highlighted locations and timestamps), (c) Mitigation and response plans in crisis (e.g., first responder teams). In addition, we provide a mechanism that allows Points of Interest (POI) to be created or updated based on videos, photos, and text messages sent by users. For improved disaster management and media coverage of a location, POI serve as markers to journalists and first responders. A task creation mechanism is also provided for the disaster management scenario with the XR4DRAMA framework, which indicates to first responders and citizens what tasks need to be performed in case of an emergency. Finally, the XR4DRAMA framework has a danger zone creation mechanism. Danger zones are regions in a map that are considered as dangerous for citizens and first responders during a disaster management scenario and are annotated by a severity score. The last two mechanisms are based on a Decision Support System (DSS).

Artificial barriers in Learning Automata (LA) is a powerful and yet under-explored concept although it was first proposed in the 1980s. Introducing artificial non-absorbing barriers makes the LA schemes resilient to being trapped in absorbing barriers, a phenomenon which is often referred to as lock in probability leading to an exclusive choice of one action after convergence. Within the field of LA and reinforcement learning in general, there is a sacristy of theoretical works and applications of schemes with artificial barriers. In this paper, we devise a LA with artificial barriers for solving a general form of stochastic bimatrix game. Classical LA systems possess properties of absorbing barriers and they are a powerful tool in game theory and were shown to converge to game’s of Nash equilibrium under limited information. However, the stream of works in LA for solving game theoretical problems can merely solve the case where the Saddle Point of the game exists in a pure strategy and fail to reach mixed Nash equilibrium when no Saddle Point exists for a pure strategy.

Furthermore, we provide experimental results that are in line with our theoretical findings.

Automated planning is a prominent area of Artificial Intelligence and an important component for intelligent autonomous agents. A cornerstone of domain-independent planning is the separation between planning logic, that is the automated reasoning side, and the knowledge model, that encodes a formal representation of domain knowledge needed to reason upon a given problem to synthesize a solution plan. Such a separation enables the use of reformulation techniques, which transform how a model is represented in order to improve the efficiency of plan generation. Over the past decades, significant research effort has been devoted to the design of reformulation techniques. In this paper, we present a systematic review of the large body of work on reformulation techniques for classical planning, aiming to provide a holistic view of the field and to foster future research in the area. As a tangible outcome, we provide a qualitative comparison of the existing classes of techniques, that can help researchers gain an overview of their strengths and weaknesses.

Efficient knowledge extraction from Big Data is quite a challenging topic. Recognizing relevant concepts from unannotated data while considering both context and domain knowledge is critical to implementing successful knowledge extraction. In this research, we provide a novel platform we call Active Learning Integrated with Knowledge Extraction (ALIKE) that overcomes the challenges of context awareness and concept extraction, which have impeded knowledge extraction in Big Data. We propose a method to extract related concepts from unorganized data with different contexts using multiple agents, synergy, reinforcement learning, and active learning.

We test ALIKE on the datasets of the COVID-19 Open Research Dataset Challenge. The experiment result suggests that the ALIKE platform can more efficiently distinguish inherent concepts from different papers than a non-agent-based method (without active learning) and that our proposed approach has a better chance to address the challenges of knowledge extraction with heterogeneous datasets. Moreover, the techniques used in ALIKE are transferable across any domain with multidisciplinary activity.

This paper presents the technical specifications of a lightweight humanoid robot platform named Robinion Sr. including its mechanical and electrical design. We describe a versatile and robust mechatronic system, efficient walking gait, and software architecture of the humanoid robot. The humanoid robot platform is targeted for use in a range of applications, including research and development, competitions, and the service industry. A reduced platform cost was an essential consideration in our design. We introduce a specialized and inexpensive mechanical design, which includes a parallel-kinematics leg design, external gears, and low-cost controllers and sensors. The humanoid robot is equipped with an efficient electronic structure and a tablet computer for task scheduling, control, and perception, as well as an embedded controller for solving forward & inverse kinematics and low-level actuator control. The perception system recognizes objects at real-time inference with Deep Learning-based detection algorithms without a dedicated GPU. We present and evaluate the capabilities of our newly developed advanced humanoid robot and believe it is a suitable platform for the academic and industrial robotics community.

In this paper, we present a model of the spread of the COVID-19 pandemic simulated by a multi-agent system (MAS) based on demographic data and medical knowledge. Demographic data are linked to the distribution of the population according to age and to an index of socioeconomic fragility with regard to the elderly. Medical knowledge are related to two risk factors: age and obesity. The contributions of this approach are as follows. Firstly, the two aggravating risk factors are introduced into the MAS using fuzzy sets. Secondly, the worsening of disease caused by these risk factors is modeled by fuzzy aggregation operators. The appearance of virus variants is also introduced into the simulation through a simplified modeling of their contagiousness. Using real data from inhabitants of an island in the Antilles (Guadeloupe, FWI), we model the rate of the population at risk which could be critical cases, if neither social distancing nor barrier gestures are respected by the entire population. The results show that hospital capacities are exceeded. The results show that hospital capacities are exceeded. The socioeconomic fragility index is used to assess mortality and also shows that the number of deaths can be significant.

Reinforcement Learning (RL) methods often rely on gradient estimates to learn an optimal policy for control problems. These expensive computations result in long training times, a poor rate of convergence, and sample inefficiency when applied to real-world problems with a large state and action space. Evolutionary Computation (EC)-based techniques offer a gradient-free apparatus to train a deep neural network for RL problems. In this work, we leverage the benefits of EC and propose a novel variant of genetic algorithm called SP-GA which utilizes a species-inspired weight initialization strategy and trains a population of deep neural networks, each estimating the Q-function for the RL problem. Efficient encoding of a neural network that utilizes less memory is also proposed which provides an intuitive mechanism to apply Gaussian mutations and single-point crossover. The results on Atari 2600 games outline comparable performance with gradient-based algorithms like Deep Q-Network (DQN), Asynchronous Advantage Actor Critic (A3C), and gradient-free algorithms like Evolution Strategy (ES) and simple Genetic Algorithm (GA) while requiring far fewer hyperparameters to train. The algorithm also improved certain Key Performance Indicators (KPIs) when applied to a Remote Electrical Tilt (RET) optimization task in the telecommunication domain.

Algorithmic bias arises in machine learning when models that may have reasonable overall accuracy are biased in favor of ‘good’ outcomes for one side of a sensitive category, for example gender or race. The bias will manifest as an underestimation of good outcomes for the under-represented minority. In a sense, we should not be surprised that a model might be biased when it has not been ‘asked’ not to be; reasonable accuracy can be achieved by ignoring the under-represented minority. A common strategy to address this issue is to include fairness as a component in the learning objective. In this paper, we consider including fairness as an additional criterion in model training and propose a multi-objective optimization strategy using Pareto Simulated Annealing that optimizes for both accuracy and underestimation bias. Our experiments show that this strategy can identify families of models with members representing different accuracy/fairness tradeoffs. We demonstrate the effectiveness of this strategy on two synthetic and two real-world datasets.

Recently, many question answering systems that derive answers from linked data repositories have been developed. The purpose of this survey is to identify the common features and approaches of the semantic question answering (SQA) systems, although many different and prototype systems have been designed. The SQA systems use a formal query language like SPARQL and knowledge of a specific vocabulary. This paper analyses different frameworks, architectures, or systems that perform SQA and classifies SQA systems based on different criteria.

Automated planning is the field of Artificial Intelligence (AI) that focuses on identifying sequences of actions allowing to reach a goal state from a given initial state. The need of using such techniques in real-world applications has brought popular languages for expressing automated planning problems to provide direct support for continuous and discrete state variables, along with changes that can be either instantaneous or durative. PDDL+ (Planning Domain Definition Language +) models support the encoding of such representations, but the resulting planning problems are notoriously difficult for AI planners to cope with due to non-linear dependencies arising from the variables and infinite search spaces. This difficulty is exacerbated by the potentially huge fully ground representations used by modern planners in order to effectively explore the search space, which can make some problems impossible to tackle.

This paper investigates two grounding techniques for PDDL+ problems, both aimed at reducing the size of the full ground representation by reasoning over the lifted, more abstract problem structure. The first method extends the simple mechanism of invariant analysis to limit the groundings of operators upfront. The second method proposes to tackle the grounding process through a PDDL+ to classical planning abstraction; this allows us to leverage the amount of research done in the classical planning area. Our empirical analysis studies the effect of these novel approaches over both real-world hybrid applications and synthetic PDDL+ problems took from standard benchmarks of the planning community; our results reveal that not only the techniques improve the running time of previous grounding mechanisms but also let the planner extend the reach to problems that were not solvable before.

Argumentation and eXplainable Artificial Intelligence (XAI) are closely related, as in the recent years, Argumentation has been used for providing Explainability to AI. Argumentation can show step by step how an AI System reaches a decision; it can provide reasoning over uncertainty and can find solutions when conflicting information is faced. In this survey, we elaborate over the topics of Argumentation and XAI combined, by reviewing all the important methods and studies, as well as implementations that use Argumentation to provide Explainability in AI. More specifically, we show how Argumentation can enable Explainability for solving various types of problems in decision-making, justification of an opinion, and dialogues. Subsequently, we elaborate on how Argumentation can help in constructing explainable systems in various applications domains, such as in Medical Informatics, Law, the Semantic Web, Security, Robotics, and some general purpose systems. Finally, we present approaches that combine Machine Learning and Argumentation Theory, toward more interpretable predictive models.

Ontology alignment is an important and inescapable problem for the interconnections of two ontologies stating the same concepts. Ontology alignment evaluation initiative (OAEI) has been taken place for more than a decade to monitor and help the progress of the field and to compare systematically existing alignment systems. As of 2018, the evaluation of systems is partly transitioned to the HOBBIT platform. This paper contains the description of our alignment system, simulated annealing-based ontology matching (SANOM), and its adaption into the HOBBIT platform. The outcomes of SANOM on the HOBBIT for several OAEI tracks are reported, and the results are compared with other competing systems in the corresponding tracks.

The aim of this critical review paper is threefold: (a) to provide an insight on the impact of ontology engineering methodologies (OEMs) to the evolution of living and reused ontologies, (b) to update the ontology engineering (OE) community on the status and trends in OEMs and of their use in practice and (c) to propose a set of recommendations for working ontologists to consider during the life cycle of living, evolved and reused ontologies. The work outlined in this critical review paper has been motivated by the need to address critical issues on keeping ontologies alive and evolving while these are shared in wide communities. It is argued that the engineering of ontologies must follow a well-defined methodology, addressing practical aspects that would allow (sometimes wide) communities of experts and ontologists to reach consensus on developments and keep the evolution of ontologies ‘in track’. In doing so, specific collaborative and iterative tool-supported tasks and phases within a complete and evaluated ontology life cycle are necessary. This way the engineered ontologies can be considered ‘shared, commonly agreed and continuously evolved “live” conceptualizations’ of domains of discourse. Today, in the era of Linked Data and Knowledge Graphs, it is more necessary than ever not to neglect to consider the recommendations that OEMs explicitly and implicitly introduce and their implications to the evolution of living ontologies. This paper reports on the status of OEMs, identifies trends and provides recommendations based on the findings of an analysis that concerns the impact of OEMs to the status of well-known, widely used and representative ontologies.

The current fourth industrial revolution, or ‘Industry 4.0’ (I4.0), is driven by digital data, connectivity, and cyber systems, and it has the potential to create impressive/new business opportunities. With the arrival of I4.0, the scenario of various intelligent systems interacting reliably and securely with each other becomes a reality which technical systems need to address. One major aspect of I4.0 is to adopt a coherent approach for the semantic communication in between multiple intelligent systems, which include human and artificial (software or hardware) agents. For this purpose, ontologies can provide the solution by formalizing the smart manufacturing knowledge in an interoperable way. Hence, this paper presents the few existing ontologies for I4.0, along with the current state of the standardization effort in the factory 4.0 domain and examples of real-world scenarios for I4.0.

User validation is one of the challenges facing the ontology alignment community, as there are limits to the quality of the alignments produced by automated alignment algorithms. In this paper, we present a broad study on user validation of ontology alignments that encompasses three distinct but inter-related aspects: the profile of the user, the services of the alignment system, and its user interface. We discuss key issues pertaining to the alignment validation process under each of these aspects and provide an overview of how current systems address them. Finally, we use experiments from the Interactive Matching track of the Ontology Alignment Evaluation Initiative 2015–2018 to assess the impact of errors in alignment validation, and how systems cope with them as function of their services.