Following Tran et al. (2005), we evaluate PosoMAS for different process criteria, the coverage of different steps and of multi-agent system concepts. A comparison with the processes O-MasE, Prometheus, and ASPECS is provided as well. The data for Prometheus is taken from Tran et al. (2005) and Padgham and Winikoff (2005), the evaluation for O-MasE is based mainly on DeLoach and Garcia-Ojeda (2010), and the one of ASPECS mainly on Cossentino et al. (2010).
The evaluation of the different process criteria for PosoMAS and the comparison with the reference processes is depicted in Table 1. Details of the evaluation for PosoMAS and ASPECS are given in Table 2. It is important to note that the tables only capture if a process has explicit supportive content for a certain criterion. It is, e.g., very much possible to build proactive multi-agent systems with PosoMAS even though the process does not include specific support for them.
While O-MasE does not explicitly prescribe the BDI agent paradigm, the elements of the used meta-model strongly suggest that the method engineers who created it had current BDI implementations in mind. The OMACS meta-model used defines elements for goals and plans. Desires and Intentions are handled exactly with these concepts in procedural reasoning engines such as Jadex. Indeed, goals exist on two levels in O-MaSE: on the organisational level they direct the creation of an organisational structure to achieve the goals, on the agent level, they define individual goals. The latter goals are achieved by plans (as in BDI), the former goals by assigning roles.
The methodology has a philosophy similar to PosoMAS: method content is defined independently of a process life-cycle. Therefore, some of the aspects—e.g., the development life-cycle—depend on the base methodology that is combined with O-MaSE's method content. PosoMAS, however, is also a process. Therefore, the "development life-cycle" criterion in Table 1 is evaluated differently. Unfortunately, the O-MaSE method fragments are not available in a repository, although the content is distributed with the download of agentTool, the program also used for the creation of the required models.
Prometheus is the oldest process in the list, but is still one of the best described ones and is still frequently cited. While it does not insist on a meta-model, it prescribes the use of BDI agents. Therefore, the methodology concentrates on the definition of goals, plans, beliefs, and events. The development is mainly from the bottom up since it is very focused on individual agent behaviours and their constitutive elements. One of the bigger differences to the other processes is the fact that adaptability is not explicitly taken into account.
The relatively recent process ASPECS is probably the one most similar to PosoMAS. The authors are veterans of agent-oriented software engineering—see, e.g., PASSIM (Cossentino et al., 2008)—and in many regards, ASPECS is the culmination of their collectively learned lessons. The process is also described on a specialised website, although the content is very similar to the one from the paper. ASPECS too employs recent AOSE techniques and is aimed at complex systems. However, there are important differences to PosoMAS, especially with regard to the level of abstraction. The main concept ASPECS deals with are holons. If a system can not be described in holonic terms, the process is not applicable. In addition, ontologies are a central focus of the design of the system. They are used to capture the concepts of the problem domain and used extensively in later stages of the design as well as in the communication between the agents—the traditional use of ontologies in MAS. Cossentino et al. (2010) argue that using ontologies enhances the design and allows using AI technologies, possibly semantic-web approaches.
ASPECS also prescribes meta-models for the "agency domain", the "problem domain", and the "solution domain". The first defines concepts such as tasks, groups, roles, and communication, while the second deals with organisations, interactions, and ontologies. The solution domain includes concepts used in the implementation and can contain platform-specific choices. The three meta-models are the basis of the model-driven approach followed by ASPECS in which models are successively refined and transformed from one domain to the next. This ensures consistency of the models but requires the development of transformation rules. PosoMAS and ASPECS are compared according to the criteria from Table 1 in more detail in Table 2. Please note that Cossentino et al. (2010) includes a similar comparison with other processes based on a criteria set "inspired" by Tran et al. (2005).
Evaluation Criteria | PosoMAS | ASPECS | INGENIAS | |
Process-Related Criteria | ||||
Development life-cycle | Based on OpenUP's risk-value life cycle. | Iterative across and within all phases. | Based on UDP's risk-value life cycle. | |
Coverage of the life-cycle | Explicit agent-focused support of conceptualisation with goal-based requirements analysis, design with a number of practices, and partially implementation, e.g., with model-driven observer synthesis. Support for implementation and deployment through OpenUP. | Claims to support all phases but is focused on analysis and design, specifically agent and holon design. Covers implementation with rudimentary activities. | Focused on analysis, design, and deployment with additional method content to cover parts of the implementation with automatic code generation for JADE. | |
Development perspectives | Analysis is performed in a top-down fashion while the design supports both perspectives. Top-down design is used in agent architecture design and system design, bottom-up approaches in the specification of the self-organisation algorithm. | Analysis starts with the definition of the environment and the overall system architecture. From that, tasks and goals are defined and successively refined until the level of the individual agent models is reached. | ||
Application domain | In principle any domain, although adaptivity and heterogeneity are a focus. | In principle any domain, although holonic approach could limit this. | In principle any domain, although competitive and open systems are not directly supported. | |
Size of MAS | Aimed at large-scale heterogeneous projects with many agents not under the control of the original developers. | Aimed at large-scale projects that profit from holonic perspective. | Not specified. | |
Agent paradigm | All method content is formulated in a paradigm-agnostic way. | The agency meta-model does not prescribe a concrete agent paradigm. | The focus on goals and the code generation are aimed at the BDI paradigm and its concrete instantiation in JADE. | |
Support for verification | The use of constraints within the process allows a limited form of formal analysis. | Although Cossentino et al. (2010) states that support for formal analysis is available, no information is provided either in the paper or on the website. | Formal analysis is not part of the process. | |
Refinability | Advocated by the practices Evolutionary Agent Architecture and OpenUP's Evolutionary Design. | Stepwise refinement of the models is an important part of the proces. | Embedded in the process in several activities for the different types of models. | |
Approach towards MAS development | While PosoMAS does not explicitly prescribe the approach, its practices are formulated in a way that strongly favours an object-oriented approach. | The definition of roles and of agent organisations is promoted. In addition, ontologies play an important role, so knowledge engineering applies as well. | The development of the MAS is based on tasks and goals for the entire system that are refined to agents and the roles they play. | |
Meta-model based | No. PosoMAS does not prescribe any meta-models. | ASPECS defines meta-models for the "agency domain", the "problem domain", and the "solution domain". | The INGENIAS meta-models cover agents, organisations, tasks and goals, environment, and interactions. | |
Model-Related Criteria | ||||
Syntax and Semantics | Syntax and semantics of the models are inherited from UML which is used as a standard modeling language. While this is a different level than using meta-models, it still ensures that the models are based on clearly defined concepts. | Enforced by the meta-models. | Enforced by the meta-models and by the model transformations. | |
Model transformations | Yes, e.g., in Model-driven Observer Synthesis. | Yes, the process explicitly promotes using model-driven engineering. | Yes, the process includes activities to define the transformation templates and perform the transformations itself. | |
Consistency | Ensured by evolutionary refinement, model transformations, and corresponding guidelines. | Ensured by model transformations and strong adherence to meta-models. | Ensured by model transformations and strong adherence to meta-models. | |
Modularity | Modular agents can be composed from individual components or with other object-oriented techniques. | Modularity based on roles, ontologies, and object-oriented techniques. | Modularity based on tasks, goals, and roles. | |
Abstraction | Yes, models are possible on different levels of abstraction. | |||
Autonomy | Yes, by the definition of observation and control models. | Yes, based on individual and collective agent goals. | Yes, based on agent and system goals, tasks, and roles. | |
Adaptability | Yes, by the observation and control models, the self-organisation algorithm, and the trust-based interactions. | Yes, by adapting the holonic structure and with autonomous agent decisions. | Yes, by autonomous agent decisions. | |
Cooperative behaviour | Yes, especially with regard to incorporating untrustworthy agents. | Yes, within holons and between holons. | Yes, by sharing common goals. | |
Inferential capability | Not included. | Not included. | Not included. | |
Communication ability | Yes, based on FIPA protocols and Trust-based Interaction Design. | Yes, based on FIPA protocols and supported by the ontology. | Yes, based on the interactions meta-model. | |
Personality | Not included. | Not included. | Not included. | |
Reactivity | Yes, by the definition of observation and control models. | Yes, with events. | Not included. | |
Proactivity | No, there is no specialised method content to account for proactivity. | Yes, based on individual and collective agent goals. | Yes, based on agent and system goals. | |
Temporal continuity | Not included. | Not included. | Not included. | |
Concurrency | Not included in the processes, but can be modelled with standard UML constructs such as parallel regions in state charts. | Not included. | ||
Model Reuse | Yes, models are reused for different purposes within both processes. | |||
Supportive Feature Criteria | ||||
Software and methodological support | Yes, standard CASE tools can be used. Guidances include many reusable assets. | Use of the INGENIAS Development Kit is enforced. | ||
Open systems and scalability | Yes, through Trust-based Interaction Design, Agent Organisation Design, and support material. | Yes, through the use of holons and ontological support. | Not explicitly supported. | |
Dynamic structure | Yes, through Agent Organisation Design. | Yes, through the use of holons. | Not included. | |
Performance and robustness | Yes, through Trust-based Interaction Design, Agent Organisation Design, and the observation and control models. | Yes, scalability through the use of holons. | Not explicitly supported. | |
Support for conventional objects | Yes, with the use of standard UML. | No. | ||
Support for mobile agents | Not included. | Not included. | Not included. | |
Support for self-interested agents | Yes, through Trust-based Interaction Design | Yes, through individual goals and autonomous decisions about participation in holons. | Yes, through individual goals. | |
Support for ontologies | Not included. | Yes, extensively. | Not included. |
The technique-related criteria shown in Evaluation and Validation Criteria are the basis for a metric to evaluate how well the steps are covered by the different processes. The list below explains the different levels of coverage used in Table 3.
None of the processes used in this comparison covers all the process steps. As described in Evaluation and Validation Criteria, this is not strictly necessary although the steps capture many important activities within an AOSE process.
Steps | PosoMAS | O-MaSE | Prometheus | ASPECS | Gaia | Tropos | INGENIAS |
Problem Domain Analysis | |||||||
1. Identify system goals | Full | Full | None | Full | None | Full | Full |
2. Identify system tasks/behaviour | Full | Full | Full | Full | Full | Full | Full |
3. Specify use case scenarios | None | Full | Full | Full | None | None | Full |
4. Identify roles | None | Full | None | Full | Full | None | Full |
5. Identify agent classes | Full | Full | Full | Full | Full | Full | Full |
6. Model domain conceptualisation | Full | Partial | Partial | Partial1 | None | Exemplified | Limited2 |
Agent Interaction Design | |||||||
7. Specify acquaintances between agent classes | Limited | Full | Full | Full | Full | Full | Full |
8. Define interaction protocols | Full | Full | Full | Full | Full | Full | Full |
9. Define content of exchanged messages | Full | Full | Included | Full | None | Full | Full |
Agent Internal Design | |||||||
10. Specify agent architecture | Full | Full | None | Full | None | None | Limited3 |
11. Define agent mental attitudes (e.g., goals, beliefs, plans, commitments, ...) | Partial | None | Full | Partial | None | Full | Full |
12. Define agent behavioural interface (e.g., capabilities, services, contracts, ...) | Included | None | Full | Partial | Full | None | Partial |
System/Environment Design | |||||||
13. Specify system architecture (i.e., overview of all components and their connections) | Described | None | None | Partial | None | None | Full |
14. Specify organisational structure/control regime/inter-agent social relationships | Full | Included | None | Full | Full | Included | Partial |
15. Model MAS environment (e.g., resources, facilities, characteristics) | Described | None | Described | Partial | Partial | Partial | Full |
16. Specify agent-environment interaction mechanism | Full | None | Full | Partial | None | None | Full |
17. Specify agent inheritance and aggregation | Included | None | None | Included | Full | None | None |
Deployment | |||||||
18. Instantiate agent classes | None | Full | None | Full | Included | None | Full |
19. Specify agent instances deployment | Included | Full | None | Full | None | Included | Full |
1: ASPECS describes the domain and conceptualises some of its aspects in the problem domain ontology but does not make them available as software engineering concepts.
2: INGENIAS offers limited domain modelling in the use cases ("Problem definition" artefact) and in obtaining the environment model but does not offer explicit notation or activities for the purpose.
3: Since INGENIAS prescribes the multi-agent framework that must be used and is limited to the BDI paradigm, variations in the agent architecture are severely limited
Apart from the validation criteria and the process steps, Tran et al. (2005) also provide a list of multi-agent systems concepts and how well the respective processes cover these concepts. The coverage of concepts provided by PosoMAS and the comparison to O-MasE and Prometheus are shown in Table 4.
Concepts | PosoMAS | O-MaSE | Prometheus | ASPECS | Gaia | Tropos | INGENIAS |
Problem Domain | |||||||
System goals | ✔ | ✔ | ✔ | ✔ | ✔ | ||
System roles | ✔ | ✔ | ✔ | ||||
System functionality/Tasks | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
Task responsibilities/Procedures | ✔ | ✔ | ✔ | ✔ | |||
Design requirements | ✔ | ✔ | ✔ | ||||
Use case/Scenarios | ✔ | ✔ | ✔ | ✔ | |||
Agent Properties | |||||||
Agent classes | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
Agent instances (including cardinality) | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Agent's roles | ✔ | ✔ | ✔ | ✔ | |||
Agent's functionality | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Agent's knowledge/beliefs | ✔ | ✔ | ✔ | ||||
Agent's plans | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Agent's goals | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Agent's capabilities | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
Agent mobility | |||||||
Agent Interaction | |||||||
Interaction pathways | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
Exchanged messages | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | |
Interaction protocols | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Interaction constraints | ✔ | ||||||
Conflict resolution mechanisms | |||||||
Contracts/commitments | ✔ | ||||||
Ontology | ✔ | ||||||
Agent Relationships | |||||||
Inheritance | ✔ | ✔ | |||||
Aggregation | ✔ | ✔ | ✔ | ✔ | |||
Association | ✔ | ✔ | ✔ | ||||
System/Environment | |||||||
Co-existing non-agent entities | ✔ | ✔ | ✔ | ✔ | |||
Infrastructure/environment facilities | ✔ | ✔ | ✔ | ||||
Organisational Structure | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Agent-environment interaction | ✔ | ✔ | ✔ | ✔ | |||
Environment characteristics | ✔ | ✔ | ✔ | ||||
Deployment | |||||||
Agent architecture | ✔ | ✔ | ✔ | ||||
System architecture | ✔ | ✔ | |||||
Location of agent instances | ✔ | ✔ | ✔ |
This material is made available under the Creative Commons—Attribution-ShareAlike License v3.0.
© Copyright 2013, 2014 by Institute for Software & Systems Engineering, University of Augsburg
Contributors
Contact: Jan-Philipp Steghöfer