EDR Processing (Extended Data Record)Record processing is a common computing problem that enterprises have to deal with. Telco companies, for instance, take the information form the calls their customers make, process the data by following some specific rules, and then draw useful conclusions and thus obtain valuable business knowledge. The transformation usually takes a lot of time and requires a considerable amount of computing resources.
- Objective: Because real-world EDR processing is time constrained and critical to the company’s objectives, the objective in this use case is to provide a high performance computing solution, based on a Grid Component version EDR program, thus improving the quality of the processing by offering redundancy, fault-tolerance, scalability, load balancing, and reduced computing times.
- Architectural design: Using GridCOMP, we distribute the transformation effort, so EDR processing can be computed in parallel among the existing computational resources. The application receives a sequential file containing the Extended Data Records as fixed length fields. The input file is split into chunks and distributed to the nodes on the grid. Each node processes its corresponding chunk, generating a partial result file that is transferred back to the application. In the end, the partial results are joined into the final result file.
 The GridCOMP Solution offers to the EDR application:
- A grid-computing component-based model that is fully portable because it is solely based on Java
- The Composition of components and the ability to follow a top-down design of the application and an easy to reuse code - Collective interfaces the hide the complexity of Grid computing - An autonomic management component that provides fault tolerance and load balancing
- The GridCOMP computation results in:
- Reduced processing time of more complex processes in less time - An improved quality of service due to the edundancy and fault tolerance - Savings on the IT infrastructure as it use existing hardware and low-profile machines - The ability to scale up easily by just adding more low-profile machines and without changing the application.
Wing design application In the aerospace sector, the software that computes the aerodynamic wing performance for a given configuration is used to test different configurations of the wing features and to eventually find an acceptable design. Merak is an application that permits to analyze parameters variation in incompressible turbulent flow around a triple element airfoil, to evaluate the stalling angle of different wing geometries. Turbulence is simulated using different k-eps models, including law-of-the-wall and two-layer low Re. Finally, the application can extract the desired information and create the graph using gnuplot.   Objective Merak, manages small amounts of information, but needs lots of computing power. Our objective is to use GridCOMP solution to wrap and grid-enable this existing legacy code and also to prove the integration of data staging for the input files and output files into this sweeping / optimization process. Proving both of these objectives is crucial for the adoption of the GCM by the industry.
- Architectural design Using GridCOMP, the Merak application receives several parameters (a range of incidence angles and Reynolds numbers and set of .geo files containing the geometry of a three-element airfoil). The Merak legacy code executable files are provisioned among the nodes; then, the application generates and distributes the combinations of incidence angles, Reynolds numbers and geometries to process. Each node runs merak using the given parameters, and transfers the results back. The application extracts the desired information from the result files and creates the graph comparing the lift coefficient of the different geometries.
GridCOMP offers to this application: - Legacy code wrapping support
- Composition of components to combine new and pre-existing components
- Collective interfaces that abstract and hide the complexity of distributed computing
- Autonomic component management that provides fault tolerance and load balancing
- Tools to design the architecture of the grid application (GIDE)
- The GridCOMP computation results in:
- Parallel computation to achieve useful response times (minutes)
- Easy deployment of legacy code.
- Enabling the re-usability of legacy code in modern, component-based, grid applications.
2-IBM Use CaseBiometric IdentificationIn recent years biometric methods for verification and identification of people have become very popular. Applications span from governmental projects like border control or criminal identification to civil purposes such as e-commerce, network access, or transport. Frequently, biometric verification is used to authenticate people meaning that a 1:1 match operation of a claimed identity to the one stored in a reference system is carried out. In an identification system, however, the complexity is much higher. Here, a person’s identity is to be determined solely on biometric information, which requires matching the live scan of biometrics against all enrolled (known) identities. Such a 1:N match operation can be quite time-consuming making it unsuitable for real-time applications.
- Objective The objective of the use case is to build a biometric identification system (BIS), based on fingerprint biometrics, which can work on a large user population of up to millions of individuals. To achieve real-time identification within a few seconds period the BIS application takes advantage of the Grid via GCM components. The goal is to simplify the programming of the distributed identification process though the use of the GCM framework while being able to deploy the resulting application on arbitrary existing, potentially heterogeneous, hardware platforms.
 Architectural design The BIS use case can be considered a business-process or workflow-driven application. Figure 1 outlines the high-level architectural design of the BIS. It is built around a workflow execution engine acting as the central control unit of the system. A number of business processes are implemented as workflow scripts running within the engine. The processes comprise functionality accessible from the demo application (e.g. identification) as well as internal system management logic required to control the distributed biometric matching. Furthermore, the BIS provides a number of adapters to the workflow engine such that the business processes can interact with external entities, namely, the database (DB) storing information about enrolled identities, and the interface to the Grid infrastructure.
The GCM adapter (c.f. Figure 1) is triggered by the workflow scripts and provides distributed biometric matching functionality via GCM components. Here, the basic approach is to have one component encapsulating the biometric matching functionality, which is then deployed on all Grid nodes in a SPMD-style setting. Then the database of enrolled identities is distributed across the nodes and this way the 1:N matching operation is executed in parallel. - The GridCOMP Solution offers to the BIS application:
- A platform independent high-level component framework for programming distributed applications.
- Advanced built-in features such as hierarchical composition, collective interfaces, virtual nodes, deployment descriptors, and support for autonomic management hiding the complexity of Grid programming.
- The GIDE, a comprehensive tool set supporting the complete software development cycle from graphical composition to component deployment and monitoring.
- The GridCOMP computation results in:
- An identification system that can work on a large user population in real-time.
- An identification system that can be easily deployed to arbitrary existing hardware and thus is cost-efficient.
- A system that is easily scalable without any software change.
- An efficient software architecture (e.g. hierarchical components, strict separation of concerns) leading to reduced development time and component reuse.
3- Atos Origin Use CaseComputing of DSO valueThe Use Case selected by Atos Origin uses PL/SQL-based source code, and the candidate application selected was the so called “Computing of DSO value”. The DSO (Days Sales Outstanding) is the mean time that clients delay to pay an invoice to Atos. This information is needed by several internal departments as much updated as possible and the process lasts about 4 hours to compute around 6.600 clients. The DSO application is based on a client/server application and there are three main elements:
- A Graphical User Interface (GUI) used to enter some input data or parameters needed for the computation. This GUI runs on the client side and connects to the Database;
- Some PL/SQL processes which are called normally by this user interface in order to access the data stored in the database and process them to compute the results. This part runs on the server side, and it is executed by the database engine;
- The Database which stores the data.
 The architecture proposed to use with the Atos Origin use case is to put the main program (using ProActive) between the user interface and the database, as the orchestrator of the whole application. In this case, the user sends a request to the main program, asking for the whole workflow to be executed. The main program will connect to the main database and read some data (tasks) to send to the remote nodes to be executed in parallel. The remote nodes will compute the information received and send the result. Them the main program will write the result in the main database finishing the process. The master database will store all data and only some part of the information will be sent to the remote node. Each node will contain a database engine to store part of the data sent by the scheduler and to start the PL/SQL code process.
The GridCOMP Solution
GridCOMP solution offers to the DSO application: - A grid-computing component-based model that is fully portable because it is solely based on Java.
- Complexity of Grid programming hided in features such as composition of components, deployment descriptors, virtual nodes, collective interfaces, autonomic management.
- An easy way to develop new applications using Grid components.
The GridCOMP computation results in: - Reduce the execution time without upgrading the infrastructure.
- Update the information more frequently and maintain or reduce infrastructure cost.
- The ability to scale up easily by just adding more low-profile machines and without changing the application.
- A system that is easily scalable without any software change
|
About GridCOMP 
