By Topic

Challenges of Large Applications in Distributed Environments, 2005. CLADE 2005. Proceedings

Date 24 July 2005

Filter Results

Displaying Results 1 - 25 of 36
  • Reverb: middleware for distributed application forensics

    Publication Year: 2005 , Page(s): 77 - 86
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (610 KB) |  | HTML iconHTML  

    We observe the recent research trend toward large-scale, component-based distributed systems that are dynamically configurable or extensible in response to changing execution environments or end-user needs. Regardless of whether these configuration changes happen automatically through predefined adaptation or self-management methods or in response to explicit user interaction, they can jeopardize the integrity of application components. Moreover, they can cause unexpected effects in system performance or even lead to disputes about middleware or application providers' responsibilities for failures experienced by end users. This paper introduces Reverb, a set of middleware abstractions and mechanisms that can be used to: (1) audit configuration actions; (2) impose controls on permissible actions; and (3) control which principals are permitted to carry out configurations. To evaluate Reverb, it has been integrated into middleware used in the high performance domain. The intent of this integration is to not only demonstrate its viability and utility, but also to show that Reverb-based configuration control has little effect on the performance of the distributed applications or middleware that use it. Experimental results attained with Reverb-enabled middleware used with resource-constrained pervasive applications demonstrate the small performance impact of Reverb's rich new functionality. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Challenges in large scale distributed computing: bioinformatics

    Publication Year: 2005 , Page(s): 57 - 65
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (462 KB) |  | HTML iconHTML  

    The amount of genomic data available for study is increasing at a rate similar to that of Moore's law. This deluge of data is challenging bioinformaticians to develop newer, faster and better algorithms for analysis and examination of this data. The growing availability of large scale computing grids coupled with high-performance networking is challenging computer scientists to develop better, faster methods of exploiting parallelism in these biological computations and deploying them across computing grids. In this paper, we describe two computations that are required to be run frequently and which require large amounts of computing resource to complete in a reasonable time. The data for these computations are very large and the sequential computational time can exceed thousands of hours. We show the importance and relevance of these computations, the nature of the data and parallelism and we show how we are meeting the challenge of efficiently distributing and managing these computations in the SEED project. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Coordination of data movement with computation scheduling on a cluster

    Publication Year: 2005 , Page(s): 25 - 34
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (489 KB) |  | HTML iconHTML  

    We are looking at the problem of scheduling compute tasks on a cluster of servers. These tasks require files that reside on a remote archive, and may also be cached on some subset of the servers. A task can only be run on a server that has the files it requires. This introduces the problem of scheduling data movement in coordination with the scheduling of computation. Our goal is to maximize throughput while minimizing data movement. FIFO scheduling is not efficient in this situation due to its lack of awareness of the data movement required. We looked at two other strategies, called shortest job first and linear programming based optimization, and compared them under various configurations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A composable data management architecture for scientific applications

    Publication Year: 2005 , Page(s): 35 - 44
    Cited by:  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (536 KB) |  | HTML iconHTML  

    With ever increasing computational power, data management has become crucial for scientific applications today. Most on-going research efforts are dedicated to generalizing universal requirements and schemas for building generic data management systems. By closely studying a broad range of scientific applications including X-ray crystallography, radiation therapy, automated photometry and comparative genomics, a composable data management architecture for scientific applications is presented, which instead aims at providing a set of orthogonal components for each scientific application to quickly and easily construct its customized data management system. The prototype architecture is described in detail, and the component interfaces are defined in SIDL (Scientific Interface Definition Language). Results of building customized data management systems for a variety of scientific applications are also discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Structure, sharing and preservation of scientific experiment data

    Publication Year: 2005 , Page(s): 107 - 114
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (474 KB) |  | HTML iconHTML  

    In mesoscale meteorology, the quantity of information has increased significantly due to sophisticated data distribution schemes combined with developments in sensors and instruments capable of monitoring the lower several kilometers of the atmosphere at higher levels of resolution. This paper introduces myLEAD, a personalized information management tool for geoscience users. MyLEAD eases the data and information overload on the scientist by providing explicit solutions to the problems of structure, sharing, and preservation. This paper describes strategies within the myLEAD system to personalize data product and representation which ultimately leads to personalized workspaces and collaborative environments. We also include experimental results from some of the experiments that we conducted. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simplifying FusionGrid security

    Publication Year: 2005 , Page(s): 95 - 103
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (499 KB) |  | HTML iconHTML  

    Since inception in 2001, FusionGrid developers have worked to secure computational resources in a multi-institutional environment with geographically dispersed users. Recent improvements to grid security have streamlined the usage and administration of resources. More than simply increasing security, these improvements have made FusionGrid security easier for resource administrators and the fusion scientists that use FusionGrid, allowing them to get work done with minimal inconvenience. Improvements in authentication, authorization, and data handling have been welcomed by fusion scientists and promise to ease the burden of adding new resources to the grid. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A user-transparent recoverable file system for distributed computing environment

    Publication Year: 2005 , Page(s): 45 - 53
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB) |  | HTML iconHTML  

    In a distributed computing environment, particularly grid, fault-tolerance is one of the core functionalities the system should provide. MPICH-GF is such a resilient system designed to resist external or internal failures, especially for message passing applications in the grid environment. But it does not stand the loss of a valuable resource: files. In a normal case, users open files and write data into them in an asynchronous manner, and checkpointing is initiated with no regard to the state of the context of the process. Therefore, the checkpointing system should automatically recognize the running process and protect the open files transparently. We have implemented a recoverable file system, named ReFS, which is incorporated into our fault-tolerant system MPICH-GF. ReFS is a versioning-like file system. ReFS provides middleware libraries with the system call interface to protect specific files at a given time. This prevents applications from processing their jobs with corrupted data and resulting in incorrect results in case of failures. We have focused not only on the reliability of the system but also on the reduction of inevitable overheads. This paper describes the design and implementation of ReFS and justifies the validity of the behavior of ReFS. We have developed ReFS on Linux, based on Ext2. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • GISolve: a grid-based problem solving environment for computationally intensive geographic information analysis

    Publication Year: 2005 , Page(s): 3 - 12
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (547 KB) |  | HTML iconHTML  

    The purpose of this paper is to demonstrate the design and implementation of GISolve - a grid-based problem solving environment for computationally intensive geographic information analysis based on geo-middleware. The geo-middleware resides between existing grid middleware and geographic information analysis applications to manage heterogeneous and dynamic resources on behalf of analysis applications. At the same time, GISolve provides adaptive domain decomposition solutions to parallel geographic information analysis applications. Based on these domain decomposition solutions, GISolve also schedules distributed tasks and manages data transfers. In GISolve, these capabilities are designed as grid services that are compliant with the open grid service architecture (OGSA) and are implemented using grid portal technologies. The GISolve implementation is illustrated based on a case study of a computationally intensive spatial statistic - [G*i(d)] that is used to assess spatial dependence among geographically distributional observations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Collaborative e-science architecture for Reaction Kinetics research community

    Publication Year: 2005 , Page(s): 13 - 22
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (547 KB) |  | HTML iconHTML  

    This paper presents a novel collaborative e-science architecture (CeSA) to address two challenging issues in e-science that arise from the management of heterogeneous distributed environments: (i) how to provide individual scientists an integrated environment to collaborate with each other in distributed, loosely coupled research communities where each member might be using a disparate range of tools; and (ii) how to provide easy access to a range of computationally intensive resources from a desktop. The Reaction Kinetics research community was used to capture the requirements and in the evaluation of the proposed architecture. The result demonstrated the feasibility of the approach and the potential benefits of the CeSA. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simultaneous scheduling of replication and computation for bioinformatics applications on the grid

    Publication Year: 2005 , Page(s): 66 - 74
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (455 KB) |  | HTML iconHTML  

    One of the first motivations of using grids comes from applications managing large data sets infield such as high energy physics or life sciences. To improve the global throughput of software environments, replicas are usually put at wisely selected sites. Moreover, computation requests have to be scheduled among the available resources. To get the best performance, scheduling and data replication have to be tightly coupled. However, there are few approaches that provide this coupling. This paper presents an algorithm that combines data management and scheduling using a steady-state approach. Our theoretical results are validated using simulation and logs from a large life science application (ACI GRID GriPPS). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • FSML: Fusion Simulation Markup Language for interoperability of data and analysis tools

    Publication Year: 2005 , Page(s): 115 - 121
    Cited by:  Patents (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (502 KB) |  | HTML iconHTML  

    As the fusion community becomes more interconnected and problems become more complex, very close collaborative efforts are expected. This requires internetworking various codes, comparing solutions from multiple solvers, and sharing of data and data analysis tools. However, the data formats and data analysis tools used in fusion and plasma simulations are highly heterogeneous. Imposing one standard data format and one type of tools is unrealistic due to historical and practical reasons. In this paper, we propose to create the Fusion Simulation Markup Language or FSML - an XML based system for describing and accessing fusion and plasma physics simulation data of various formats used in the community. The system consists of syntactic and semantic metadata organized in specialized XML schemas and APIs written for accessing data from major data analysis and visualization tools. We present the preliminary results in formulation the FSML schema and APIs in AVS/Express modules, and demonstrate their application for two large three-dimension fusion simulation codes M3D and NIMROD. The results show that FSML schema and the set of tools developed provide a strong initial momentum and technology for the community effort to enhance data exchange and interoperability of analysis tools. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Automatic selection of loop scheduling algorithms using reinforcement learning

    Publication Year: 2005 , Page(s): 87 - 94
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (483 KB) |  | HTML iconHTML  

    This paper presents the design and implementation of a reinforcement learning agent that automatically selects appropriate loop scheduling algorithms for parallel loops embedded in time-stepping scientific applications executing on clusters. There may be a number of such loops in an application, and the loops may have different load balancing requirements. Further, loop characteristics may also change as the application progresses. Following a model-free learning approach, the learning agent assigned to a loop selects from a library the best scheduling algorithm for the loop during the lifetime of the application. The utility of the learning agent is demonstrated by its successful integration into the simulation of wave packets - an application arising from quantum mechanics. Results of statistical analysis using pairwise comparison of means on the running time of the simulation with and without the learning agent validate the effectiveness of the agent in improving the parallel performance of the simulation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Searching against distributed data using a Web service architecture

    Publication Year: 2005 , Page(s): 122 - 129
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (417 KB) |  | HTML iconHTML  

    Many condition health-monitoring applications require access to distributed data assets. The DAME project has investigated one such example based upon condition monitoring of civil aero-engine sensor data. A service-based solution is introduced that has been implemented within the Globus grid framework. It provides a general architecture for distributed search and identifies the generic functionality that is required. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Proceedings. Challenges of Large Applications in Distributed Environments

    Publication Year: 2005
    Save to Project icon | Request Permissions | PDF file iconPDF (78 KB)  
    Freely Available from IEEE
  • [Blank page]

    Publication Year: 2005 , Page(s): 0_2
    Save to Project icon | Request Permissions | PDF file iconPDF (13 KB)  
    Freely Available from IEEE
  • Proceedings Challenges of Large Applications in Distributed Environments CLADE 2005 [Title]

    Publication Year: 2005 , Page(s): i
    Save to Project icon | Request Permissions | PDF file iconPDF (40 KB)  
    Freely Available from IEEE
  • Copyright

    Publication Year: 2005 , Page(s): ii
    Save to Project icon | Request Permissions | PDF file iconPDF (73 KB)  
    Freely Available from IEEE
  • Table of contents

    Publication Year: 2005 , Page(s): iii - iv
    Save to Project icon | Request Permissions | PDF file iconPDF (170 KB)  
    Freely Available from IEEE
  • Welcome from the Workshop Chairs [CLADE 2005]

    Publication Year: 2005 , Page(s): v
    Save to Project icon | Request Permissions | PDF file iconPDF (163 KB) |  | HTML iconHTML  
    Freely Available from IEEE
  • Conference organization

    Publication Year: 2005 , Page(s): vi
    Save to Project icon | Request Permissions | PDF file iconPDF (105 KB)  
    Freely Available from IEEE
  • Applications

    Publication Year: 2005 , Page(s): 1
    Save to Project icon | Request Permissions | PDF file iconPDF (16 KB)  
    Freely Available from IEEE
  • [Breaker page]

    Publication Year: 2005 , Page(s): 2
    Save to Project icon | Request Permissions | PDF file iconPDF (11 KB)  
    Freely Available from IEEE
  • Data Management

    Publication Year: 2005 , Page(s): 23
    Save to Project icon | Request Permissions | PDF file iconPDF (17 KB)  
    Freely Available from IEEE
  • [Breaker page]

    Publication Year: 2005 , Page(s): 24
    Save to Project icon | Request Permissions | PDF file iconPDF (12 KB)  
    Freely Available from IEEE
  • [Breaker page]

    Publication Year: 2005 , Page(s): 54
    Save to Project icon | Request Permissions | PDF file iconPDF (13 KB)  
    Freely Available from IEEE