By Topic

Challenges of Large Applications in Distributed Environments, 2004. CLADE 2004. Proceedings of the Second International Workshop on

Date 7 June 2004

Filter Results

Displaying Results 1 - 22 of 22
  • A collaborative informatics infrastructure for multi-scale science

    Page(s): 24 - 33
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (326 KB)  

    The collaboratory for multiscale chemical science (CMCS) is developing a powerful informatics-based approach to synthesizing multiscale information to support a systems-based research approach and is applying it in support of combustion research. An open source multiscale informatics toolkit is being developed that addresses a number of issues core to the emerging concept of knowledge grids including provenance tracking and lightweight federation of data and application resources into cross-scale information flows. The CMCS portal is currently in use by a number of high-profile pilot groups and is playing a significant role in enabling their efforts to improve and extend community maintained chemical reference information. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An on demand path marking and capacity reservation method using split agent

    Page(s): 128 - 137
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (310 KB) |  | HTML iconHTML  

    Different schemes for large scale networks hosting distributed applications have been recently adopted for network path marking based on adaptive behavior of swarm-based agents. Topologically complex networks must use efficient routing methods in order to route data traffic from a source to a destination. In this paper the split agent-based routing technique (SART) is applied to a network in which a certain amount of data traffic is injected to mark different paths. split agent-based routing technique (SART) is a variant of swarm-based routing [R. Schoonderwoerd et al. (1997), H. G. Sandalidis et al. (2004)] where agents are split after their departure to the next node on a hop-by-hop basis at the same time performing path marking. Packets that are delay sensitive are marked as prioritized. Agents recognize these packets as being a part of, and try to influence the two way routing tables. Thorough examination has been done for the performance and path recoverability using SART algorithm for marking any requested path, taking into account a number of metrics. It is shown that the split agent scheme offers an efficient on demand path marking and path-capacity reservation in a decentralized manner (particularly efficient for large scale networks). SART method undoubtedly represents a solution for significant network optimization particularly for the equal share of network resources. Also it is shown that this scheme offers a way to increase overall performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Grids for experimental science: the virtual control room

    Page(s): 4 - 11
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (586 KB) |  | HTML iconHTML  

    The National Fusion Collaboratory focuses on enabling fusion scientists to explore grid capabilities in support of experimental science. Fusion experiments are structured as a series of plasma pulses initiated roughly every 20 minutes. In the between-pulse intervals scientists perform data analysis and discuss results to reach decisions affecting changes to the next plasma pulse. This interaction can be made more efficient by performing more analysis and engaging more expertise from a geographically distributed team of scientists and resources. In this paper, we describe a virtual control room experiment that unites collaborative, visualization, and grid technologies to provide such environment and shows how their combined effect can advance experimental science. We also report on FusionGrid services whose use during the fusion experimental cycle became possible for the first time thanks to this technology. We also describe the Access Grid, experimental data presentation tools, and agreement-based resource management and workflow systems enabling time-bounded end-to-end application execution. The first virtual control room experiment represented a mock-up of a remote interaction with the DIII-D control room and was presented at SC03 and later reviewed at an international ITER Grid Workshop. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Engineering a peer-to-peer collaboratory for tissue microarray research

    Page(s): 64 - 73
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (589 KB) |  | HTML iconHTML  

    This work presents the design of a prototype peer-to-peer collaboratory for imaging, analyzing, and seamlessly sharing tissue microarrays (TMA), correlated clinical data, and experimental results across a consortium of distributed clinical and research sites. The overarching goal of this project is to facilitate cooperative oncology research aimed at improved understanding of the underlying mechanisms of disease onset and progression while simultaneously providing new insight into therapy planning and treatment. Key components of the collaboratory include a specification of metadata schematics for characterizing TMA specimens and abstracting their interpretations, an framework for automated and accurate analysis of digitized TMAs and a peer-to-peer infrastructure for indexing and discovery of TMA data and metadata, and a novel, optimized decentralized search engine that supports flexible querying with search guarantees and bounded costs. Prototype implementations of the automated TMA analysis component and the storage/discovery component and their evaluations are presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Grid service for visualization and analysis of remote fusion data

    Page(s): 34 - 43
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (311 KB) |  | HTML iconHTML  

    Simulations and experiments in the fusion and plasma physics community generate large datasets at remote sites. Visualization and analysis of these datasets are difficult because of the incompatibility among the various data formats adopted by simulation, experiments, and analysis tools, and the large sizes of analyzed data. Grids and Web services technologies are capable of providing solutions for such heterogeneous settings, but need to be customized to the field-specific needs and merged with distributed technologies currently used by the community. This paper describes how we are addressing these issues in the fusion grid service under development. We also present performance results of relevant data transfer mechanisms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Support for data-intensive, variable-granularity grid applications via distributed file system virtualization - a case study of light scattering spectroscopy

    Page(s): 12 - 21
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (307 KB) |  | HTML iconHTML  

    A key challenge faced by large-scale, distributed applications in grid environments is efficient, seamless data management. In particular, for applications that can benefit from access to data at variable granularities, data management can pose additional programming burdens to an application developer. This work presents a case for the use of virtualized distributed file systems as a basis for data management for data-intensive, variable-granularity applications. The approach leverages on-demand transfer mechanisms of existing, de-facto network file system clients and servers that support transfers of partial data sets in an application-transparent fashion, and complement them with user-level performance and functionality enhancements such as caching and encrypted communication channels. The paper uses a nascent application from the medical imaging field (light scattering spectroscopy -LSS) as a motivation for the approach, and as a basis for evaluating its performance. Results from performance experiments that consider the 16-processor parallel execution of LSS analysis and database generation programs show that, in the presence of data locality, a virtualized wide-area distributed file system setup and configured by grid middleware can achieve performance levels close (13% overhead or less) to that of a local disk, and superior (up to 680% speedup) to nonvirtualized distributed file systems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • SIBIOS: a system for the integration of bioinformatics services

    Page(s): 74 - 83
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (395 KB) |  | HTML iconHTML  

    Modern biological and chemical studies rely on life science databases as well as sophisticated software tools (e.g., homology search tools, modeling and visualization tools). These tools often have to be combined and integrated in order to support a given study. SIBIOS (system for the integration of bioinformatics services) serves this purpose. The services are both life science database search services and software tools. The task engine is the core component of SIBIOS. It supports the execution of dynamic workflows that incorporate multiple bioinformatics services. The architecture of SIBIOS, the approaches used to address the heterogeneity as well as interoperability of bioinformatics services, including data integration are presented in this paper. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The MicroGrid: using online simulation to predict application performance in diverse grid network environments

    Page(s): 52 - 61
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (467 KB) |  | HTML iconHTML  

    Improvements in networking and middleware technology are enabling large-scale grids that aggregate resources over wide-area networks to support applications at unprecedented levels of scale and performance. Unfortunately, existing middleware and tools provide little information to users as to the suitability of a given grid topology for a specific grid application. Instead, users generally use ad-hoc performance models to evaluate mappings of their applications to resource and network topologies. Grid application behavior alone is complex, and adding resource and network behavior makes the situation even worse. As a result, users typically employ nearly blind experimentation to find good deployments of their applications in each new grid environment. Only through actual deployment and execution can a user discovers if the mapping was a good one. Further, even after finding a good configuration, there is no basis to determine if a much better configuration has been missed. This approach slows effective grid application development and deployment. We present a richer methodology for evaluating grid software and diverse grid environments based on the MicroGrid grid online simulator. With the MicroGrid, users, grid researchers, or grid operators can define and simulate arbitrary collections of resources and networks. This allows study of an existing grid testbed under controlled conditions or even to study the efficacy of higher performance environments than are available today. Further, the MicroGrid supports direct execution of grid applications unchanged. These application can be written with MPI, C, C++, Perl, and/or Python and use the Globus middleware. This enables detailed and accurate study of application behavior. This work presents: (1) the first validation of the MicroGrid for studying whole-program performance of MPI grid applications and (2) a demonstration of the MicroGrid as a tool for predicting the performance of applications on a range of grid resources and novel network topologies. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • FastMap: a distributed scheme for mapping large scale applications onto computational grids

    Page(s): 118 - 127
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (378 KB) |  | HTML iconHTML  

    Many large and complex computational applications can be modeled as irregular graphs and are typically characterized by a large number of vertices and edges. This paper proposes a new and fast mapping heuristic, called FastMap, to map this class of applications onto heterogeneous metacomputing platforms such as computational grids. While previous approaches have delved into graph partitioning of the application, we attempt to solve this problem from the clustering perspective. We exploit a hierarchical resource management infrastructure on the grid to distribute the overhead of mapping among a tree of schedulers and develop a scheme that proves to be almost linear in its scalability. Furthermore, we optimize on the result of the mapping with the help of a genetic algorithm at each scheduler node. Our experiments include a 50,000-node application graph from NASA and several other synthetically-generated graphs with as many as 100,000 vertices. Comparisons with another heterogeneous partitioner, MiniMax, show an improvement factor of over 100 on the mapping time, yet with superior quality mapping. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance optimization of a de-centralized task allocation protocol via bandwidth and buffer management

    Page(s): 108 - 117
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (584 KB) |  | HTML iconHTML  

    Sharing the resources among various users and the lack of a centralized control are two key characteristics of many distributed heterogeneous computing systems. A critical challenge for designing applications in such systems is to coordinate the resources in a decentralized fashion while adapting to the changes in the system. In this paper, we consider the computation of a large set of equal-sized independent tasks. This represents the computation paradigm for a variety of large scale applications such as SETI@home and Monte Carlo simulations. We focus on the performance optimization for a decentralized adaptive task allocation protocol. We develop a bandwidth allocation strategy based on our decentralized task allocation algorithm, and a simple task buffer management policy. Simulation results show that our task allocation protocol achieves close to the optimal system throughput. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On building parallel & grid applications: component technology and distributed services

    Page(s): 44 - 51
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (367 KB) |  | HTML iconHTML  

    Software component frameworks are well known in the commercial business application world and now this technology is being explored with great interest as a way to build large-scale scientific application on parallel computers. In the case of grid systems, the current architectural model is based on the emerging Web services framework. We describe progress that has been made on the common component architecture model (CCA) and discuss its success and limitations when applied to problems in grid computing. Our primary conclusion is that a component model fits very well with a services-oriented grid, but the model of composition must allow for a very dynamic (both in space and it time) control of composition. We note that this adds a new dimension to conventional service workflow and it extends the "inversion of control" aspects of must component systems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Application-controllable policies in the NSM distributed mass storage system

    Page(s): 96 - 105
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (510 KB) |  | HTML iconHTML  

    Distributed storage systems are increasingly being used to provide shared parallel-access environments to meet the growing demands of data-intensive applications. In distributed storage systems, performance enhancement strategies such as proper data layout, prefetching, and cache management policies are useful to better fulfill the needs of the application. However, because of the diverse nature and domains of applications, an algorithm that is optimal for one application may be the worst case scenario for others. We discuss application-controllable policies in the network storage manager (NSM). The system has a unique architecture that divides its storage policies into three categories: core system features, application-controllable policies, and fine-tunable characteristics. The system enables flexible application-defined storage policies by providing standard algorithms that are commonly used in distributed high-performance environments. In addition, application programmers can plug-in their optimized implementations of the controllable policies. The paper presents experimental results that demonstrate significant improvement in data throughput and cache hit rates leading to significant overall performance enhancement. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Morphable messaging: efficient support for evolution in distributed applications

    Page(s): 86 - 95
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (309 KB) |  | HTML iconHTML  

    All but the most briefly used systems must evolve as their mission and roles change over time. Evolution in the context of large distributed systems is extraordinarily complex because of the difficulty of upgrading all components simultaneously, and the fact that such systems are often very sensitive to changes in the message formats that underlay their communication. Prior approaches to the problem of implementing changes in a deployed system have relied upon ad-hoc solutions or protocol negotiation to avoid message format mismatches. We present a novel approach that combines message meta-data and dynamic code generation to create a robust messaging system that naturally supports application evolution. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Proceedings of the Second International Workshop on Challenges of Large Applications in Distributed Environments

    Save to Project icon | Request Permissions | PDF file iconPDF (158 KB)  
    Freely Available from IEEE
  • [Blank page]

    Page(s): 0_2
    Save to Project icon | Request Permissions | PDF file iconPDF (157 KB)  
    Freely Available from IEEE
  • Proceedings of the Second International Workshop on Challenges of Large Applications in Distributed Environments

    Page(s): 0_3
    Save to Project icon | Request Permissions | PDF file iconPDF (204 KB)  
    Freely Available from IEEE
  • Copyright page

    Page(s): 0_4
    Save to Project icon | Request Permissions | PDF file iconPDF (192 KB)  
    Freely Available from IEEE
  • Proceedings of the Second International Workshop on Challenges of Large Applications in Distributed Environments Table of contents

    Page(s): 0_5 - 0_6
    Save to Project icon | Request Permissions | PDF file iconPDF (182 KB)  
    Freely Available from IEEE
  • Welcome from the workshop chairs

    Page(s): 0_7
    Save to Project icon | Request Permissions | PDF file iconPDF (173 KB)  
    Freely Available from IEEE
  • Conference organization

    Page(s): 0_8
    Save to Project icon | Request Permissions | PDF file iconPDF (176 KB)  
    Freely Available from IEEE
  • A grid based diagnostics and prognosis system for rolls royce aero engines: the DAME project

    Page(s): 2
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB) |  | HTML iconHTML  

    This talk will describe an implementation of a distributed data intensive Grid application aimed at diagnosis and prognosis of Rolls- Royce Aero Engines. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author index

    Page(s): 138
    Save to Project icon | Request Permissions | PDF file iconPDF (169 KB)  
    Freely Available from IEEE