Scalability - IT Library : ZDNet Asia

White Papers

Case Studies

Vendor Webcast

Show all content

Title Date Added Company

A Scalable Peer-to-Peer System for Music Content and Information Retrieval 2008-01-01 Johns Hopkins University

Currently a large percentage of Internet traffic consists of music files, typically stored in MP3 compressed audio format, shared and exchanged over Peer-to-Peer (P2P) networks. Searching for music is performed by specifying keywords and naive string matching techniques. In the past years the emerging research area of Music Information Retrieval (MIR) has produced a variety of new ways of looking at the problem of music search. Such MIR techniques can significantly enhance the ways user search for music over P2P networks. In order for that to happen there are two main challenges that need to be addressed: scalability to large collections and number of peers, richer set of search semantics that can support MIR especially when retrieval is content-based.

Tags: Network Design

VDCBPI: An Approximate Scalable Algorithm for Large POMDPs 2008-01-01 University of Toronto

Existing algorithms for discrete partially observable Markov decision processes can at best solve problems of a few thousand states due to two important sources of intractability: the curse of dimensionality and the policy space complexity. This paper describes a new algorithm (VDCBPI) that mitigates both sources of intractability by combining the Value Directed Compression (VDC) technique with Bounded Policy Iteration (BPI). The scalability of VDCBPI is demonstrated on synthetic network management problems with up to 33 million states.

Tags: Software Engineering

Scalable Hybrid Search on Distributed Databases 2008-01-01 Florida State University

A hybrid keyword search that combines metadata search with a traditional keyword search over unstructured context data has been previously described. This hybrid search paradigm provides the inquirer additional options to narrow the search with some semantic aspect from the XML metadata query. But in earlier work, the scalability limitations of a single-machine implementation was experienced. This paper describes a scalable hybrid search on distributed databases. This scalable hybrid search provides a total query result from the collection of individual inquiries against independent data fragments distributed in a computer cluster. The paper demonstrates the architecture extends the scalability of a native XML query limited in a single machine and improves the performance for some queries.

Tags: XML, Database Management

A Scalable Content-Addressable Network 2008-01-01 University of California

Hash tables - which map "Keys" onto "Values" - are an essential building block in modern software systems. It is believed that a similar functionality would be equally valuable to large distributed systems. This paper introduces the concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales. The CAN design is scalable, fault-tolerant and completely self-organizing, and the paper demonstrates its scalability, robustness and low-latency properties through simulation.

Tags: Network Design

Pastis: A Highly-Scalable Multi-User Peer-to-Peer File System 2008-01-01 French National Institute for Research in Computer Science and Control

This paper introduces Pastis, a completely decentralized multi-user read-write peer-to-peer file system. In Pastis every file is described by a modifiable inode-like structure which contains the addresses of the immutable blocks in which the le contents are stored. All data are stored using the Past Distributed Hash Table (DHT), which the paper has modified in order to reduce the number of network messages it generates, thus optimizing replica retrieval. Pastis' design is simple compared to other existing systems, as it does not require complex algorithms like Byzantine-Fault Tolerant (BFT) replication or a central administrative authority. It is also highly scalable in terms of the number of network nodes and users sharing a given file or portion of the file system.

Tags: Network Design, Fault-Tolerant Servers

An Incrementally Scalable Multiprocessor Interconnection Network With Flexible Topology and Low-Cost Distributed Switching 2008-01-01 Monash University

Massively parallel computing architectures are becoming widely accepted in many computationally intensive areas. One of the prime advantages touted is their scalability, and yet while in principle a good degree of scalability is possible, in practice the unit of scaling is very coarse. The net effect of this is to make incremental expansion of such machines impractical except for large and expensive expansion to multiples of the original size and in some cases expansion by powers of two is required. In this paper a multiprocessor interconnection scheme with low fixed overheads and linear incremental scaling costs is described.

Tags: Switching, Processors

Shared Receive Queue Based Scalable MPI Design for InfiniBand Clusters 2008-01-01 Ohio State University

Clusters of several thousand nodes interconnected with InniBand, an emerging high-performance interconnect, have already appeared in the Top 500 list. The next-generation InniBand clusters are expected to be even larger with tens-of-thousands of nodes. A high-performance scalable MPI design is crucial for MPI applications in order to exploit the massive potential for parallelism in these very large clusters. MVAPICH is a popular implementation of MPI over InniBand based on its reliable connection oriented model. The requirement of this model to make communication buffers available for each connection imposes a memory scalability problem. In order to mitigate this issue, the latest InniBand standard includes a new feature called Shared Receive Queue (SRQ) which allows sharing of communication buffers across multiple connections.

Tags: Network Design

DOH: A Content Delivery Peer-to-Peer Network 2008-01-01 Swedish Institute of Computer Science

Many SMEs and non-profit organizations suffer when their Web servers become unavailable due to flash crowd effects when their web site becomes popular. One of the solutions to the flash-crowd problem is to place the web site on a scalable CDN (Content Delivery Network) that replicates the content and distributes the load in order to improve its response time. This paper presents the approach to building a scalable Web Hosting environment as a CDN on top of a structured peer-to-peer system of collaborative web-servers integrated to share the load and to improve the overall system performance, scalability, availability and robustness. Unlike cluster-based solutions, it can run on heterogeneous hardware, over geographically dispersed areas.

Tags: Network Design, Collaborative Web

Elements of a Scalable Network File System Protocol 2008-01-01 EMC

Standardized Network File System protocols (e.g. NFS, CIFS) have been highly successful in enabling data to be shared among multiple clients and across heterogeneous computing platforms. The key to this success is that the protocols present data objects at a level of abstraction at which there is a broad common view among the multiple competing computing platforms: files and directories make sense in many operating systems. The flip side of this success is that by strictly enforcing this abstract view, the network file server becomes a road block to effectively scaling the level of shared access to the data.

Tags: File and Network Servers

On the Scalability of Fair Queueing 2008-01-01 Politecnico di Torino

The scalability of fair queueing depends on the number of flows that have, or recently have had, packets in the queue. Applying known facts about statistical traffic characteristics at flow level, the paper demonstrates that this number is typically very small for any link capacity. This suggests fair queueing may be perfectly feasible opening interesting possibilities for the development of the Internet architecture. The evaluation is based on trace simulations and an analytical traffic model.

Jump to 1 2 [3] 4 5 6 7 8 9 10

Title	Date Added	Company
A Scalable Peer-to-Peer System for Music Content and Information Retrieval	2008-01-01	Johns Hopkins University
Currently a large percentage of Internet traffic consists of music files, typically stored in MP3 compressed audio format, shared and exchanged over Peer-to-Peer (P2P) networks. Searching for music is performed by specifying keywords and naive string matching techniques. In the past years the emerging research area of Music Information Retrieval (MIR) has produced a variety of new ways of looking at the problem of music search. Such MIR techniques can significantly enhance the ways user search for music over P2P networks. In order for that to happen there are two main challenges that need to be addressed: scalability to large collections and number of peers, richer set of search semantics that can support MIR especially when retrieval is content-based. Tags: Network Design
VDCBPI: An Approximate Scalable Algorithm for Large POMDPs	2008-01-01	University of Toronto
Existing algorithms for discrete partially observable Markov decision processes can at best solve problems of a few thousand states due to two important sources of intractability: the curse of dimensionality and the policy space complexity. This paper describes a new algorithm (VDCBPI) that mitigates both sources of intractability by combining the Value Directed Compression (VDC) technique with Bounded Policy Iteration (BPI). The scalability of VDCBPI is demonstrated on synthetic network management problems with up to 33 million states. Tags: Software Engineering
Scalable Hybrid Search on Distributed Databases	2008-01-01	Florida State University
A hybrid keyword search that combines metadata search with a traditional keyword search over unstructured context data has been previously described. This hybrid search paradigm provides the inquirer additional options to narrow the search with some semantic aspect from the XML metadata query. But in earlier work, the scalability limitations of a single-machine implementation was experienced. This paper describes a scalable hybrid search on distributed databases. This scalable hybrid search provides a total query result from the collection of individual inquiries against independent data fragments distributed in a computer cluster. The paper demonstrates the architecture extends the scalability of a native XML query limited in a single machine and improves the performance for some queries. Tags: XML, Database Management
A Scalable Content-Addressable Network	2008-01-01	University of California
Hash tables - which map "Keys" onto "Values" - are an essential building block in modern software systems. It is believed that a similar functionality would be equally valuable to large distributed systems. This paper introduces the concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales. The CAN design is scalable, fault-tolerant and completely self-organizing, and the paper demonstrates its scalability, robustness and low-latency properties through simulation. Tags: Network Design
Pastis: A Highly-Scalable Multi-User Peer-to-Peer File System	2008-01-01	French National Institute for Research in Computer Science and Control
This paper introduces Pastis, a completely decentralized multi-user read-write peer-to-peer file system. In Pastis every file is described by a modifiable inode-like structure which contains the addresses of the immutable blocks in which the le contents are stored. All data are stored using the Past Distributed Hash Table (DHT), which the paper has modified in order to reduce the number of network messages it generates, thus optimizing replica retrieval. Pastis' design is simple compared to other existing systems, as it does not require complex algorithms like Byzantine-Fault Tolerant (BFT) replication or a central administrative authority. It is also highly scalable in terms of the number of network nodes and users sharing a given file or portion of the file system. Tags: Network Design, Fault-Tolerant Servers
An Incrementally Scalable Multiprocessor Interconnection Network With Flexible Topology and Low-Cost Distributed Switching	2008-01-01	Monash University
Massively parallel computing architectures are becoming widely accepted in many computationally intensive areas. One of the prime advantages touted is their scalability, and yet while in principle a good degree of scalability is possible, in practice the unit of scaling is very coarse. The net effect of this is to make incremental expansion of such machines impractical except for large and expensive expansion to multiples of the original size and in some cases expansion by powers of two is required. In this paper a multiprocessor interconnection scheme with low fixed overheads and linear incremental scaling costs is described. Tags: Switching, Processors
Shared Receive Queue Based Scalable MPI Design for InfiniBand Clusters	2008-01-01	Ohio State University
Clusters of several thousand nodes interconnected with InniBand, an emerging high-performance interconnect, have already appeared in the Top 500 list. The next-generation InniBand clusters are expected to be even larger with tens-of-thousands of nodes. A high-performance scalable MPI design is crucial for MPI applications in order to exploit the massive potential for parallelism in these very large clusters. MVAPICH is a popular implementation of MPI over InniBand based on its reliable connection oriented model. The requirement of this model to make communication buffers available for each connection imposes a memory scalability problem. In order to mitigate this issue, the latest InniBand standard includes a new feature called Shared Receive Queue (SRQ) which allows sharing of communication buffers across multiple connections. Tags: Network Design
DOH: A Content Delivery Peer-to-Peer Network	2008-01-01	Swedish Institute of Computer Science
Many SMEs and non-profit organizations suffer when their Web servers become unavailable due to flash crowd effects when their web site becomes popular. One of the solutions to the flash-crowd problem is to place the web site on a scalable CDN (Content Delivery Network) that replicates the content and distributes the load in order to improve its response time. This paper presents the approach to building a scalable Web Hosting environment as a CDN on top of a structured peer-to-peer system of collaborative web-servers integrated to share the load and to improve the overall system performance, scalability, availability and robustness. Unlike cluster-based solutions, it can run on heterogeneous hardware, over geographically dispersed areas. Tags: Network Design, Collaborative Web
Elements of a Scalable Network File System Protocol	2008-01-01	EMC
Standardized Network File System protocols (e.g. NFS, CIFS) have been highly successful in enabling data to be shared among multiple clients and across heterogeneous computing platforms. The key to this success is that the protocols present data objects at a level of abstraction at which there is a broad common view among the multiple competing computing platforms: files and directories make sense in many operating systems. The flip side of this success is that by strictly enforcing this abstract view, the network file server becomes a road block to effectively scaling the level of shared access to the data. Tags: File and Network Servers
On the Scalability of Fair Queueing	2008-01-01	Politecnico di Torino
The scalability of fair queueing depends on the number of flows that have, or recently have had, packets in the queue. Applying known facts about statistical traffic characteristics at flow level, the paper demonstrates that this number is typically very small for any link capacity. This suggests fair queueing may be perfectly feasible opening interesting possibilities for the development of the Internet architecture. The evaluation is based on trace simulations and an analytical traffic model.