Ktl-icon-tai-lieu

Thuật toán Tối ưu

Được đăng lên bởi Dương Béo
Số trang: 3 trang   |   Lượt xem: 448 lần   |   Lượt tải: 0 lần
GFS is optimized for Google's core data storage and usage needs (primarily the search
engine), which can generate enormous amounts of data that needs to be retained;[1] Google
File System grew out of an earlier Google effort, "BigFiles", developed by Larry Page and
Sergey Brin in the early days of Google, while it was still located in Stanford.[1] Files are
divided into fixed-size chunks of 64 megabytes,[1] similar to clusters or sectors in regular file
systems, which are only extremely rarely overwritten, or shrunk; files are usually appended to
or read. It is also designed and optimized to run on Google's computing clusters, dense nodes
which consist of cheap, "commodity" computers, which means precautions must be taken
against the high failure rate of individual nodes and the subsequent data loss. Other design
decisions select for high data throughputs, even when it comes at the cost of latency.
A GFS cluster consists of multiple nodes. These nodes are divided into two types: one
Master node and a large number of Chunkservers. Each file is divided into fixed-size chunks.
Chunkservers store these chunks. Each chunk is assigned a unique 64-bit label by the master
node at the time of creation, and logical mappings of files to constituent chunks are
maintained. Each chunk is replicated several times throughout the network, with the
minimum being three, but even more for files that have high end-in demand or need more
redundancy.
The Master server doesn't usually store the actual chunks, but rather all the metadata
associated with the chunks, such as the tables mapping the 64-bit labels to chunk locations
and the files they make up, the locations of the copies of the chunks, what processes are
reading or writing to a particular chunk, or taking a "snapshot" of the chunk pursuant to
replicate it (usually at the instigation of the Master server, when, due to node failures, the
number of copies of a chunk has fallen beneath the set number). All this metadata is kept
current by the Master server periodically receiving updates from each chunk server ("Heartbeat messages").
Permissions for modifications are handled by a system of time-limited, expiring "leases",
where the Master server grants permission to a process for a finite period of time during
which no other process will be granted permission by the Master server to modify the chunk.
The modifying chunkserver, which is always the primary chunk holder, then propagates the
changes to the chunkservers with the...
GFS is optimized for Google's core data storage and usage needs (primarily the search
engine), which can generate enormous amounts of data that needs to be retained;
[1]
Google
File System grew out of an earlier Google effort, "BigFiles", developed by Larry Page and
Sergey Brin in the early days of Google, while it was still located in Stanford.
[1]
Files are
divided into fixed-size chunks of 64 megabytes,
[1]
similar to clusters or sectors in regular file
systems, which are only extremely rarely overwritten, or shrunk; files are usually appended to
or read. It is also designed and optimized to run on Google's computing clusters, dense nodes
which consist of cheap, "commodity" computers, which means precautions must be taken
against the high failure rate of individual nodes and the subsequent data loss. Other design
decisions select for high data throughputs, even when it comes at the cost of latency.
A GFS cluster consists of multiple nodes. These nodes are divided into two types: one
Master node and a large number of Chunkservers. Each file is divided into fixed-size chunks.
Chunkservers store these chunks. Each chunk is assigned a unique 64-bit label by the master
node at the time of creation, and logical mappings of files to constituent chunks are
maintained. Each chunk is replicated several times throughout the network, with the
minimum being three, but even more for files that have high end-in demand or need more
redundancy.
The Master server doesn't usually store the actual chunks, but rather all the metadata
associated with the chunks, such as the tables mapping the 64-bit labels to chunk locations
and the files they make up, the locations of the copies of the chunks, what processes are
reading or writing to a particular chunk, or taking a "snapshot" of the chunk pursuant to
replicate it (usually at the instigation of the Master server, when, due to node failures, the
number of copies of a chunk has fallen beneath the set number). All this metadata is kept
current by the Master server periodically receiving updates from each chunk server ("Heart-
beat messages").
Permissions for modifications are handled by a system of time-limited, expiring "leases",
where the Master server grants permission to a process for a finite period of time during
which no other process will be granted permission by the Master server to modify the chunk.
The modifying chunkserver, which is always the primary chunk holder, then propagates the
changes to the chunkservers with the backup copies. The changes are not saved until all
chunkservers acknowledge, thus guaranteeing the completion and atomicity of the operation.
Programs access the chunks by first querying the Master server for the locations of the
desired chunks; if the chunks are not being operated on (i.e. no outstanding leases exist), the
Master replies with the locations, and the program then contacts and receives the data from
the chunkserver directly (similar to Kazaa and its supernodes).
Unlike most other file systems, GFS is not implemented in the kernel of an operating system,
but is instead provided as a userspace library.
Performance
Deciding from benchmarking results,
[3]
when used with relatively small number of servers
(15), the file system achieves reading performance comparable to that of a single disk (80–
100 MB/s), but has a reduced write performance (30 MB/s), and is relatively slow (5 MB/s)
Thuật toán Tối ưu - Trang 2
Thuật toán Tối ưu - Người đăng: Dương Béo
5 Tài liệu rất hay! Được đăng lên bởi - 1 giờ trước Đúng là cái mình đang tìm. Rất hay và bổ ích. Cảm ơn bạn!
3 Vietnamese
Thuật toán Tối ưu 9 10 877