A “cache” duplicates, some or all the data stored on one or more databases for fast and scalable access by applications. Leveraging a caching system is valuable when the original data is expensive to fetch from the source database due to resource constraints. Once the source data is populated in the cache, future requests for the same data make use of the cached copy rather than fetching the data from the original source. Introducing a “caching layer” typically makes data access times faster and improves the ability for the underlying database to scale in order to accommodate consistently higher loads or spikes in data requests. Often, the cache is “distributed”, so that the contents of the cache can be spread out over multiple systems to make more efficient use of available memory, network and
other computing resources.
Memcached is a powerful open source distributed memory caching system. It is a high performance distributed memory for object caching system. It is generic in nature but highly intended to be used for speeding up application by alleviating database load.
Many of the largest and most heavily trafficked web properties on the Internet like Facebook, Fotolog, YouTube, Mixi.jp, Yahoo, and Wikipedia deploy Memcached and MySQL to satisfy the demands of millions of users and billions of page views every month. By integrating a caching tier into their web scale architectures, these organizations have improved their application performance while minimizing database load.
How Memcached works ??
Memcached is a hash procedure for turning data into a small integer that serves as an index into an array. The net result is that it speeds up table lookup or data comparison tasks. Memcached leverages a two-stage hash that acts as like a giant hash table looking up key = value pairs. Memcached can be thought of as having two core components, a server and a client. In the course of a memcached lookup, the client hashes the key against a list of servers. When the
server is identified, the client sends its request to the server who performs a hash key lookup for the actual data. Because the client performs one stage of the hashing, memcached naturally lends itself towards the ability to easily add dozens of additional nodes.
Basic Example:
A basic memcached lookup can be illustrated in the example below with Clients C1, C2, C3 and Servers S1, S2 and S3.
‘Set’ the Key and Value
-Client C1 wants to set the key “ key1” with value “value1”
-Client C1 takes the list of available memcached servers (S1,S2,S3) and hashes the key against them
-Server S2 is selected
-Client C1 directly connects to Server S2, and sets key “key1” with the value “value1” ‘Get’ the Key and Value
-Client C3 wants to get key “key1”
-Client C3 is able to use the same hashing process to determine that key “key1” is on Server S2
-Client C3 directly requests key “key1” from Server S2 and gets back “value1”
-Subsequent requests from Clients C1, C2 or C3 for key “key1” will access the data directly from Server S2.
Components of Memcached
There are two core components to memcached, the server and the client. In this section we cover some of the core capabilities of the memcached server, including how the memcached server deals with memory allocation, the caching of data.
Memached server
The memached server is implemented as a non-blocking event-based server with an emphasis on scalability and low resource consumption.
Memory Allocation
By default, the memcached server allocates memory by leveraging what is internally referred to as a “slab allocator”. The reason why this internal slab allocator is used over malloc/free (a standard C/C++ library for performing dynamic memory allocation) is to avoid fragmentation and the operating system having to spend cycles searching for contiguous blocks of memory. These tasks overall, tend to consume more resources than the memcached process itself. With the slab allocator, memory is allocated in chunks and in turn, is constantly being reused. Because memory is allocated into different sized slabs, it does open up the potential to waste memory if the data being cached does not fit perfectly into the slab. There are also some practical limits to be aware of concerning key and data size limits. For example, keys are restricted to 250 characters and cached data cannot exceed the largest
slab size currently supported, 1 megabyte.
The hashing algorithm that memcached makes use of to determine which server a key is cached on does not take into account the size of all the memory available across all the servers participating in a memcached cluster. For those concerned about this issue, a potential workaround is to run multiple memcached instances on servers with more memory with each instance using the same size cache as all the other servers.
Caching Structure
When memcached’s distributed hash table becomes full, subsequent inserts force older cached data to be cycled out in a least recently used (LRU) order with associated expiration timeouts. How long data is “valid” within the cache is set via configuration options. This “validity” time may be a short, long or permanent. As mentioned, when the memcached server exhausts its memory allocation, the expired slabs are cycled out with the next oldest, unused slabs queued up next for expiration. Memcached makes use of lazy expiration. This means it does not make use of additional CPU cycles to expire items. When data is requested via a ‘get’ request, memcached references the expiration time to confirm if the data is valid before returning it to the client requesting the data. When new data is being added to the cache via a ‘set’, and memcached is unable to allocate an additional slab, expired data will be cycled out prior to any data that qualifies for the LRU criteria.
Data Redundancy and Fail Over
By design, there are no data redundancy features built into memcached. Memcached is designed to be a scalable and high performance caching-layer, including data redundancy functionality would only add complexity and overhead to the system. In the event one of the memcached servers does suffer a loss of data, under normal circumstances it should still be able to retrieve its data from the original source database. A prudent caching design involves ensuring that your application can continue to function without the availability of one or more memcached nodes. Some precautions to take in order not to suddenly overwhelm the database(s) in the event of memcached failures is to add additional memcached nodes to minimize the impact of any individual node failure. Another option is to leverage a “hot backup”, in other words a server that can take over the IP address of the failed memcached server. Also, by design, memcached does not have any built in fail over capabilities. However, there are some strategies one can employ to help minimize the impact of failed memcached nodes. The first technique involves simply having an (over) abundance of nodes. Because memcached is designed to scale straight out-of-the-box, this is a key characteristic to exploit. Having plenty of memcached nodes minimizes the overall impact an outage of one or more nodes will have on the system as a whole.
One can also remove failed nodes from the server list against which the memcached clients hash against. A practical consideration here is that when clients add or remove servers from the server list, they will invalidate the entire cache. The likely effect being that the majority of the keys will in-turn hash to different servers. Essentially, this will force all the data to be re-keyed into memcached from the source database(s). As mentioned previously, leveraging a “hot backup” server which can take over the IP address of the failed memcached server can go a long way to minimize the impact of having to reload an invalidated cache. Loading and Unloading Memcached In general, “warming-up” memcached from a database dump is not the best course of action to take. Several issues arise with this method. First, changes to data that have occurred between the data dump and load will not be accounted for. Second, there must be some strategy for dealing with data that may have expired prior to the dump being loaded. In situations where there are large amounts of fairly static or permanent data to be cached,using a data dump/load can be useful for warming up the cache quickly.
Memcached Threads
With the release of version 1.2, memcached can now utilize multiple CPUs and share the cached between the CPUs engaged in processing. At its core is a simple locking mechanism that is used when particular values or data needs to be updated. Memcached threads presents considerable value to those making use of ‘multi-gets’, which in-turn become more efficient and makes memcached overall easier to manage, as it avoids having to run multiple nodes on the same server to achieve the desired level of performance.
Memached Client
It is important to note that there are different memcached client implementations. Not only does the manner in which they store data into memcached vary between client libraries, but also in how they implement the hashing algorithm. Because of this, the implications of mixing and matching client libraries and versions should be carefully considered. However, unlike the variances found between client versions, memcached servers always store data
and apply hashing algorithms consistently. Spymemcached – Java Client for Memcached.
1. Working with Memcached & MySql.
In the above architecture we combine several memcached servers and a stand-alone MySQL server. This architecture allows for scaling a read intensive application. The memcached clients are illustrated separately; however, they typically will be co-located with the application server.
2.Memcached & MySql with Replication.
In this architecture we combine several memcached servers with a master MySQL server and multiple slave servers. This architecture allows for scaling a read intensive application. The memcached clients illustrated are co-located with the application servers. Read requests can be satisfied from memcached servers or from MySQL slave servers. Writes are directed to the MySQL master server.
3.Memcached Sharding & MySql with Replication.
With sharding (application partitioning) we partition data across multiple physical servers to gain read and write scalability. In this example we have created two shards partitioned by customer number. Read requests can be satisfied from memcached servers or from MySQL slave servers. Writes are directed to the appropriate MySQL master server.
Conclusion
-Implement a scalable, high performance data caching solution for their online
applications
-Reduce database Total Cost of Ownership (TCO) by eliminating licensing costs for Proprietary data caching software
-Reduce system TCO by making better use of resources like idle or spare RAM on existing systems
-Incrementally add/remove data caching capacity, on-demand to quickly meet
changing requirements.
No comments:
Post a Comment