Hello community,
here is the log from the commit of package memcached for openSUSE:Factory
checked in at Sat Jun 6 00:46:30 CEST 2009.
--------
--- memcached/memcached.changes 2008-11-28 16:05:59.000000000 +0100
+++ memcached/memcached.changes 2009-06-05 15:26:50.000000000 +0200
@@ -1,0 +2,38 @@
+Fri Jun 5 03:19:40 CEST 2009 - mrueckert@suse.de
+
+- update to version 1.2.8:
+ - make -b command actually work
+ - *critical bugfix*. In 1.2.7 under multithreaded mode, memcached
+ would never restart accepting connections after hitting the
+ maximum connection limit.
+ - remove 'stats maps' command, as it is a potential information
+ leak, usable if versions prior to 1.2.8 ever have buffer
+ overflows discovered. (bnc#501656) CVE-2009-1494
+- additional changes from version 1.2.7
+ - reset new stats with 'stats reset'
+ - Slew of new tests. (misc, mostly Dustin Sallings)
+ - Minor bug fixes. (misc, mostly Dustin Sallings, some Dormando)
+ (see git history for full list)
+ - -b command for setting the tcp listen backlog (Chris Goffinet)
+ - Workaround for a more major bug that very rarely makes
+ memcached stop allowing new data to be set. (Dormando)
+ - Print why a key was expired in -vv mode (Dormando)
+ - cmd_flush stat (Dormando)
+ - listen_disabled_num stat for determining if you've hit the
+ maxconns limit (Dormando)
+ - Display error status on listen failures (Dormando)
+ - Remove managed instance code. Incomplete/etc. (Dormando)
+ - Handle broken IPV6 stacks better (Brian Aker)
+ - Generate warnings on setsockopt() failures (Brian Aker)
+ - Fix some indentation issues (Brian Aker)
+ - UDP/TCP can be disabled by setting their port to zero (Brian
+ Aker)
+ - Zero out libevent thread structures before use (Ricky Zhou)
+ - New stat: Last accessed time for last evicted item per slab
+ class. (Dormando)
+ - Use a dedicated socket accept thread (Facebook)
+ - Add -R option. Limit the number of requests processed by a
+ connection at once. Prevents starving other threads if bulk
+ loading. (Facebook)
+
+-------------------------------------------------------------------
calling whatdependson for head-i586
Old:
----
memcached-1.2.6.tar.bz2
New:
----
memcached-1.2.8.tar.bz2
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ memcached.spec ++++++
--- /var/tmp/diff_new_pack.A18958/_old 2009-06-06 00:45:03.000000000 +0200
+++ /var/tmp/diff_new_pack.A18958/_new 2009-06-06 00:45:03.000000000 +0200
@@ -1,7 +1,7 @@
#
-# spec file for package memcached (Version 1.2.6)
+# spec file for package memcached (Version 1.2.8)
#
-# Copyright (c) 2008 SUSE LINUX Products GmbH, Nuernberg, Germany.
+# Copyright (c) 2009 SUSE LINUX Products GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -19,8 +19,8 @@
Name: memcached
-Version: 1.2.6
-Release: 5
+Version: 1.2.8
+Release: 1
%define pkg_name memcached
%define pkg_version %{version}
#
@@ -114,6 +114,41 @@
%dir %attr(755,root,root) %{home_dir}
%changelog
+* Fri Jun 05 2009 mrueckert@suse.de
+- update to version 1.2.8:
+ - make -b command actually work
+ - *critical bugfix*. In 1.2.7 under multithreaded mode, memcached
+ would never restart accepting connections after hitting the
+ maximum connection limit.
+ - remove 'stats maps' command, as it is a potential information
+ leak, usable if versions prior to 1.2.8 ever have buffer
+ overflows discovered. (bnc#501656) CVE-2009-1494
+- additional changes from version 1.2.7
+ - reset new stats with 'stats reset'
+ - Slew of new tests. (misc, mostly Dustin Sallings)
+ - Minor bug fixes. (misc, mostly Dustin Sallings, some Dormando)
+ (see git history for full list)
+ - -b command for setting the tcp listen backlog (Chris Goffinet)
+ - Workaround for a more major bug that very rarely makes
+ memcached stop allowing new data to be set. (Dormando)
+ - Print why a key was expired in -vv mode (Dormando)
+ - cmd_flush stat (Dormando)
+ - listen_disabled_num stat for determining if you've hit the
+ maxconns limit (Dormando)
+ - Display error status on listen failures (Dormando)
+ - Remove managed instance code. Incomplete/etc. (Dormando)
+ - Handle broken IPV6 stacks better (Brian Aker)
+ - Generate warnings on setsockopt() failures (Brian Aker)
+ - Fix some indentation issues (Brian Aker)
+ - UDP/TCP can be disabled by setting their port to zero (Brian
+ Aker)
+ - Zero out libevent thread structures before use (Ricky Zhou)
+ - New stat: Last accessed time for last evicted item per slab
+ class. (Dormando)
+ - Use a dedicated socket accept thread (Facebook)
+ - Add -R option. Limit the number of requests processed by a
+ connection at once. Prevents starving other threads if bulk
+ loading. (Facebook)
* Fri Nov 28 2008 ro@suse.de
- ignore test suite results for the moment
(will not work without networking support in build environment)
@@ -143,7 +178,7 @@
cleanup with performance improvements. Memcached now collects
eviction and per-object-type statistics.
- synced with memcached-unstable
-* Tue Dec 05 2006 mrueckert@suse.de
+* Wed Dec 06 2006 mrueckert@suse.de
- Update to version 1.2.1:
o mainly fixes a stability issue reported on the mailinglist
o a few optimization fixes
++++++ memcached-1.2.6.tar.bz2 -> memcached-1.2.8.tar.bz2 ++++++
++++ 7612 lines of diff (skipped)
++++ retrying with extended exclude list
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/ChangeLog new/memcached-1.2.8/ChangeLog
--- old/memcached-1.2.6/ChangeLog 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/ChangeLog 2009-04-11 08:25:00.000000000 +0200
@@ -1,3 +1,64 @@
+2009-04-10 [Version 1.2.8 released]
+
+ * make -b command actually work
+
+ * *critical bugfix*. In 1.2.7 under multithreaded mode, memcached would
+ never restart accepting connections after hitting the maximum connection
+ limit.
+
+ * remove 'stats maps' command, as it is a potential information leak,
+ usable if versions prior to 1.2.8 ever have buffer overflows
+ discovered.
+
+2009-04-02 [Version 1.2.7 released]
+
+ * reset new stats with 'stats reset'
+
+2009-03-30 [Version 1.2.7-rc1 released]
+
+ * Slew of new tests. (misc, mostly Dustin Sallings)
+
+ * Minor bug fixes. (misc, mostly Dustin Sallings, some Dormando)
+ (see git history for full list)
+
+ * -b command for setting the tcp listen backlog (Chris Goffinet)
+
+ * Workaround for a more major bug that very rarely makes memcached stop
+ allowing new data to be set. (Dormando)
+
+ * Print why a key was expired in -vv mode (Dormando)
+
+ * cmd_flush stat (Dormando)
+
+ * listen_disabled_num stat for determining if you've hit the maxconns
+ limit (Dormando)
+
+2008-09-06
+
+ * Display error status on listen failures (Dormando)
+
+ * Remove managed instance code. Incomplete/etc. (Dormando)
+
+ * Handle broken IPV6 stacks better (Brian Aker)
+
+ * Generate warnings on setsockopt() failures (Brian Aker)
+
+ * Fix some indentation issues (Brian Aker)
+
+ * UDP/TCP can be disabled by setting their port to zero (Brian Aker)
+
+ * Zero out libevent thread structures before use (Ricky Zhou)
+
+ * New stat: Last accessed time for last evicted item per slab class.
+ (Dormando)
+
+ * Use a dedicated socket accept thread (Facebook)
+
+ * Add -R option. Limit the number of requests processed by a connection
+ at once. Prevents starving other threads if bulk loading. (Facebook)
+
+2008-07-29 [Version 1.2.6 released]
+
2008-07-24 [Version 1.2.6-rc1 released]
* Add support for newer automake (Facebook)
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/configure.ac new/memcached-1.2.8/configure.ac
--- old/memcached-1.2.6/configure.ac 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/configure.ac 2009-04-11 09:03:06.000000000 +0200
@@ -1,5 +1,5 @@
AC_PREREQ(2.52)
-AC_INIT(memcached, 1.2.6, brad@danga.com)
+AC_INIT(memcached, 1.2.8, brad@danga.com)
AC_CANONICAL_SYSTEM
AC_CONFIG_SRCDIR(memcached.c)
AM_INIT_AUTOMAKE(AC_PACKAGE_NAME, AC_PACKAGE_VERSION)
@@ -29,7 +29,7 @@
AC_SUBST(DTRACEFLAGS)
AC_ARG_ENABLE(64bit,
- [AS_HELP_STRING([--enable-64bit],[build 64bit verison])])
+ [AS_HELP_STRING([--enable-64bit],[build 64bit version])])
if test "x$enable_64bit" == "xyes"
then
org_cflags=$CFLAGS
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/daemon.c new/memcached-1.2.8/daemon.c
--- old/memcached-1.2.6/daemon.c 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/daemon.c 2009-04-11 05:48:08.000000000 +0200
@@ -35,6 +35,7 @@
#endif
#include
+#include
#include
#include
@@ -54,15 +55,33 @@
if (setsid() == -1)
return (-1);
- if (nochdir == 0)
- (void)chdir("/");
+ if (nochdir == 0) {
+ if(chdir("/") != 0) {
+ perror("chdir");
+ return (-1);
+ }
+ }
if (noclose == 0 && (fd = open("/dev/null", O_RDWR, 0)) != -1) {
- (void)dup2(fd, STDIN_FILENO);
- (void)dup2(fd, STDOUT_FILENO);
- (void)dup2(fd, STDERR_FILENO);
- if (fd > STDERR_FILENO)
- (void)close(fd);
+ if(dup2(fd, STDIN_FILENO) < 0) {
+ perror("dup2 stdin");
+ return (-1);
+ }
+ if(dup2(fd, STDOUT_FILENO) < 0) {
+ perror("dup2 stdout");
+ return (-1);
+ }
+ if(dup2(fd, STDERR_FILENO) < 0) {
+ perror("dup2 stderr");
+ return (-1);
+ }
+
+ if (fd > STDERR_FILENO) {
+ if(close(fd) < 0) {
+ perror("close");
+ return (-1);
+ }
+ }
}
return (0);
}
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/doc/CONTRIBUTORS new/memcached-1.2.8/doc/CONTRIBUTORS
--- old/memcached-1.2.6/doc/CONTRIBUTORS 2008-04-28 07:14:16.000000000 +0200
+++ new/memcached-1.2.8/doc/CONTRIBUTORS 2009-04-11 05:48:08.000000000 +0200
@@ -37,3 +37,7 @@
Jean-Francois Bustarret
Paul G
Paul Lindner
+Dustin Sallings
+Tomash Brechko
+Brian Aker
+Dormando
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/doc/Makefile new/memcached-1.2.8/doc/Makefile
--- old/memcached-1.2.6/doc/Makefile 2008-07-29 18:39:11.000000000 +0200
+++ new/memcached-1.2.8/doc/Makefile 2009-04-11 09:03:20.000000000 +0200
@@ -1,4 +1,4 @@
-# Makefile.in generated by automake 1.10.1 from Makefile.am.
+# Makefile.in generated by automake 1.10.2 from Makefile.am.
# doc/Makefile. Generated from Makefile.in by configure.
# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
@@ -88,14 +88,14 @@
PACKAGE = memcached
PACKAGE_BUGREPORT = brad@danga.com
PACKAGE_NAME = memcached
-PACKAGE_STRING = memcached 1.2.6
+PACKAGE_STRING = memcached 1.2.8
PACKAGE_TARNAME = memcached
-PACKAGE_VERSION = 1.2.6
+PACKAGE_VERSION = 1.2.8
PATH_SEPARATOR = :
SET_MAKE =
SHELL = /bin/sh
STRIP =
-VERSION = 1.2.6
+VERSION = 1.2.8
abs_builddir = /home/dormando/p/danga/memcached_new/doc
abs_srcdir = /home/dormando/p/danga/memcached_new/doc
abs_top_builddir = /home/dormando/p/danga/memcached_new
@@ -147,6 +147,7 @@
target_cpu = x86_64
target_os = linux-gnu
target_vendor = unknown
+top_build_prefix = ../
top_builddir = ..
top_srcdir = ..
man_MANS = memcached.1
@@ -158,8 +159,8 @@
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
- cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh \
- && exit 0; \
+ ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \
+ && { if test -f $@; then exit 0; else break; fi; }; \
exit 1;; \
esac; \
done; \
@@ -194,8 +195,8 @@
esac; \
done; \
for i in $$list; do \
- if test -f $(srcdir)/$$i; then file=$(srcdir)/$$i; \
- else file=$$i; fi; \
+ if test -f $$i; then file=$$i; \
+ else file=$(srcdir)/$$i; fi; \
ext=`echo $$i | sed -e 's/^.*\\.//'`; \
case "$$ext" in \
1*) ;; \
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/doc/memcached.1 new/memcached-1.2.8/doc/memcached.1
--- old/memcached-1.2.6/doc/memcached.1 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/doc/memcached.1 2009-04-11 05:48:08.000000000 +0200
@@ -52,7 +52,7 @@
Listen on TCP port <num>, the default is port 11211.
.TP
.B \-U <num>
-Listen on UDP port <num>, the default is 0, off.
+Listen on UDP port <num>, the default is port 11211, 0 is off.
.TP
.B \-M
Disable automatic removal of items from the cache when out of memory.
@@ -61,9 +61,6 @@
.B \-r
Raise the core file size limit to the maximum allowable.
.TP
-.B \-b
-Run a managed instanced (mnemonic: buckets)\n".
-.TP
.B \-f <factor>
Use <factor> as the multiplier for computing the sizes of memory chunks that
items are stored in. A lower value may result in less wasted memory depending
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/doc/protocol.txt new/memcached-1.2.8/doc/protocol.txt
--- old/memcached-1.2.6/doc/protocol.txt 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/doc/protocol.txt 2009-04-11 05:48:08.000000000 +0200
@@ -391,14 +391,15 @@
the server started running
connection_structures 32u Number of connection structures allocated
by the server
+cmd_flush 64u Cumulative number of flush requests
cmd_get 64u Cumulative number of retrieval requests
cmd_set 64u Cumulative number of storage requests
get_hits 64u Number of keys that have been requested and
found present
get_misses 64u Number of items that have been requested
and not found
-evictions 64u Number of valid items removed from cache
- to free memory for new items
+evictions 64u Number of valid items removed from cache
+ to free memory for new items
bytes_read 64u Total number of bytes read by this server
from network
bytes_written 64u Total number of bytes sent by this server to
@@ -407,6 +408,10 @@
use for storage.
threads 32u Number of worker threads requested.
(see doc/threads.txt)
+accepting_conns 32u Whether or not new connections are being
+ accepted.
+listen_disabled_num 64u Number of times we stopped listening for
+ new connections.
Item statistics
@@ -436,9 +441,15 @@
age Age of the oldest item in the LRU.
evicted Number of times an item had to be evicted from the LRU
before it expired.
+evicted_time Seconds since the last access for the most recent item
+ evicted from this class. Use this to judge how
+ recently active your evicted data is.
outofmemory Number of times the underlying slab class was unable to
store a new item. This means you are running with -M or
an eviction failed.
+tailrepairs Number of times we self-healed a slab with a refcount
+ leak. If this counter is increasing a lot, please
+ report your situation to the developers.
Note this will only display information about slabs which exist, so an empty
cache will return an empty set.
@@ -494,7 +505,7 @@
chunk_size The amount of space each chunk uses. One item will use
one chunk of the appropriate size.
chunks_per_page How many chunks exist within one page. A page by
- default is one megabyte in size. Slabs are allocated per
+ default is one megabyte in size. Slabs are allocated per
page, then broken into chunks.
total_pages Total number of pages allocated to the slab class.
total_chunks Total number of chunks allocated to the slab class.
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/items.c new/memcached-1.2.8/items.c
--- old/memcached-1.2.6/items.c 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/items.c 2009-04-11 05:48:08.000000000 +0200
@@ -17,7 +17,6 @@
/* Forward Declarations */
static void item_link_q(item *it);
static void item_unlink_q(item *it);
-static uint64_t get_cas_id();
/*
* We only reposition items in the LRU queue if they haven't been repositioned
@@ -29,7 +28,9 @@
#define LARGEST_ID 255
typedef struct {
unsigned int evicted;
+ rel_time_t evicted_time;
unsigned int outofmemory;
+ unsigned int tailrepairs;
} itemstats_t;
static item *heads[LARGEST_ID];
@@ -125,6 +126,7 @@
if (search->refcount == 0) {
if (search->exptime == 0 || search->exptime > current_time) {
itemstats[id].evicted++;
+ itemstats[id].evicted_time = current_time - search->time;
STATS_LOCK();
stats.evictions++;
STATS_UNLOCK();
@@ -136,7 +138,26 @@
it = slabs_alloc(ntotal, id);
if (it == 0) {
itemstats[id].outofmemory++;
- return NULL;
+ /* Last ditch effort. There is a very rare bug which causes
+ * refcount leaks. We've fixed most of them, but it still happens,
+ * and it may happen in the future.
+ * We can reasonably assume no item can stay locked for more than
+ * three hours, so if we find one in the tail which is that old,
+ * free it anyway.
+ */
+ tries = 50;
+ for (search = tails[id]; tries > 0 && search != NULL; tries--, search=search->prev) {
+ if (search->refcount != 0 && search->time + 10800 < current_time) {
+ itemstats[id].tailrepairs++;
+ search->refcount = 0;
+ do_item_unlink(search);
+ break;
+ }
+ }
+ it = slabs_alloc(ntotal, id);
+ if (it == 0) {
+ return NULL;
+ }
}
}
@@ -334,7 +355,7 @@
}
char *do_item_stats(int *bytes) {
- size_t bufleft = (size_t) LARGEST_ID * 160;
+ size_t bufleft = (size_t) LARGEST_ID * 360;
char *buffer = malloc(bufleft);
char *bufcurr = buffer;
rel_time_t now = current_time;
@@ -351,9 +372,14 @@
"STAT items:%d:number %u\r\n"
"STAT items:%d:age %u\r\n"
"STAT items:%d:evicted %u\r\n"
- "STAT items:%d:outofmemory %u\r\n",
- i, sizes[i], i, now - tails[i]->time, i,
- itemstats[i].evicted, i, itemstats[i].outofmemory);
+ "STAT items:%d:evicted_time %u\r\n"
+ "STAT items:%d:outofmemory %u\r\n"
+ "STAT items:%d:tailrepairs %u\r\n",
+ i, sizes[i], i, now - tails[i]->time,
+ i, itemstats[i].evicted,
+ i, itemstats[i].evicted_time,
+ i, itemstats[i].outofmemory,
+ i, itemstats[i].tailrepairs);
if (linelen + sizeof("END\r\n") < bufleft) {
bufcurr += linelen;
bufleft -= linelen;
@@ -420,7 +446,18 @@
/** wrapper around assoc_find which does the lazy expiration/deletion logic */
item *do_item_get_notedeleted(const char *key, const size_t nkey, bool *delete_locked) {
item *it = assoc_find(key, nkey);
+ int was_found = 0;
if (delete_locked) *delete_locked = false;
+
+ if (settings.verbose > 2) {
+ if (it == NULL) {
+ fprintf(stderr, "> NOT FOUND %s", key);
+ } else {
+ fprintf(stderr, "> FOUND KEY %s", ITEM_key(it));
+ was_found++;
+ }
+ }
+
if (it != NULL && (it->it_flags & ITEM_DELETED)) {
/* it's flagged as delete-locked. let's see if that condition
is past due, and the 5-second delete_timer just hasn't
@@ -430,20 +467,41 @@
it = NULL;
}
}
+
+ if (it == NULL && was_found) {
+ fprintf(stderr, " -nuked by delete lock");
+ was_found--;
+ }
+
if (it != NULL && settings.oldest_live != 0 && settings.oldest_live <= current_time &&
it->time <= settings.oldest_live) {
do_item_unlink(it); /* MTSAFE - cache_lock held */
it = NULL;
}
+
+ if (it == NULL && was_found) {
+ fprintf(stderr, " -nuked by flush");
+ was_found--;
+ }
+
if (it != NULL && it->exptime != 0 && it->exptime <= current_time) {
do_item_unlink(it); /* MTSAFE - cache_lock held */
it = NULL;
}
+ if (it == NULL && was_found) {
+ fprintf(stderr, " -nuked by expire");
+ was_found--;
+ }
+
if (it != NULL) {
it->refcount++;
DEBUG_REFCNT(it, '+');
}
+
+ if (settings.verbose > 2)
+ fprintf(stderr, "\n");
+
return it;
}
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/items.h new/memcached-1.2.8/items.h
--- old/memcached-1.2.6/items.h 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/items.h 2009-04-11 05:48:08.000000000 +0200
@@ -1,5 +1,7 @@
/* See items.c */
void item_init(void);
+uint64_t get_cas_id(void);
+
/*@null@*/
item *do_item_alloc(char *key, const size_t nkey, const int flags, const rel_time_t exptime, const int nbytes);
void item_free(item *it);
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/memcached.c new/memcached-1.2.8/memcached.c
--- old/memcached-1.2.6/memcached.c 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/memcached.c 2009-04-11 08:12:29.000000000 +0200
@@ -80,7 +80,6 @@
static void event_handler(const int fd, const short which, void *arg);
static void conn_close(conn *c);
static void conn_init(void);
-static void accept_new_conns(const bool do_accept);
static bool update_event(conn *c, const int new_flags);
static void complete_nread(conn *c);
static void process_command(conn *c, char *command);
@@ -112,8 +111,6 @@
#define TRANSMIT_SOFT_ERROR 2
#define TRANSMIT_HARD_ERROR 3
-static int *buckets = 0; /* bucket->generation array for a managed instance */
-
#define REALTIME_MAXDELTA 60*60*24*30
/*
* given time value that's either unix time or delta from current unix time, return
@@ -143,7 +140,9 @@
static void stats_init(void) {
stats.curr_items = stats.total_items = stats.curr_conns = stats.total_conns = stats.conn_structs = 0;
stats.get_cmds = stats.set_cmds = stats.get_hits = stats.get_misses = stats.evictions = 0;
- stats.curr_bytes = stats.bytes_read = stats.bytes_written = 0;
+ stats.curr_bytes = stats.bytes_read = stats.bytes_written = stats.flush_cmds = 0;
+ stats.listen_disabled_num = 0;
+ stats.accepting_conns = 1; /* assuming we start in this state. */
/* make the time we started always be 2 seconds before we really
did, so time(0) - time.started is never zero. if so, things
@@ -158,6 +157,7 @@
stats.total_items = stats.total_conns = 0;
stats.get_cmds = stats.set_cmds = stats.get_hits = stats.get_misses = stats.evictions = 0;
stats.bytes_read = stats.bytes_written = 0;
+ stats.flush_cmds = stats.listen_disabled_num = 0;
stats_prefix_clear();
STATS_UNLOCK();
}
@@ -165,7 +165,7 @@
static void settings_init(void) {
settings.access=0700;
settings.port = 11211;
- settings.udpport = 0;
+ settings.udpport = 11211;
/* By default this string should be NULL for getaddrinfo() */
settings.inter = NULL;
settings.maxbytes = 64 * 1024 * 1024; /* default is 64MB */
@@ -174,7 +174,6 @@
settings.oldest_live = 0;
settings.evict_to_free = 1; /* push old items out of cache when memory runs out */
settings.socketpath = NULL; /* by default, not using a unix socket */
- settings.managed = false;
settings.factor = 1.25;
settings.chunk_size = 48; /* space for a modest key and value */
#ifdef USE_THREADS
@@ -182,8 +181,11 @@
#else
settings.num_threads = 1;
#endif
+ settings.num_threads++; /* N workers + 1 dispatcher */
settings.prefix_delimiter = ':';
settings.detail_enabled = 0;
+ settings.reqs_per_event = 20;
+ settings.backlog = 1024;
}
/* returns true if a deleted item's delete-locked-time is over, and it
@@ -365,8 +367,6 @@
c->write_and_go = conn_read;
c->write_and_free = 0;
c->item = 0;
- c->bucket = -1;
- c->gen = 0;
c->noreply = false;
@@ -1062,7 +1062,7 @@
command = tokens[COMMAND_TOKEN].value;
if (ntokens == 2 && strcmp(command, "stats") == 0) {
- char temp[1024];
+ char temp[2048];
pid_t pid = getpid();
char *pos = temp;
@@ -1087,6 +1087,7 @@
pos += sprintf(pos, "STAT curr_connections %u\r\n", stats.curr_conns - 1); /* ignore listening conn */
pos += sprintf(pos, "STAT total_connections %u\r\n", stats.total_conns);
pos += sprintf(pos, "STAT connection_structures %u\r\n", stats.conn_structs);
+ pos += sprintf(pos, "STAT cmd_flush %llu\r\n", stats.flush_cmds);
pos += sprintf(pos, "STAT cmd_get %llu\r\n", stats.get_cmds);
pos += sprintf(pos, "STAT cmd_set %llu\r\n", stats.set_cmds);
pos += sprintf(pos, "STAT get_hits %llu\r\n", stats.get_hits);
@@ -1096,6 +1097,8 @@
pos += sprintf(pos, "STAT bytes_written %llu\r\n", stats.bytes_written);
pos += sprintf(pos, "STAT limit_maxbytes %llu\r\n", (uint64_t) settings.maxbytes);
pos += sprintf(pos, "STAT threads %u\r\n", settings.num_threads);
+ pos += sprintf(pos, "STAT accepting_conns %u\r\n", stats.accepting_conns);
+ pos += sprintf(pos, "STAT listen_disabled_num %llu\r\n", stats.listen_disabled_num);
pos += sprintf(pos, "END");
STATS_UNLOCK();
out_string(c, temp);
@@ -1134,43 +1137,6 @@
#endif /* HAVE_STRUCT_MALLINFO */
#endif /* HAVE_MALLOC_H */
-#if !defined(WIN32) || !defined(__APPLE__)
- if (strcmp(subcommand, "maps") == 0) {
- char *wbuf;
- int wsize = 8192; /* should be enough */
- int fd;
- int res;
-
- if ((wbuf = (char *)malloc(wsize)) == NULL) {
- out_string(c, "SERVER_ERROR out of memory writing stats maps");
- return;
- }
-
- fd = open("/proc/self/maps", O_RDONLY);
- if (fd == -1) {
- out_string(c, "SERVER_ERROR cannot open the maps file");
- free(wbuf);
- return;
- }
-
- res = read(fd, wbuf, wsize - 6); /* 6 = END\r\n\0 */
- if (res == wsize - 6) {
- out_string(c, "SERVER_ERROR buffer overflow");
- free(wbuf); close(fd);
- return;
- }
- if (res == 0 || res == -1) {
- out_string(c, "SERVER_ERROR can't read the maps file");
- free(wbuf); close(fd);
- return;
- }
- memcpy(wbuf + res, "END\r\n", 5);
- write_and_free(c, wbuf, res + 5);
- close(fd);
- return;
- }
-#endif
-
if (strcmp(subcommand, "cachedump") == 0) {
char *buf;
@@ -1239,19 +1205,6 @@
int stats_get_misses = 0;
assert(c != NULL);
- if (settings.managed) {
- int bucket = c->bucket;
- if (bucket == -1) {
- out_string(c, "CLIENT_ERROR no BG data in managed mode");
- return;
- }
- c->bucket = -1;
- if (buckets[bucket] != c->gen) {
- out_string(c, "ERROR_NOT_OWNER");
- return;
- }
- }
-
do {
while(key_token->length != 0) {
@@ -1453,19 +1406,6 @@
stats_prefix_record_set(key);
}
- if (settings.managed) {
- int bucket = c->bucket;
- if (bucket == -1) {
- out_string(c, "CLIENT_ERROR no BG data in managed mode");
- return;
- }
- c->bucket = -1;
- if (buckets[bucket] != c->gen) {
- out_string(c, "ERROR_NOT_OWNER");
- return;
- }
- }
-
it = item_alloc(key, nkey, flags, realtime(exptime), vlen+2);
if (it == 0) {
@@ -1518,19 +1458,6 @@
key = tokens[KEY_TOKEN].value;
nkey = tokens[KEY_TOKEN].length;
- if (settings.managed) {
- int bucket = c->bucket;
- if (bucket == -1) {
- out_string(c, "CLIENT_ERROR no BG data in managed mode");
- return;
- }
- c->bucket = -1;
- if (buckets[bucket] != c->gen) {
- out_string(c, "ERROR_NOT_OWNER");
- return;
- }
- }
-
delta = strtoll(tokens[2].value, NULL, 10);
if(errno == ERANGE) {
@@ -1561,7 +1488,7 @@
*/
char *do_add_delta(conn *c, item *it, const bool incr, const int64_t delta, char *buf) {
char *ptr;
- int64_t value;
+ uint64_t value;
int res;
ptr = ITEM_data(it);
@@ -1594,6 +1521,10 @@
do_item_replace(it, new_it);
do_item_remove(new_it); /* release our reference */
} else { /* replace in-place */
+ /* When changing the value without replacing the item, we
+ need to update the CAS on the existing item. */
+ it->cas_id = get_cas_id();
+
memcpy(ITEM_data(it), buf, res);
memset(ITEM_data(it) + res, ' ', it->nbytes - res - 2);
}
@@ -1611,19 +1542,6 @@
set_noreply_maybe(c, tokens, ntokens);
- if (settings.managed) {
- int bucket = c->bucket;
- if (bucket == -1) {
- out_string(c, "CLIENT_ERROR no BG data in managed mode");
- return;
- }
- c->bucket = -1;
- if (buckets[bucket] != c->gen) {
- out_string(c, "ERROR_NOT_OWNER");
- return;
- }
- }
-
key = tokens[KEY_TOKEN].value;
nkey = tokens[KEY_TOKEN].length;
@@ -1766,67 +1684,6 @@
process_delete_command(c, tokens, ntokens);
- } else if (ntokens == 3 && strcmp(tokens[COMMAND_TOKEN].value, "own") == 0) {
- unsigned int bucket, gen;
- if (!settings.managed) {
- out_string(c, "CLIENT_ERROR not a managed instance");
- return;
- }
-
- if (sscanf(tokens[1].value, "%u:%u", &bucket,&gen) == 2) {
- if ((bucket < 0) || (bucket >= MAX_BUCKETS)) {
- out_string(c, "CLIENT_ERROR bucket number out of range");
- return;
- }
- buckets[bucket] = gen;
- out_string(c, "OWNED");
- return;
- } else {
- out_string(c, "CLIENT_ERROR bad format");
- return;
- }
-
- } else if (ntokens == 3 && (strcmp(tokens[COMMAND_TOKEN].value, "disown")) == 0) {
-
- int bucket;
- if (!settings.managed) {
- out_string(c, "CLIENT_ERROR not a managed instance");
- return;
- }
- if (sscanf(tokens[1].value, "%u", &bucket) == 1) {
- if ((bucket < 0) || (bucket >= MAX_BUCKETS)) {
- out_string(c, "CLIENT_ERROR bucket number out of range");
- return;
- }
- buckets[bucket] = 0;
- out_string(c, "DISOWNED");
- return;
- } else {
- out_string(c, "CLIENT_ERROR bad format");
- return;
- }
-
- } else if (ntokens == 3 && (strcmp(tokens[COMMAND_TOKEN].value, "bg")) == 0) {
- int bucket, gen;
- if (!settings.managed) {
- out_string(c, "CLIENT_ERROR not a managed instance");
- return;
- }
- if (sscanf(tokens[1].value, "%u:%u", &bucket, &gen) == 2) {
- /* we never write anything back, even if input's wrong */
- if ((bucket < 0) || (bucket >= MAX_BUCKETS) || (gen <= 0)) {
- /* do nothing, bad input */
- } else {
- c->bucket = bucket;
- c->gen = gen;
- }
- conn_set_state(c, conn_read);
- return;
- } else {
- out_string(c, "CLIENT_ERROR bad format");
- return;
- }
-
} else if (ntokens >= 2 && (strcmp(tokens[COMMAND_TOKEN].value, "stats") == 0)) {
process_stat(c, tokens, ntokens);
@@ -1837,6 +1694,10 @@
set_noreply_maybe(c, tokens, ntokens);
+ STATS_LOCK();
+ stats.flush_cmds++;
+ STATS_UNLOCK();
+
if(ntokens == (c->noreply ? 3 : 2)) {
settings.oldest_live = current_time - 1;
item_flush_expired();
@@ -2069,16 +1930,13 @@
/*
* Sets whether we are listening for new connections or not.
*/
-void accept_new_conns(const bool do_accept) {
+void do_accept_new_conns(const bool do_accept) {
conn *next;
- if (! is_listen_thread())
- return;
-
for (next = listen_conn; next; next = next->next) {
if (do_accept) {
update_event(next, EV_READ | EV_PERSIST);
- if (listen(next->sfd, 1024) != 0) {
+ if (listen(next->sfd, settings.backlog) != 0) {
perror("listen");
}
}
@@ -2088,7 +1946,18 @@
perror("listen");
}
}
- }
+ }
+
+ if (do_accept) {
+ STATS_LOCK();
+ stats.accepting_conns = 1;
+ STATS_UNLOCK();
+ } else {
+ STATS_LOCK();
+ stats.accepting_conns = 0;
+ stats.listen_disabled_num++;
+ STATS_UNLOCK();
+ }
}
@@ -2164,6 +2033,7 @@
int sfd, flags = 1;
socklen_t addrlen;
struct sockaddr_storage addr;
+ int nreqs = settings.reqs_per_event;
int res;
assert(c != NULL);
@@ -2202,7 +2072,9 @@
if (try_read_command(c) != 0) {
continue;
}
- if ((c->udp ? try_read_udp(c) : try_read_network(c)) != 0) {
+ /* Only process nreqs at a time to avoid starving other
+ connections */
+ if (--nreqs && (c->udp ? try_read_udp(c) : try_read_network(c)) != 0) {
continue;
}
/* we have no command line and no data to read from network */
@@ -2409,7 +2281,6 @@
int flags;
if ((sfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol)) == -1) {
- perror("socket()");
return -1;
}
@@ -2474,43 +2345,61 @@
* that otherwise mess things up.
*/
memset(&hints, 0, sizeof (hints));
- hints.ai_flags = AI_PASSIVE|AI_ADDRCONFIG;
+ hints.ai_flags = AI_PASSIVE;
+ hints.ai_family = AF_UNSPEC;
if (is_udp)
{
- hints.ai_protocol = IPPROTO_UDP;
hints.ai_socktype = SOCK_DGRAM;
- hints.ai_family = AF_INET; /* This left here because of issues with OSX 10.5 */
} else {
- hints.ai_family = AF_UNSPEC;
- hints.ai_protocol = IPPROTO_TCP;
hints.ai_socktype = SOCK_STREAM;
}
snprintf(port_buf, NI_MAXSERV, "%d", port);
error= getaddrinfo(settings.inter, port_buf, &hints, &ai);
if (error != 0) {
- if (error != EAI_SYSTEM)
- fprintf(stderr, "getaddrinfo(): %s\n", gai_strerror(error));
- else
- perror("getaddrinfo()");
+ if (error != EAI_SYSTEM)
+ fprintf(stderr, "getaddrinfo(): %s\n", gai_strerror(error));
+ else
+ perror("getaddrinfo()");
- return 1;
+ return 1;
}
for (next= ai; next; next= next->ai_next) {
conn *listen_conn_add;
if ((sfd = new_socket(next)) == -1) {
- freeaddrinfo(ai);
- return 1;
+ /* getaddrinfo can return "junk" addresses,
+ * we make sure at least one works before erroring.
+ */
+ continue;
}
+#ifdef IPV6_V6ONLY
+ if (next->ai_family == AF_INET6) {
+ error = setsockopt(sfd, IPPROTO_IPV6, IPV6_V6ONLY, (char *) &flags, sizeof(flags));
+ if (error != 0) {
+ perror("setsockopt");
+ close(sfd);
+ continue;
+ }
+ }
+#endif
+
setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR, (void *)&flags, sizeof(flags));
if (is_udp) {
maximize_sndbuf(sfd);
} else {
- setsockopt(sfd, SOL_SOCKET, SO_KEEPALIVE, (void *)&flags, sizeof(flags));
- setsockopt(sfd, SOL_SOCKET, SO_LINGER, (void *)&ling, sizeof(ling));
- setsockopt(sfd, IPPROTO_TCP, TCP_NODELAY, (void *)&flags, sizeof(flags));
+ error = setsockopt(sfd, SOL_SOCKET, SO_KEEPALIVE, (void *)&flags, sizeof(flags));
+ if (error != 0)
+ perror("setsockopt");
+
+ error = setsockopt(sfd, SOL_SOCKET, SO_LINGER, (void *)&ling, sizeof(ling));
+ if (error != 0)
+ perror("setsockopt");
+
+ error = setsockopt(sfd, IPPROTO_TCP, TCP_NODELAY, (void *)&flags, sizeof(flags));
+ if (error != 0)
+ perror("setsockopt");
}
if (bind(sfd, next->ai_addr, next->ai_addrlen) == -1) {
@@ -2523,34 +2412,34 @@
close(sfd);
continue;
} else {
- success++;
- if (!is_udp && listen(sfd, 1024) == -1) {
- perror("listen()");
- close(sfd);
- freeaddrinfo(ai);
- return 1;
- }
- }
+ success++;
+ if (!is_udp && listen(sfd, settings.backlog) == -1) {
+ perror("listen()");
+ close(sfd);
+ freeaddrinfo(ai);
+ return 1;
+ }
+ }
- if (is_udp)
- {
- int c;
+ if (is_udp)
+ {
+ int c;
- for (c = 0; c < settings.num_threads; c++) {
- /* this is guaranteed to hit all threads because we round-robin */
- dispatch_conn_new(sfd, conn_read, EV_READ | EV_PERSIST,
- UDP_READ_BUFFER_SIZE, 1);
- }
- } else {
- if (!(listen_conn_add = conn_new(sfd, conn_listening,
- EV_READ | EV_PERSIST, 1, false, main_base))) {
- fprintf(stderr, "failed to create listening connection\n");
- exit(EXIT_FAILURE);
- }
+ for (c = 1; c < settings.num_threads; c++) {
+ /* this is guaranteed to hit all threads because we round-robin */
+ dispatch_conn_new(sfd, conn_read, EV_READ | EV_PERSIST,
+ UDP_READ_BUFFER_SIZE, is_udp);
+ }
+ } else {
+ if (!(listen_conn_add = conn_new(sfd, conn_listening,
+ EV_READ | EV_PERSIST, 1, false, main_base))) {
+ fprintf(stderr, "failed to create listening connection\n");
+ exit(EXIT_FAILURE);
+ }
- listen_conn_add->next = listen_conn;
- listen_conn = listen_conn_add;
- }
+ listen_conn_add->next = listen_conn;
+ listen_conn = listen_conn_add;
+ }
}
freeaddrinfo(ai);
@@ -2621,7 +2510,7 @@
return 1;
}
umask(old_umask);
- if (listen(sfd, 1024) == -1) {
+ if (listen(sfd, settings.backlog) == -1) {
perror("listen()");
close(sfd);
return 1;
@@ -2714,7 +2603,7 @@
static void usage(void) {
printf(PACKAGE " " VERSION "\n");
printf("-p <num> TCP port number to listen on (default: 11211)\n"
- "-U <num> UDP port number to listen on (default: 0, off)\n"
+ "-U <num> UDP port number to listen on (default: 11211, 0 is off)\n"
"-s <file> unix socket path to listen on (disables network support)\n"
"-a <mask> access mask for unix socket, in octal (default 0700)\n"
"-l interface to listen on, default is INDRR_ANY\n"
@@ -2734,7 +2623,6 @@
"-vv very verbose (also print client commands/reponses)\n"
"-h print this help and exit\n"
"-i print memcached and libevent license\n"
- "-b run a managed instanced (mnemonic: buckets)\n"
"-P <file> save PID in <file>, only used with -d option\n"
"-f <factor> chunk size growth factor, default 1.25\n"
"-n <bytes> minimum space allocated for key+value+flags, default 48\n"
@@ -2751,6 +2639,10 @@
#ifdef USE_THREADS
printf("-t <num> number of threads to use, default 4\n");
#endif
+ printf("-R Maximum number of requests per event\n"
+ " limits the number of requests process for a given con nection\n"
+ " to prevent starvation. default 20\n");
+ printf("-b Set the backlog queue limit (default 1024)\n");
return;
}
@@ -2928,7 +2820,7 @@
setbuf(stderr, NULL);
/* process arguments */
- while ((c = getopt(argc, argv, "a:bp:s:U:m:Mc:khirvdl:u:P:f:s:n:t:D:L")) != -1) {
+ while ((c = getopt(argc, argv, "a:p:s:U:m:Mc:khirvdl:u:P:f:s:n:t:D:LR:b:")) != -1) {
switch (c) {
case 'a':
/* access for unix domain socket, as octal mask (like chmod)*/
@@ -2938,9 +2830,6 @@
case 'U':
settings.udpport = atoi(optarg);
break;
- case 'b':
- settings.managed = true;
- break;
case 'p':
settings.port = atoi(optarg);
break;
@@ -2977,6 +2866,13 @@
case 'r':
maxcore = 1;
break;
+ case 'R':
+ settings.reqs_per_event = atoi(optarg);
+ if (settings.reqs_per_event == 0) {
+ fprintf(stderr, "Number of requests per event must be greater than 0\n");
+ return 1;
+ }
+ break;
case 'u':
username = optarg;
break;
@@ -2998,8 +2894,8 @@
}
break;
case 't':
- settings.num_threads = atoi(optarg);
- if (settings.num_threads == 0) {
+ settings.num_threads = atoi(optarg) + 1; /* Extra dispatch thread */
+ if (settings.num_threads < 2) {
fprintf(stderr, "Number of threads must be greater than 0\n");
return 1;
}
@@ -3019,6 +2915,9 @@
}
break;
#endif
+ case 'b' :
+ settings.backlog = atoi(optarg);
+ break;
default:
fprintf(stderr, "Illegal argument \"%c\"\n", c);
return 1;
@@ -3123,16 +3022,6 @@
suffix_init();
slabs_init(settings.maxbytes, settings.factor, preallocate);
- /* managed instance? alloc and zero a bucket array */
- if (settings.managed) {
- buckets = malloc(sizeof(int) * MAX_BUCKETS);
- if (buckets == 0) {
- fprintf(stderr, "failed to allocate the bucket array");
- exit(EXIT_FAILURE);
- }
- memset(buckets, 0, sizeof(int) * MAX_BUCKETS);
- }
-
/*
* ignore SIGPIPE signals; we can use errno==EPIPE if we
* need that information
@@ -3163,18 +3052,22 @@
/* create unix mode sockets after dropping privileges */
if (settings.socketpath != NULL) {
+ errno = 0;
if (server_socket_unix(settings.socketpath,settings.access)) {
- fprintf(stderr, "failed to listen\n");
+ fprintf(stderr, "failed to listen on UNIX socket: %s\n", settings.socketpath);
+ if (errno != 0)
+ perror("socket listen");
exit(EXIT_FAILURE);
}
}
/* create the listening socket, bind it, and init */
if (settings.socketpath == NULL) {
- int udp_port;
-
- if (server_socket(settings.port, 0)) {
- fprintf(stderr, "failed to listen\n");
+ errno = 0;
+ if (settings.port && server_socket(settings.port, 0)) {
+ fprintf(stderr, "failed to listen on TCP port %d\n", settings.port);
+ if (errno != 0)
+ perror("tcp listen");
exit(EXIT_FAILURE);
}
/*
@@ -3183,11 +3076,13 @@
* then daemonise if needed, then init libevent (in some cases
* descriptors created by libevent wouldn't survive forking).
*/
- udp_port = settings.udpport ? settings.udpport : settings.port;
/* create the UDP listening socket and bind it */
- if (server_socket(udp_port, 1)) {
+ errno = 0;
+ if (settings.udpport && server_socket(settings.udpport, 1)) {
fprintf(stderr, "failed to listen on UDP port %d\n", settings.udpport);
+ if (errno != 0)
+ perror("udp listen");
exit(EXIT_FAILURE);
}
}
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/memcached.h new/memcached-1.2.8/memcached.h
--- old/memcached-1.2.6/memcached.h 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/memcached.h 2009-04-11 07:56:09.000000000 +0200
@@ -71,10 +71,13 @@
uint64_t set_cmds;
uint64_t get_hits;
uint64_t get_misses;
+ uint64_t flush_cmds;
uint64_t evictions;
time_t started; /* when the process was started */
uint64_t bytes_read;
uint64_t bytes_written;
+ unsigned int accepting_conns; /* whether we are currently accepting */
+ uint64_t listen_disabled_num;
};
#define MAX_VERBOSITY_LEVEL 2
@@ -87,7 +90,6 @@
char *inter;
int verbose;
rel_time_t oldest_live; /* ignore existing items older than this */
- bool managed; /* if 1, a tracker manages virtual buckets */
int evict_to_free;
char *socketpath; /* path to unix socket if using local socket */
int access; /* access mask (a la chmod) for unix domain socket */
@@ -96,6 +98,9 @@
int num_threads; /* number of libevent threads to run */
char prefix_delimiter; /* character that marks a key prefix (for stats) */
int detail_enabled; /* nonzero if we're collecting detailed stats */
+ int reqs_per_event; /* Maximum number of io to process on each
+ io-event. */
+ int backlog;
};
extern struct stats stats;
@@ -217,16 +222,10 @@
int hdrsize; /* number of headers' worth of space is allocated */
int binary; /* are we in binary mode */
- int bucket; /* bucket number for the next command, if running as
- a managed instance. -1 (_not_ 0) means invalid. */
- int gen; /* generation requested for the bucket */
bool noreply; /* True if the reply should not be sent. */
conn *next; /* Used for generating a list of conn structures */
};
-/* number of virtual buckets for a managed instance */
-#define MAX_BUCKETS 32768
-
/* current time of day (updated periodically) */
extern volatile rel_time_t current_time;
@@ -234,6 +233,7 @@
* Functions
*/
+void do_accept_new_conns(const bool do_accept);
conn *do_conn_from_freelist();
bool do_conn_add_to_freelist(conn *c);
char *do_suffix_from_freelist();
@@ -275,6 +275,7 @@
char *mt_add_delta(conn *c, item *item, const int incr, const int64_t delta,
char *buf);
void mt_assoc_move_next_bucket(void);
+void mt_accept_new_conns(const bool do_accept);
conn *mt_conn_from_freelist(void);
bool mt_conn_add_to_freelist(conn *c);
char *mt_suffix_from_freelist(void);
@@ -304,6 +305,7 @@
# define add_delta(c,x,y,z,a) mt_add_delta(c,x,y,z,a)
# define assoc_move_next_bucket() mt_assoc_move_next_bucket()
+# define accept_new_conns(x) mt_accept_new_conns(x)
# define conn_from_freelist() mt_conn_from_freelist()
# define conn_add_to_freelist(x) mt_conn_add_to_freelist(x)
# define suffix_from_freelist() mt_suffix_from_freelist()
@@ -335,6 +337,7 @@
# define add_delta(c,x,y,z,a) do_add_delta(c,x,y,z,a)
# define assoc_move_next_bucket() do_assoc_move_next_bucket()
+# define accept_new_conns(x) do_accept_new_conns(x)
# define conn_from_freelist() do_conn_from_freelist()
# define conn_add_to_freelist(x) do_conn_add_to_freelist(x)
# define suffix_from_freelist() do_suffix_from_freelist()
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/memcached.spec new/memcached-1.2.8/memcached.spec
--- old/memcached-1.2.6/memcached.spec 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/memcached.spec 2009-04-11 08:23:21.000000000 +0200
@@ -1,5 +1,5 @@
Name: memcached
-Version: 1.2.6
+Version: 1.2.8
Release: 1%{?dist}
Summary: High Performance, Distributed Memory Object Cache
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/slabs.c new/memcached-1.2.8/slabs.c
--- old/memcached-1.2.6/slabs.c 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/slabs.c 2009-04-11 05:48:08.000000000 +0200
@@ -323,7 +323,7 @@
bufcurr += sprintf(bufcurr, "STAT %d:chunks_per_page %u\r\n", i, perslab);
bufcurr += sprintf(bufcurr, "STAT %d:total_pages %u\r\n", i, slabs);
bufcurr += sprintf(bufcurr, "STAT %d:total_chunks %u\r\n", i, slabs*perslab);
- bufcurr += sprintf(bufcurr, "STAT %d:used_chunks %u\r\n", i, slabs*perslab - p->sl_curr);
+ bufcurr += sprintf(bufcurr, "STAT %d:used_chunks %u\r\n", i, slabs*perslab - p->sl_curr - p->end_page_free);
bufcurr += sprintf(bufcurr, "STAT %d:free_chunks %u\r\n", i, p->sl_curr);
bufcurr += sprintf(bufcurr, "STAT %d:free_chunks_end %u\r\n", i, p->end_page_free);
total++;
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/00-startup.t new/memcached-1.2.8/t/00-startup.t
--- old/memcached-1.2.6/t/00-startup.t 2008-04-26 23:58:33.000000000 +0200
+++ new/memcached-1.2.8/t/00-startup.t 2009-04-11 05:48:08.000000000 +0200
@@ -1,14 +1,16 @@
#!/usr/bin/perl
use strict;
-use Test::More tests => 3;
+use Test::More tests => 4;
use FindBin qw($Bin);
use lib "$Bin/lib";
use MemcachedTest;
-my $server = new_memcached();
-
-ok($server, "started the server");
+eval {
+ my $server = new_memcached();
+ ok($server, "started the server");
+};
+is($@, '', 'Basic startup works');
eval {
my $server = new_memcached("-l fooble");
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/cas.t new/memcached-1.2.8/t/cas.t
--- old/memcached-1.2.6/t/cas.t 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/t/cas.t 2009-04-11 05:48:08.000000000 +0200
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-use Test::More tests => 30;
+use Test::More tests => 39;
use FindBin qw($Bin);
use lib "$Bin/lib";
use MemcachedTest;
@@ -115,3 +115,28 @@
( $res1 eq "EXISTS\r\n" && $res2 eq "STORED\r\n"),
"cas on same item from two sockets");
+### bug 15: http://code.google.com/p/memcached/issues/detail?id=15
+
+# set foo
+print $sock "set bug15 0 0 1\r\n0\r\n";
+is(scalar <$sock>, "STORED\r\n", "stored 0");
+
+# Check out the first gets.
+print $sock "gets bug15\r\n";
+ok(scalar <$sock> =~ /VALUE bug15 0 1 (\d+)\r\n/, "gets bug15 regexp success");
+my $bug15_cas = $1;
+is(scalar <$sock>, "0\r\n", "gets bug15 data is 0");
+is(scalar <$sock>, "END\r\n","gets bug15 END");
+
+# Increment
+print $sock "incr bug15 1\r\n";
+is(scalar <$sock>, "1\r\n", "incr worked");
+
+# Validate a changed CAS
+print $sock "gets bug15\r\n";
+ok(scalar <$sock> =~ /VALUE bug15 0 1 (\d+)\r\n/, "gets bug15 regexp success");
+my $next_bug15_cas = $1;
+is(scalar <$sock>, "1\r\n", "gets bug15 data is 0");
+is(scalar <$sock>, "END\r\n","gets bug15 END");
+
+ok($bug15_cas != $next_bug15_cas, "CAS changed");
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/expirations.t new/memcached-1.2.8/t/expirations.t
--- old/memcached-1.2.6/t/expirations.t 2008-04-26 23:58:33.000000000 +0200
+++ new/memcached-1.2.8/t/expirations.t 2009-04-06 07:33:32.000000000 +0200
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-use Test::More tests => 10;
+use Test::More tests => 15;
use FindBin qw($Bin);
use lib "$Bin/lib";
use MemcachedTest;
@@ -52,3 +52,13 @@
is(scalar <$sock>, "STORED\r\n", "stored boo");
mem_get_is($sock, "boo", undef, "now expired");
+print $sock "add add 0 2 6\r\naddval\r\n";
+is(scalar <$sock>, "STORED\r\n", "stored add");
+mem_get_is($sock, "add", "addval");
+# second add fails
+print $sock "add add 0 2 7\r\naddval2\r\n";
+is(scalar <$sock>, "NOT_STORED\r\n", "add failure");
+sleep(2.3);
+print $sock "add add 0 2 7\r\naddval3\r\n";
+is(scalar <$sock>, "STORED\r\n", "stored add again");
+mem_get_is($sock, "add", "addval3");
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/incrdecr.t new/memcached-1.2.8/t/incrdecr.t
--- old/memcached-1.2.6/t/incrdecr.t 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/t/incrdecr.t 2009-04-11 05:48:08.000000000 +0200
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-use Test::More tests => 17;
+use Test::More tests => 21;
use FindBin qw($Bin);
use lib "$Bin/lib";
use MemcachedTest;
@@ -9,6 +9,16 @@
my $server = new_memcached();
my $sock = $server->sock;
+# Bug 21
+print $sock "set bug21 0 0 19\r\n9223372036854775807\r\n";
+is(scalar <$sock>, "STORED\r\n", "stored text");
+print $sock "incr bug21 1\r\n";
+is(scalar <$sock>, "9223372036854775808\r\n", "bug21 incr 1");
+print $sock "incr bug21 1\r\n";
+is(scalar <$sock>, "9223372036854775809\r\n", "bug21 incr 2");
+print $sock "decr bug21 1\r\n";
+is(scalar <$sock>, "9223372036854775808\r\n", "bug21 decr");
+
print $sock "set num 0 0 1\r\n1\r\n";
is(scalar <$sock>, "STORED\r\n", "stored num");
mem_get_is($sock, "num", 1, "stored 1");
@@ -52,3 +62,4 @@
is(scalar <$sock>, "STORED\r\n", "stored text");
print $sock "incr text 1\r\n";
is(scalar <$sock>, "1\r\n", "hi - 1 = 0");
+
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/issue_29.t new/memcached-1.2.8/t/issue_29.t
--- old/memcached-1.2.6/t/issue_29.t 1970-01-01 01:00:00.000000000 +0100
+++ new/memcached-1.2.8/t/issue_29.t 2009-04-06 07:33:32.000000000 +0200
@@ -0,0 +1,26 @@
+#!/usr/bin/perl
+
+use strict;
+use Test::More tests => 4;
+use FindBin qw($Bin);
+use lib "$Bin/lib";
+use MemcachedTest;
+
+my $server = new_memcached();
+my $sock = $server->sock;
+
+print $sock "set issue29 0 0 0\r\n\r\n";
+is (scalar <$sock>, "STORED\r\n", "stored issue29");
+
+my $first_stats = mem_stats($sock, "slabs");
+my $first_used = $first_stats->{"1:used_chunks"};
+
+is(1, $first_used, "Used one");
+
+print $sock "set issue29_b 0 0 0\r\n\r\n";
+is (scalar <$sock>, "STORED\r\n", "stored issue29_b");
+
+my $second_stats = mem_stats($sock, "slabs");
+my $second_used = $second_stats->{"1:used_chunks"};
+
+is(2, $second_used, "Used two")
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/lib/MemcachedTest.pm new/memcached-1.2.8/t/lib/MemcachedTest.pm
--- old/memcached-1.2.6/t/lib/MemcachedTest.pm 2008-04-28 02:37:56.000000000 +0200
+++ new/memcached-1.2.8/t/lib/MemcachedTest.pm 2009-04-11 05:48:08.000000000 +0200
@@ -26,9 +26,9 @@
my $stats = {};
while (<$sock>) {
last if /^(\.|END)/;
- /^STAT (\S+) (\d+)/;
+ /^(STAT|ITEM) (\S+)\s+([^\r\n]+)/;
#print " slabs: $_";
- $stats->{$1} = $2;
+ $stats->{$2} = $3;
}
return $stats;
}
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/line-lengths.t new/memcached-1.2.8/t/line-lengths.t
--- old/memcached-1.2.6/t/line-lengths.t 1970-01-01 01:00:00.000000000 +0100
+++ new/memcached-1.2.8/t/line-lengths.t 2009-04-06 07:33:32.000000000 +0200
@@ -0,0 +1,25 @@
+#!/usr/bin/perl
+use strict;
+use FindBin qw($Bin);
+our @files;
+
+BEGIN {
+ chdir "$Bin/.." or die;
+ @files = ( "doc/protocol.txt" );
+}
+
+use Test::More tests => scalar(@files);
+
+foreach my $f (@files) {
+ open(my $fh, $f) or die("Can't open $f");
+ my @long_lines = ();
+ my $line_number = 0;
+ while(<$fh>) {
+ $line_number++;
+ if(length($_) > 80) {
+ push(@long_lines, $line_number);
+ }
+ }
+ close($fh);
+ ok(@long_lines == 0, "$f has a long lines: @long_lines");
+}
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/managed-buckets.t new/memcached-1.2.8/t/managed-buckets.t
--- old/memcached-1.2.6/t/managed-buckets.t 2008-04-20 07:00:41.000000000 +0200
+++ new/memcached-1.2.8/t/managed-buckets.t 1970-01-01 01:00:00.000000000 +0100
@@ -1,11 +0,0 @@
-#!/usr/bin/perl
-
-use strict;
-use Test::More skip_all => "Tests not written."; # tests => 1
-use FindBin qw($Bin);
-use lib "$Bin/lib";
-use MemcachedTest;
-
-my $server = new_memcached();
-my $sock = $server->sock;
-
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/t/stats.t new/memcached-1.2.8/t/stats.t
--- old/memcached-1.2.6/t/stats.t 2008-04-26 23:58:33.000000000 +0200
+++ new/memcached-1.2.8/t/stats.t 2009-04-11 05:48:08.000000000 +0200
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-use Test::More tests => 17;
+use Test::More tests => 23;
use FindBin qw($Bin);
use lib "$Bin/lib";
use MemcachedTest;
@@ -25,6 +25,7 @@
## STAT curr_connections 1
## STAT total_connections 2
## STAT connection_structures 2
+## STAT cmd_flush 0
## STAT cmd_get 0
## STAT cmd_set 0
## STAT get_hits 0
@@ -37,12 +38,14 @@
my $stats = mem_stats($sock);
# Test number of keys
-is(scalar(keys(%$stats)), 22, "22 stats values");
+is(scalar(keys(%$stats)), 25, "25 stats values");
# Test initial state
-foreach my $key (qw(curr_items total_items bytes cmd_get cmd_set get_hits evictions get_misses bytes_written)) {
+foreach my $key (qw(curr_items total_items bytes cmd_flush cmd_get cmd_set get_hits evictions get_misses
+ bytes_written listen_disabled_num)) {
is($stats->{$key}, 0, "initial $key is zero");
}
+is($stats->{accepting_conns}, 1, "initial accepting_conns is one");
# Do some operations
@@ -55,3 +58,13 @@
foreach my $key (qw(total_items curr_items cmd_get cmd_set get_hits)) {
is($stats->{$key}, 1, "after one set/one get $key is 1");
}
+
+my $cache_dump = mem_stats($sock, " cachedump 1 100");
+ok(defined $cache_dump->{'foo'}, "got foo from cachedump");
+
+print $sock "flush_all\r\n";
+is(scalar <$sock>, "OK\r\n", "flushed");
+
+my $stats = mem_stats($sock);
+is($stats->{cmd_flush}, 1, "after one flush cmd_flush is 1");
+
diff -urN --exclude=CVS --exclude=.cvsignore --exclude=.svn --exclude=.svnignore --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/memcached-1.2.6/thread.c new/memcached-1.2.8/thread.c
--- old/memcached-1.2.6/thread.c 2008-07-29 18:37:27.000000000 +0200
+++ new/memcached-1.2.8/thread.c 2009-04-11 07:55:26.000000000 +0200
@@ -5,6 +5,7 @@
* $Id$
*/
#include "memcached.h"
+#include
#include
#include
#include
@@ -218,6 +219,14 @@
}
}
+/*
+ * Sets whether or not we accept new connections.
+ */
+void mt_accept_new_conns(const bool do_accept) {
+ pthread_mutex_lock(&conn_lock);
+ do_accept_new_conns(do_accept);
+ pthread_mutex_unlock(&conn_lock);
+}
/*
* Pulls a conn structure from the freelist, if one is available.
@@ -360,7 +369,7 @@
}
/* Which thread we assigned a connection to most recently. */
-static int last_thread = -1;
+static int last_thread = 0;
/*
* Dispatches a new connection to another thread. This is only ever called
@@ -370,9 +379,16 @@
void dispatch_conn_new(int sfd, int init_state, int event_flags,
int read_buffer_size, int is_udp) {
CQ_ITEM *item = cqi_new();
- int thread = (last_thread + 1) % settings.num_threads;
- last_thread = thread;
+ int tid = last_thread % (settings.num_threads - 1);
+
+ /* Skip the dispatch thread (0) */
+ tid++;
+ assert(tid != 0);
+ assert(tid < settings.num_threads);
+ LIBEVENT_THREAD *thread = threads + tid;
+
+ last_thread = tid;
item->sfd = sfd;
item->init_state = init_state;
@@ -380,10 +396,10 @@
item->read_buffer_size = read_buffer_size;
item->is_udp = is_udp;
- cq_push(&threads[thread].new_conn_queue, item);
+ cq_push(&thread->new_conn_queue, item);
- MEMCACHED_CONN_DISPATCH(sfd, threads[thread].thread_id);
- if (write(threads[thread].notify_send_fd, "", 1) != 1) {
+ MEMCACHED_CONN_DISPATCH(sfd, thread->thread_id);
+ if (write(thread->notify_send_fd, "", 1) != 1) {
perror("Writing to thread notify pipe");
}
}
@@ -639,7 +655,7 @@
pthread_mutex_init(&cqi_freelist_lock, NULL);
cqi_freelist = NULL;
- threads = malloc(sizeof(LIBEVENT_THREAD) * nthreads);
+ threads = calloc(nthreads, sizeof(LIBEVENT_THREAD));
if (! threads) {
perror("Can't allocate thread descriptors");
exit(1);
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Remember to have fun...
--
To unsubscribe, e-mail: opensuse-commit+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse-commit+help@opensuse.org
