Andrew Bresticker c1e2ee2dc4 memcg: replace ss->id_lock with a rwlock
While back-porting Johannes Weiner's patch "mm: memcg-aware global
reclaim" for an internal effort, we noticed a significant performance
regression during page-reclaim heavy workloads due to high contention of
the ss->id_lock.  This lock protects idr map, and serializes calls to
idr_get_next() in css_get_next() (which is used during the memcg hierarchy
walk).

Since idr_get_next() is just doing a look up, we need only serialize it
with respect to idr_remove()/idr_get_new().  By making the ss->id_lock a
rwlock, contention is greatly reduced and performance improves.

Tested: cat a 256m file from a ramdisk in a 128m container 50 times on
each core (one file + container per core) in parallel on a NUMA machine.
Result is the time for the test to complete in 1 of the containers.
Both kernels included Johannes' memcg-aware global reclaim patches.

Before rwlock patch: 1710.778s
After rwlock patch: 152.227s

Signed-off-by: Andrew Bresticker <abrestic@google.com>
Cc: Paul Menage <menage@gmail.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Ying Han <yinghan@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-11-02 16:07:03 -07:00
..
2011-10-26 23:15:28 +02:00
2011-09-13 15:42:30 -04:00
2011-07-22 08:25:37 -07:00
2011-11-02 16:07:03 -07:00
2011-07-20 20:47:43 -04:00
2011-07-26 16:49:47 -07:00
2011-10-26 15:43:25 -04:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-08-03 11:30:42 -04:00
2011-07-31 22:05:09 +02:00
2011-07-26 16:49:47 -07:00
2011-09-27 09:25:02 +01:00
2011-10-31 17:30:44 -07:00
2011-07-26 16:49:47 -07:00
2011-08-25 16:25:33 -07:00
2011-08-03 19:06:37 -04:00
2011-07-06 14:44:42 -07:00
2011-07-25 20:57:16 -07:00
2011-07-05 23:42:17 -07:00
2011-07-26 16:49:47 -07:00
2011-10-02 00:19:34 +02:00
2011-05-29 13:03:09 +01:00
2011-07-26 16:49:47 -07:00
2011-09-14 15:24:51 -04:00
2011-06-25 17:29:52 +02:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-08-08 16:08:08 +02:00
2011-07-01 15:34:45 -07:00
2011-10-25 09:59:04 +02:00
2011-06-28 10:48:34 +02:00
2011-07-01 10:37:15 +02:00
2011-10-30 04:43:30 -04:00
2011-07-21 13:47:54 -07:00
2011-09-27 18:08:04 +02:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-09-25 19:52:59 +03:00
2011-10-30 12:24:05 +02:00
2011-09-16 19:20:20 -04:00
2011-11-02 16:06:57 -07:00
2011-11-02 16:06:58 -07:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-09-27 08:12:33 -07:00
2011-07-31 12:18:16 -04:00
2011-07-28 16:19:22 -06:00
2011-10-31 14:03:22 +01:00
2011-11-02 16:07:02 -07:00
2011-07-26 16:49:47 -07:00
2011-07-26 14:50:01 -07:00
2011-06-07 10:02:35 +02:00
2011-07-30 08:44:19 -10:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-07-25 20:57:11 -07:00
2011-10-31 17:30:47 -07:00
2011-08-16 00:16:49 -07:00
2011-08-03 14:25:22 -10:00
2011-10-31 17:30:44 -07:00
2011-11-02 16:07:02 -07:00
2011-10-03 14:01:21 -04:00
2011-06-15 20:03:59 -07:00
2011-06-27 20:30:08 +02:00
2011-08-23 10:10:38 -07:00
2011-06-07 09:05:42 -07:00
2011-09-26 15:51:10 -07:00
2011-11-02 16:07:02 -07:00
2011-10-24 02:07:21 -04:00
2011-07-26 16:49:47 -07:00
2011-07-26 16:49:47 -07:00
2011-09-14 15:24:51 -04:00