Jack Steiner
6adef3ebe5
cpusets: new round-robin rotor for SLAB allocations ...
We have observed several workloads running on multi-node systems where
memory is assigned unevenly across the nodes in the system. There are
numerous reasons for this but one is the round-robin rotor in
cpuset_mem_spread_node().
For example, a simple test that writes a multi-page file will allocate
pages on nodes 0 2 4 6 ... Odd nodes are skipped. (Sometimes it
allocates on odd nodes & skips even nodes).
An example is shown below. The program "lfile" writes a file consisting
of 10 pages. The program then mmaps the file & uses get_mempolicy(...,
MPOL_F_NODE) to determine the nodes where the file pages were allocated.
The output is shown below:
# ./lfile
allocated on nodes: 2 4 6 0 1 2 6 0 2
There is a single rotor that is used for allocating both file pages & slab
pages. Writing the file allocates both a data page & a slab page
(buffer_head). This advances the RR rotor 2 nodes for each page
allocated.
A quick confirmation seems to confirm this is the cause of the uneven
allocation:
# echo 0 >/dev/cpuset/memory_spread_slab
# ./lfile
allocated on nodes: 6 7 8 9 0 1 2 3 4 5
This patch introduces a second rotor that is used for slab allocations.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Acked-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Paul Menage <menage@google.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: Robin Holt <holt@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-05-27 09:12:44 -07:00
2010-05-25 00:38:26 -06:00
2010-05-26 08:30:15 -07:00
2010-04-28 12:55:14 -07:00
2010-05-17 22:39:48 -07:00
2010-03-12 15:52:44 -08:00
2010-05-19 12:57:48 -03:00
2010-05-27 09:12:41 -07:00
2010-04-13 23:27:16 -07:00
2010-02-16 16:01:22 -08:00
2010-05-25 10:16:02 +01:00
2010-04-21 16:37:49 -07:00
2010-05-27 09:12:40 -07:00
2010-05-14 01:56:12 +01:00
2010-05-20 15:59:16 +02:00
2010-02-24 18:32:59 +01:00
2010-02-24 18:34:48 +01:00
2010-03-08 13:17:01 +01:00
2010-05-03 08:33:00 -04:00
2010-05-25 10:16:01 +01:00
2010-05-25 00:38:26 -06:00
2010-04-26 13:51:09 -04:00
2010-05-14 15:09:37 -04:00
2010-05-20 21:26:12 -07:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:53:10 -08:00
2010-05-04 13:34:27 -07:00
2010-05-14 17:08:01 -04:00
2010-05-21 09:34:30 -07:00
2010-05-21 09:34:30 -07:00
2010-05-19 13:38:54 -04:00
2010-05-21 20:00:35 +02:00
2010-03-16 19:47:54 +00:00
2010-03-06 11:26:46 -08:00
2010-05-18 09:17:01 -07:00
2010-05-21 15:26:46 -07:00
2010-02-17 11:17:38 +09:00
2010-02-12 09:41:59 -08:00
2010-02-17 17:27:40 -08:00
2010-02-22 13:44:45 +01:00
2010-02-22 13:44:45 +01:00
2010-05-27 09:12:44 -07:00
2010-03-24 16:31:22 -07:00
2010-03-12 19:10:29 +01:00
2010-05-10 14:24:26 +02:00
2010-04-28 09:20:33 +02:00
2010-05-25 08:07:00 -07:00
2010-03-12 15:52:32 -08:00
2010-03-12 15:52:32 -08:00
2010-04-09 14:07:56 -07:00
2010-03-06 11:26:29 -08:00
2010-05-27 09:12:44 -07:00
2010-03-04 11:46:14 +01:00
2010-05-19 22:03:14 +03:00
2010-05-15 07:16:33 -04:00
2010-04-22 18:32:12 -07:00
2010-05-10 16:08:01 -07:00
2010-03-30 22:02:32 +09:00
2010-03-06 02:32:27 +00:00
2010-05-22 00:36:56 -06:00
2010-03-06 02:32:33 +00:00
2010-03-06 02:32:31 +00:00
2010-03-12 15:52:42 -08:00
2010-05-17 16:30:58 -07:00
2010-05-21 19:30:44 +02:00
2010-05-18 01:25:00 +02:00
2010-05-18 01:25:00 +02:00
2010-05-21 21:12:01 +02:00
2010-04-26 22:03:42 +04:00
2010-05-25 08:07:05 -07:00
2010-02-26 08:25:35 +01:00
2010-04-09 08:36:07 +02:00
2010-05-17 10:00:15 +02:00
2010-05-21 20:29:10 -07:00
2010-05-25 08:07:02 -07:00
2010-04-07 21:54:42 -07:00
2010-02-20 13:13:44 -08:00
2010-05-21 19:30:39 +02:00
2010-03-29 14:30:19 -07:00
2010-03-29 14:30:19 -07:00
2010-02-26 19:19:39 +02:00
2010-05-25 08:07:09 -07:00
2010-05-21 21:12:40 +02:00
2010-02-25 10:34:49 +01:00
2010-05-24 00:36:13 -07:00
2010-04-26 16:13:54 +02:00
2010-04-22 16:05:44 -07:00
2010-04-15 22:18:36 +02:00
2010-04-15 22:18:36 +02:00
2010-03-06 11:26:25 -08:00
2010-05-21 09:37:28 -07:00
2010-03-26 23:51:44 +01:00
2010-05-21 19:37:45 -07:00
2010-03-29 09:14:47 -07:00
2010-03-30 22:02:32 +09:00
2010-05-21 07:19:18 -07:00
2010-05-18 08:35:04 -07:00
2010-03-30 22:02:32 +09:00
2010-05-21 18:31:18 -04:00
2010-04-03 14:56:05 -07:00
2010-03-16 08:55:32 +01:00
2010-05-25 08:07:01 -07:00
2010-03-01 14:08:10 +00:00
2010-05-27 09:12:42 -07:00
2010-05-21 09:34:29 -07:00
2010-05-19 14:05:06 +02:00
2010-05-25 08:07:01 -07:00
2010-04-06 21:50:03 +02:00
2010-03-07 22:17:09 +01:00
2010-05-01 04:32:14 +02:00
2010-03-13 20:56:56 +01:00
2010-05-20 00:18:59 +01:00
2010-02-12 17:32:41 +00:00
2010-03-02 12:23:42 +01:00
2010-03-14 11:14:58 -07:00
2010-05-22 00:36:56 -06:00
2010-03-24 08:20:03 +01:00
2010-02-18 14:30:17 -08:00
2010-05-24 08:05:29 -07:00
2010-04-20 11:52:21 -04:00
2010-03-30 19:08:43 -07:00
2010-03-30 19:08:43 -07:00
2010-05-17 22:49:55 -07:00
2010-05-15 23:48:02 -07:00
2010-04-13 01:30:48 -07:00
2010-04-03 14:56:04 -07:00
2010-04-03 14:56:04 -07:00
2010-05-03 12:33:13 +03:00
2010-03-21 21:19:02 -07:00
2010-04-22 16:12:36 -07:00
2010-05-06 00:47:21 -07:00
2010-04-23 23:35:28 -07:00
2010-02-16 15:55:17 -08:00
2010-05-11 16:10:47 -07:00
2010-04-13 23:27:16 -07:00
2010-05-21 10:48:12 -07:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:52:36 -08:00
2010-03-07 18:01:13 +01:00
2010-05-11 12:01:10 -07:00
2010-03-12 15:52:40 -08:00
2010-03-12 15:52:40 -08:00
2010-05-11 14:40:55 +02:00
2010-05-03 11:50:57 +02:00
2010-05-19 17:33:38 +09:30
2010-04-01 16:12:48 -07:00
2010-05-25 08:07:09 -07:00
2010-03-30 22:02:32 +09:00
2010-05-21 19:30:40 +02:00
2010-02-25 12:51:41 +00:00
2010-04-22 16:12:36 -07:00
2010-05-20 21:04:27 -05:00
2010-05-25 08:07:05 -07:00
2010-02-16 16:01:21 -08:00
2010-02-17 11:17:38 +09:00
2010-04-23 02:08:44 +02:00
2010-05-20 21:04:29 -05:00
2010-03-12 15:53:10 -08:00
2010-03-15 15:29:39 +01:00
2010-05-21 09:37:32 -07:00
2010-02-25 17:49:25 +01:00
2010-05-21 09:37:29 -07:00
2010-04-21 16:33:29 -07:00
2010-05-14 15:09:32 -04:00
2010-05-19 11:36:29 +03:00
2010-05-17 12:17:10 +03:00
2010-04-03 14:56:05 -07:00
2010-03-15 12:47:59 +01:00
2010-05-19 13:38:55 -04:00
2010-05-25 08:07:07 -07:00
2010-03-06 11:26:35 -08:00
2010-05-21 09:37:30 -07:00
2010-04-28 08:51:12 +10:00
2010-02-19 03:35:12 -05:00
2010-05-25 08:07:09 -07:00
2010-05-27 09:12:44 -07:00
2010-05-25 08:07:00 -07:00
2010-03-17 18:43:47 -07:00
2010-05-25 08:06:57 -07:00
2010-05-18 00:52:36 -03:00
2010-05-25 08:06:59 -07:00
2010-05-25 15:08:26 -07:00
2010-03-12 15:52:28 -08:00
2010-05-25 08:07:02 -07:00
2010-05-25 08:07:02 -07:00
2010-03-03 14:07:59 -05:00
2010-05-21 17:15:44 -07:00
2010-04-08 10:18:47 +02:00
2010-03-04 08:15:33 -08:00
2010-05-11 14:40:55 +02:00
2010-04-13 14:49:34 -07:00
2010-04-28 15:16:48 -07:00
2010-04-22 12:31:11 +02:00
2010-05-17 05:27:42 +02:00
2010-05-01 15:00:15 -07:00
2010-05-23 23:21:13 -07:00
2010-04-20 16:22:01 +02:00
2010-03-25 11:17:26 +01:00
2010-02-19 08:03:28 +01:00
2010-05-21 09:37:32 -07:00
2010-05-06 01:31:27 -07:00
2010-05-14 15:09:30 -04:00
2010-05-14 15:09:21 -04:00
2010-05-14 15:09:30 -04:00
2010-05-10 11:32:33 +09:00
2010-05-07 14:55:50 -04:00
2010-03-12 15:52:38 -08:00
2010-05-06 00:47:21 -07:00
2010-05-22 00:10:41 -06:00
2010-04-28 18:20:29 -06:00
2010-05-22 00:10:40 -06:00
2010-03-18 07:30:31 -06:00
2010-02-15 15:14:34 +02:00
2010-05-19 13:44:27 +10:00
2010-05-27 09:12:44 -07:00
2010-02-22 16:15:17 -08:00
2010-05-11 12:01:40 -07:00
2010-05-21 18:58:52 -07:00
2010-02-22 16:21:02 -08:00
2010-03-12 15:52:42 -08:00
2010-04-09 17:00:00 +01:00
2010-04-16 19:14:34 +04:00
2010-03-03 07:34:18 -08:00
2010-04-05 09:39:11 -07:00
2010-05-11 08:31:49 +02:00
2010-04-02 14:30:39 -07:00
2010-05-21 21:12:52 +02:00
2010-02-24 08:30:08 +01:00
2010-05-17 21:37:40 +02:00
2010-05-10 23:08:19 +02:00
2010-05-10 23:09:30 +02:00
2010-05-10 23:08:15 +02:00
2010-03-06 21:28:37 +01:00
2010-04-24 11:31:25 -07:00
2010-03-12 15:53:11 -08:00
2010-05-19 12:14:42 +04:00
2010-04-03 14:56:02 -07:00
2010-03-26 11:33:55 +01:00
2010-05-21 19:30:44 +02:00
2010-05-21 19:30:45 +02:00
2010-04-09 10:12:03 -07:00
2010-05-21 18:31:26 -04:00
2010-05-25 08:07:03 -07:00
2010-05-18 09:28:04 -07:00
2010-02-25 09:41:02 +01:00
2010-05-03 15:53:54 -07:00
2010-05-10 16:53:55 -07:00
2010-05-18 08:27:54 -07:00
2010-05-18 08:27:54 -07:00
2010-03-12 15:52:44 -08:00
2010-05-21 18:31:19 -04:00
2010-03-05 13:25:52 -05:00
2010-05-21 18:31:19 -04:00
2010-02-26 20:39:09 +01:00
2010-03-02 14:28:49 -05:00
2010-04-27 13:06:35 -04:00
2010-05-25 08:06:58 -07:00
2010-05-11 10:09:47 +02:00
2010-03-12 15:52:28 -08:00
2010-03-12 15:53:10 -08:00
2010-05-11 14:40:55 +02:00
2010-03-13 01:21:21 +01:00
2010-05-27 09:12:44 -07:00
2010-05-27 09:12:39 -07:00
2010-05-06 10:56:07 +10:00
2010-02-22 15:45:54 -08:00
2010-05-21 11:08:05 -07:00
2010-03-23 17:19:30 +09:00
2010-04-13 23:27:41 -07:00
2010-05-13 18:42:25 +09:00
2010-04-26 15:50:50 +09:00
2010-04-15 13:13:52 +09:00
2010-03-12 15:52:44 -08:00
2010-05-17 17:18:50 -07:00
2010-05-19 22:03:13 +03:00
2010-04-09 10:09:50 -07:00
2010-05-19 22:03:13 +03:00
2010-05-19 22:03:13 +03:00
2010-04-22 12:37:07 +02:00
2010-03-06 11:26:32 -08:00
2010-04-03 15:09:04 -07:00
2010-04-06 23:53:30 -07:00
2010-03-13 01:21:21 +01:00
2010-05-21 21:12:40 +02:00
2010-05-10 11:08:35 -07:00
2010-04-14 04:49:51 -07:00
2010-05-08 17:12:33 +02:00
2010-05-27 09:12:43 -07:00
2010-03-22 13:12:33 -07:00
2010-05-15 23:28:39 -07:00
2010-03-07 17:04:47 -08:00
2010-05-21 09:37:31 -07:00
2010-04-13 23:26:02 -07:00
2010-03-30 22:02:32 +09:00
2010-05-19 11:36:34 +03:00
2010-05-03 23:50:42 -07:00
2010-02-18 15:43:09 -08:00
2010-05-09 19:35:27 +02:00
2010-03-25 17:18:43 -07:00
2010-04-13 12:43:42 +02:00
2010-04-06 21:50:02 +02:00
2010-03-23 17:19:38 +01:00
2010-05-12 23:02:23 -07:00
2010-05-12 23:02:23 -07:00
2010-05-05 11:46:17 -04:00
2010-03-02 14:55:11 -08:00
2010-03-12 15:52:43 -08:00
2010-05-21 09:34:29 -07:00
2010-05-17 07:57:27 -07:00
2010-05-20 13:21:50 -07:00
2010-03-01 22:21:58 +11:00
2010-05-11 16:07:59 -06:00
2010-05-11 16:07:59 -06:00
2010-05-11 16:07:59 -06:00
2010-05-19 12:58:37 -03:00
2010-03-13 08:57:28 -06:00
2010-02-24 14:22:08 +10:30
2010-05-19 22:15:40 +09:30
2010-05-19 22:15:49 +09:30
2010-05-19 22:15:46 +09:30
2010-05-25 08:06:59 -07:00
2010-03-19 07:17:52 -07:00
2010-05-20 21:26:12 -07:00
2010-03-23 16:50:26 -04:00
2010-05-21 20:01:03 +02:00
2010-05-21 18:31:18 -04:00
2010-02-22 16:19:45 -08:00
2010-04-06 20:35:58 +04:00
2010-05-17 21:37:42 +02:00
6adef3ebe5