Nick Piggin 21cc199baa mm: introduce get_user_pages_fast
Introduce a new get_user_pages_fast mm API, which is basically a
get_user_pages with a less general API (but still tends to be suited to
the common case):

- task and mm are always current and current->mm
- force is always 0
- pages is always non-NULL
- don't pass back vmas

This restricted API can be implemented in a much more scalable way on many
architectures when the ptes are present, by walking the page tables
locklessly (no mmap_sem or page table locks).  When the ptes are not
populated, get_user_pages_fast() could be slower.

This is implemented locklessly on x86, and used in some key direct IO call
sites, in later patches, which provides nearly 10% performance improvement
on a threaded database workload.

Lots of other code could use this too, depending on use cases (eg.  grep
drivers/).  And it might inspire some new and clever ways to use it.

[akpm@linux-foundation.org: build fix]
[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: Dave Kleikamp <shaggy@austin.ibm.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Dave Kleikamp <shaggy@austin.ibm.com>
Cc: Badari Pulavarty <pbadari@us.ibm.com>
Cc: Zach Brown <zach.brown@oracle.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Reviewed-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-07-26 12:00:05 -07:00
..
2008-07-25 10:53:30 -07:00
2008-07-24 13:32:56 +03:00
2008-07-01 11:28:06 +10:00
2008-06-19 10:42:17 +10:00
2008-06-19 10:42:17 +10:00
2008-07-24 10:47:24 -07:00
2008-07-24 10:47:33 -07:00
2008-07-24 10:47:33 -07:00
2008-07-03 13:21:14 +02:00
2008-04-29 08:11:16 -07:00
2008-06-06 11:29:10 -07:00
2008-05-29 01:38:46 -07:00
2008-04-22 15:16:32 -05:00
2008-04-29 08:06:01 -07:00
2008-07-04 10:40:05 -07:00
2008-07-25 10:53:33 -07:00
2008-05-01 08:03:59 -07:00
2008-07-20 17:12:38 -07:00
2008-07-22 17:30:57 -07:00
2008-07-11 10:46:50 +02:00
2008-07-21 12:00:37 +01:00
2008-04-21 11:22:28 -05:00
2008-04-25 13:26:55 +01:00
2008-04-25 13:26:55 +01:00
2008-07-21 12:00:37 +01:00
2008-04-25 13:26:55 +01:00
2008-04-28 08:58:30 -07:00
2008-07-24 10:47:29 -07:00
2008-05-07 09:48:23 +02:00
2008-07-24 10:47:41 -07:00
2008-05-16 17:22:26 -04:00
2008-05-01 13:08:16 -04:00
2008-07-08 17:55:41 +02:00
2008-07-16 23:27:03 +02:00
2008-07-11 15:49:21 +02:00
2008-07-25 10:53:48 -07:00
2008-07-20 17:12:36 -07:00
2008-07-25 10:53:30 -07:00
2008-07-20 17:12:36 -07:00
2008-07-23 15:25:21 +02:00
2008-05-20 16:44:43 +02:00
2008-04-22 11:34:59 +02:00
2008-07-25 10:53:42 -07:00
2008-06-11 21:00:38 -07:00
2008-07-18 18:05:19 -07:00
2008-06-11 21:00:38 -07:00
2008-07-14 22:49:06 -07:00
2008-07-03 17:51:57 +09:00
2008-07-25 10:53:39 -07:00
2008-05-05 16:47:14 +10:00
2008-07-16 23:27:07 +02:00
2008-06-11 21:00:38 -07:00
2008-04-29 08:06:15 -07:00
2008-04-19 19:10:28 -07:00
2008-07-20 17:12:37 -07:00
2008-05-01 08:03:58 -07:00
2008-05-20 12:17:39 -04:00
2008-07-25 10:53:29 -07:00
2008-07-25 10:53:29 -07:00
2008-07-26 12:00:04 -07:00
2008-07-24 10:47:17 -07:00
2008-07-04 10:40:05 -07:00
2008-04-19 19:44:57 +02:00
2008-07-20 12:42:32 +03:00
2008-07-23 09:49:56 +01:00
2008-07-24 10:47:41 -07:00
2008-05-23 22:26:03 +02:00
2008-04-29 08:06:01 -07:00
2008-07-25 10:53:40 -07:00
2008-07-26 12:00:05 -07:00
2008-07-24 10:47:24 -07:00
2008-07-25 10:53:34 -07:00
2008-06-06 11:29:12 -07:00
2008-04-28 08:58:29 -07:00
2008-07-25 10:53:29 -07:00
2008-04-19 16:54:56 -04:00
2008-07-24 10:47:15 -07:00
2008-07-26 12:00:03 -07:00
2008-07-24 11:50:15 -07:00
2008-06-10 14:37:03 -07:00
2008-07-25 10:53:48 -07:00
2008-07-07 16:26:50 -07:00
2008-04-20 21:47:03 -07:00
2008-04-29 08:06:02 -07:00
2008-04-29 08:06:02 -07:00
2008-05-06 12:01:41 -04:00
2008-06-11 21:00:38 -07:00
2008-06-11 21:00:38 -07:00
2008-07-25 10:53:44 -07:00
2008-05-19 16:25:39 +01:00
2008-07-25 10:53:29 -07:00
2008-05-19 10:01:36 +02:00
2008-07-04 10:40:07 -07:00
2008-07-25 10:53:29 -07:00
2008-04-29 08:05:59 -07:00
2008-05-25 07:09:47 +02:00
2008-07-24 10:47:33 -07:00
2008-07-24 10:47:22 -07:00
2008-07-25 10:53:42 -07:00
2008-07-24 08:31:21 -04:00
2008-04-25 00:25:08 +02:00
2008-04-28 08:58:32 -07:00
2008-07-24 10:47:24 -07:00
2008-07-14 22:49:06 -07:00
2008-07-24 10:47:21 -07:00
2008-07-04 10:40:04 -07:00
2008-07-25 10:53:30 -07:00
2008-06-20 14:05:53 -06:00
2008-07-20 17:12:37 -07:00
2008-04-28 08:58:20 -07:00
2008-06-12 10:48:00 +02:00
2008-05-01 08:03:59 -07:00
2008-07-22 13:03:28 -07:00
2008-07-20 17:12:34 -07:00
2008-07-22 13:03:27 -07:00
2008-07-21 15:15:55 -07:00
2008-05-09 07:45:18 -07:00
2008-04-28 08:58:21 -07:00
2008-07-24 10:47:14 -07:00
2008-07-25 10:53:40 -07:00