malhuda/msm-4.14
1
0
Fork 0
mirror of https://github.com/rd-stuffs/msm-4.14.git synced 2025-02-20 11:45:48 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

locking/rwsem: Don't hog RCU read lock while optimistically spinning

Browse Source 
There's no reason to hold an RCU read lock the entire time while
optimistically spinning for a rwsem. This can needlessly lengthen RCU
grace periods and slow down synchronize_rcu() when it doesn't brute
force the RCU grace period via rcupdate.rcu_expedited=1.

Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
Signed-off-by: azrim <mirzaspc@gmail.com>
This commit is contained in:
Sultan Alsawaf 2020-12-26 12:55:02 -08:00 committed by azrim
parent 2fab6f5aa1
commit 4b03b283a3
No known key found for this signature in database
GPG Key ID: 497F8FB059B45D1C
1 changed files with 15 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
kernel/locking/rwsem-xadd.c
View File

@ -400,32 +400,36 @@ static noinline bool rwsem_spin_on_owner(struct rw_semaphore *sem)
{
struct task_struct *owner = READ_ONCE(sem->owner);
if (!is_rwsem_owner_spinnable(owner))
if (!owner || !is_rwsem_owner_spinnable(owner))
return false;
rcu_read_lock();
while (owner && (READ_ONCE(sem->owner) == owner)) {
while (true) {
bool on_cpu, same_owner;
/*
* Ensure we emit the owner->on_cpu, dereference _after_
* checking sem->owner still matches owner, if that fails,
* Ensure sem->owner still matches owner. If that fails,
* owner might point to free()d memory, if it still matches,
* the rcu_read_lock() ensures the memory stays valid.
*/
barrier();
rcu_read_lock();
same_owner = sem->owner == owner;
if (same_owner)
on_cpu = owner->on_cpu &&
!vcpu_is_preempted(task_cpu(owner));
rcu_read_unlock();
if (!same_owner)
break;
/*
* abort spinning when need_resched or owner is not running or
* owner's cpu is preempted.
*/
if (!owner->on_cpu || need_resched() ||
vcpu_is_preempted(task_cpu(owner))) {
rcu_read_unlock();
if (!on_cpu || need_resched())
return false;
}
cpu_relax();
}
rcu_read_unlock();
/*
* If there is a new owner or the owner is not set, we continue