agp: fix OOM and buffer overflow
authorVasiliy Kulikov <segoon@openwall.com>
Thu, 14 Apr 2011 16:55:19 +0000 (20:55 +0400)
committerPaolo Pisati <paolo.pisati@canonical.com>
Tue, 31 May 2011 13:27:06 +0000 (15:27 +0200)
commite001fc1f8340837021509bfc482c039c0d5700a7
tree9ddaea4f3dc81daa7a0b97aa29682b8611bbaecc
parent59ee034d0a0cca0947f8ab22f15f9b3a2f139ab8
agp: fix OOM and buffer overflow

BugLink: http://bugs.launchpad.net/bugs/775809
commit b522f02184b413955f3bc952e3776ce41edc6355 upstream.

page_count is copied from userspace.  agp_allocate_memory() tries to
check whether this number is too big, but doesn't take into account the
wrap case.  Also agp_create_user_memory() doesn't check whether
alloc_size is calculated from num_agp_pages variable without overflow.
This may lead to allocation of too small buffer with following buffer
overflow.

Another problem in agp code is not addressed in the patch - kernel memory
exhaustion (AGPIOC_RESERVE and AGPIOC_ALLOCATE ioctls).  It is not checked
whether requested pid is a pid of the caller (no check in agpioc_reserve_wrap()).
Each allocation is limited to 16KB, though, there is no per-process limit.
This might lead to OOM situation, which is not even solved in case of the
caller death by OOM killer - the memory is allocated for another (faked) process.

Signed-off-by: Vasiliy Kulikov <segoon@openwall.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: Tim Gardner <tim.gardner@canonical.com>
Signed-off-by: Paolo Pisati <paolo.pisati@canonical.com>
drivers/char/agp/generic.c