how to reboot into an updated kernel

The Red Hat Network Update Agent alerts me to the fact that my system, while currently running kernel-2.4.18-14, has a newly installed version 2.4.18-24.8.0 sitting somewhere.. and that I should reboot. (Both assertions are correct, based on what I can tell).
I have rebooted, but not into the updated kernel. I always boot into the older kernel. Is this because I use a boot disk that is generated by the older version? If so, how do I get around this?

You need to update your boot disk. But since there is many ways of having a linux boot-disk, i cannot tell you how unless you know more details about it. Best way: download a new disk image from redhat.com

You might have yerself a nice shiny new kernel, but you prolly don't got it all configured. That'd be fun, reboot to new kernel with no support for what you got. go to /usr/src and see what you got there for kernel sources
go into the directory with the newer version, or the version you want to use.
#make mrproper
#make menuconfig
This will get you into the ncurses cli gui deal to configure your kernel options. go over it once or twice to make sure you support what you got. yay.
exit, save changes
#make dep && make clean bzImage modules modules_install
this will compile up your new kernel image.
when it's done make a backup of your /boot/bzImage and then
#cp /usr/src/linuxkernelsourcediryouchose/arch/i386/boot/bzImage /boot

or do a find from /usr/src/linuxkernelsourcediryouchose
find ./ -name bzImage
to locate it for your architecture.

after copying that over you can reboot into your new kernel.

A lot of distributions sport a mkbootdisk tool...

Try these:

Compiling your own kernel
Configuring your bootloader

and for background info if you're interested:

Introduction to the Linux kernel