What version is the router? Just asking cause you can flash DD-WRT onto certain versions of that model. Pray you dont have v7, since thats the only version not supported.
Next thing, you need to convert your modem into bridge mode. This lets the modem act as a modem only, and lets your router do the routing. If you provide a brand / model number for the modem I can walk you through it.
If you dont have v7, see below for installation instructions.
http://dd-wrt.com/wiki/index.php/Linksys
This firmware is so much better for these devices, you will love it. After you get it up and running.