If you're outfitting your system with 4gb of ram (or more), then you'll basically need a 64 bit system to take advantage of it. Although a 32bit system can address 4gbs, any single application can only make use of at most 2gb at a time. This generally won't matter for most apps (most won't use anywhere near this much memory), but 'bigger' apps (e.g. photoshop, some games, etc.) can run in 64bit mode and take advantage of the full amount of memory. Memory constraints aside, 64bit applications and systems generally perform better than their 32bit counterparts, although the differences are rarely big (outside of some very specific situations, such as math heavy apps).
The posters above gave a good synopsis of the current state of compatibility and operating modes, but I'll just add my own seal of approval; I switched my desktop and laptop over to Vista 64 about 2 years ago and haven't had any issues in ages. Most of the 'warnings' you see on the net refer to the state of compatibility when Vista 64 and 64bit-configured systems first started being released by the major manufacturers several years ago. Unless you have a particularly old piece of hardware or software that you must use, you're very unlikely to run into any problems. On a new system, it'll almost certainly work flawlessly.