One of the truths of computer security, to which there are a few exceptions. See also: physical security.

Even if a system is impenetrable from over a network, unrestricted physical access is an easily forgotten or disregarded security vulnerability. If an attacker can simply walk into the room the machine is in without restriction of her actions there, it is only a matter of time before she can begin to do Bad Things to it, such as changing its root password.

Means by which to exploit such a weakness include:

Exploit a lack of access control on the console.
Some operating systems, such as Microsoft Windows, do not or can be configured not to require someone at the machine's console to authenticate (usually by entering a password) at all. Alternatively, if the system does require such authentication at the console, but someone is still logged in there, then you can use whatever privileges they have, such as access to sensitive data.

Exploit a lack of access control in the boot loader.
This can be accomplished by means such as pressing F8 while recent versions of Microsoft Windows are booting, specifying 'init=/bin/sh' as a boot parameter to Linux (usually done at LILO's 'boot:' prompt), etc.

Use a bootable floppy disk or CD-ROM.
Reboot the machine and boot from it, and voila, you can do whatever you like. The program on your boot disk might simply give you an unrestricted command prompt, or it might be some automated procedure for e.g. installing a rootkit. The machine's BIOS might be set to not boot from floppy or CD-ROM, in which case that setting will need to be changed. On an IBM PC-like machine, if a BIOS password is set which prevents this, it can be removed by opening up the chassis and pulling out the battery on its motherboard that powers the NVRAM that holds this password. However, all of the other BIOS settings will be erased as well!

Pull the recording media.
If you only need to get at what's on it, and you have the time, you can simply power off the machine, physically remove its recording media (usually one or more hard disks), and leave. The machine probably won't work after you do this, in which case it will be apparent that someone has done something to it; if you close everything up and make it look like it hasn't been tampered with (e.g., by replacing the chassis cover and plugging everything back in), it may take longer for someone to realize what's happened.

Swipe the whole thing.
If you can do so without being noticed or stopped, you may want to unplug the entire machine and haul it out. Stealing the machine outright means that, aside from getting at whatever is on its recording media, you can then use the machine itself for a new Web server or something. :) The viability of this method is significantly affected by the physical size of the machine: laptops and PDAs are a lot easier to steal than mainframes.

Destroy the machine.
If you only need to sabotage the system, rather than use it or get at its data, it might be easier or otherwise more viable to simply render the machine permanently inoperable. There are, of course, plenty of ways to accomplish this: shoot it, blow it up, hack it to pieces with an axe, etc. Use your imagination. It will probably be important that any recording media are properly destroyed, so a post-mortem inspection of the machine before leaving would be wise.

There are, of course, countermeasures:

BIOS settings.
Setting the BIOS to require a password to change BIOS settings and not boot from removable media will, at the very least, slow an attacker down.

Boot loader settings.
Setting the boot loader to require a password to do anything other than a normal boot would prevent attacks that exploit the lack of such restrictions.

Deny physical access to the machine itself.
If an attacker only has physical access to the console, and the console requires authentication as described above, the attacker will not be able to compromise the system by removing its recording media, etc.

Deny physical access, period.
Keeping the whole machine — console and all — under lock and key will theoretically deny physical access entirely, preventing all of the above attacks. Of course, if the attacker can penetrate the physical barrier protecting the machine (e.g., by getting ahold of the key to the door), then this won't help.

Security cameras / guards.
If your activities look suspicious, and someone can see you (e.g., through a security camera), then you might wind up having some guards walking in on you, thus thwarting your attack outright. Of course, this won't work if the only people that see you are working with you.

Chassis intrusion detection and self-destruct.
Chassis intrusion detection hardware, which can tell if someone has broken into the chassis (e.g. by removing the chassis cover), combined with some sort of self-destruct mechanism (e.g. a small explosive that blows away the recording media), could prevent sensitive information from falling into the hands of an attacker, by destroying it before the attacker even knows that there is a self-destruct mechanism. Such a system could be broken by anyone (e.g., a disgruntled employee) that knows how to disarm it.

Encryption of sensitive information.
If the sensitive information on the machine's recording media is encrypted, an attacker who simply swipes the recording media (or the whole machine) will have to crack the encryption, which is often infeasible due to the enormous keyspaces allowed for by modern ciphers. Such encryption might be accomplished with a straightforward encryption tool such as PGP, or with something more sophisticated, such as an encrypted filesystem. Such encryption can be defeated by covertly installing a piece of software on the machine that waits for someone on the machine to decrypt the data, then records the decrypted data or the key that was used to access it, for the attacker to pick up later. As the keys are usually passwords, the attacker could install a keylogger on the machine, leave it for a while, and wait for someone to type in the encryption key. A more sophisticated attack would be needed to break the encryption if the key and/or encryption engine is on a smart card (perhaps by intercepting the decrypted data on its way back from the smart card). Of course, if the encrypted data is only stored on the machine, and never decrypted there, then such an attack will not work.