Introduction
This writeup is about war in space - a theme of countless stories and films, usually depicted without any connection to reality. Because of that disconnection, this writeup will begin with the things that are so terribly wrong in popular films about space war, and move from there to a realistic extrapolation of a hypothetical high-tech space war.
In the Films
Problems with Star Wars Fights
- The distances are far too small, allowing for faster combat than realistic. After all, even 100'000 km isn't that much in the scale of solar systems (1/1500 of Earth-Sun, 1/3 of Earth-Moon)
- Lasers are displayed as small green bolts moving at perhaps a few times the speed of the ships (which isn't very big, considering the distances). In fact, lasers are invisible in space, and move at light speed (bloody obvious). Of course, invisible shots are impractical for space battle scenes.
- The capital ships move more slowly than the space fighters. There is no reason for that, after all, speed is determined by acceleration, and acceleration is determined by thrust and mass. So speed is determined by the mass/thrust ratio. There is no reason why the same ratio can't be realised on a capital ship than on a fighter. You just have to devote the same proportional mass to engines.
Problems with Star Trek Fights
- Many of the same problems as above, but additionally:
-They think two-dimensionally. All ships are always orientated the same way. True, they've improved, but they should still listen to their own advice - what Spock said about Khan in Star Trek Two: "He's intelligent, but not experienced. His pattern indicates two-dimensional thinking..."
Speculations about real war in space
Distance, Speed and Time
Distance is the most important difference in space combat. War on a planet is different, since the distances are measured in kilometres at the most. That means that the unaided eye normally can see the enemy. This isn't the case with space battles. The enemy can only be perceived with technological help. Also, the enormeous distances involved result in delays. That means that if one ship is 20 million km away from an other, that one will see the enemy with a delay of one minute, which makes targeting absolutely impossible, since the enemy could go anywhere within that minute. For example, if the enemy had an acceleration of 10 m/s^2, it could be anywhere within a sphere of r = 36 km, V = 150 km^3. (See below for a more complete discussion of whether this shape is a sphere or a bullet-shape.)
These effects are further increased by the fact that space is three-dimensional, which is a challenge to human two-dimensional thinking. It also makes it harder to defend against attacks from all sides and makes blockades very difficult.
Since spaceships tend to be pretty fast, short-range weapons would have to be fired by high-precision automated systems. As shokwave writes below, Luke can't hit a spaceship whizzing past with several thousand m/s. (Well, maybe he can, but that's not a technological question.
Weapons
Of course the enormous distances play an important role in the question of what weapons would be suitable for a war in space. Surely not artillery. Artillery shells are far too slow and inaccurate to be of any use. No, the most obvious weapon is a nuclear missile. After all, there are no problems with nuclear contamination in space, and the destructive power of nuclear weapons makes sure that nothing of an enemy ship will remain after a missile strikes. Of course, these missiles would be guided. The trouble with those weapons is that they can be shot down. After all, the distances are so extreme that a ship shouldn't have any trouble detecting incoming missiles. Also, their guidance systems might be jammed.
Another option would be shrapnel cannon or missiles. If a shell containing billions of little metal splinters were accelerated towards the enemy ship, the shell exploding soon after firing, space would be filled with a hailstorm of high-velocity micrometeors which could tear a ship to pieces. A more feasible variety is a missile that contains these splinters, seeking out the enemy, and detonating while still out of range of the enemy defense guns, releasing the splinters towards the enemy ship. The problem with this type of weapon is whether it's possible to hit anything with a weapon of even this considerable spread, and whether it's possible to accelerate the splinters to a velocity at which they could do harm to a spaceship. Designing such a missile would mean trade-offs between speed (larger burn stage-to-warhead ratio), damage potential (magnitude of the nuclear weapon and the amount of shrapnel carried in the warhead), spread (a smaller spread means a denser hail of shrapnel but a larger chance that that hail misses the enemy entirely) and accuracy (how close the missile goes before it releases the shrapnel - closer means more accuracy but a greater likelihood to be shot down before exploding).
Lasers would actually be an option as well, because laser beams move at light speed, which makes exact targetting possible. However, a laser beam capable of actually hurting a ship would consume huge amounts of power and would have to be very focussed, making the area of effect very small. Probably the best way to use a laser would be to cover the approximate location of the enemy with a tight zig-zag. Another problem is that lasers can be stopped by reflective surfaces.
Perhaps it is possible to accelerate a small shell to near lightspeed. Such a weapon would do devastating damage and would be relatively easy to aim, like a laser, and like a laser, one would have to fire several shots at the approximate position of the enemy to make sure to hit him. Also, recoil would be a serious problem here. Such a weapon is commonly called a Railgun.
Stealth
Even though sensors will likely be very advanced, it is possible to be very stealthy in space. The recipe is simple: emit nothing, reflect nothing. That is entirely possible. If a ship is fitted with a radar-absorbing hull and painted black (like a stealth fighter), then it's already quite hard to detect. The main problem at that point is heat emission and engines. If the engines and other systems that produce lots of heat are turned off, it's pretty much impossible to detect a ship. However, a ship cannot maneuver without showing its position. It can only follow a trajectory dictated by gravity and inertia, which is sufficient if the position of its target can be predicted. Imagine an undetectable nuclear warhead aimed precisely at a planet or space station. On a planet, the nuke would only be noticed when it entered the atmosphere, on a space station even only at the moment it hit. So unpredictability is the best defense in space. If your position can be calculated, you're a sitting duck (meaning very easy to hit), whatever your actual speed. Because of this, planets and space stations are very unsafe places to be.
There is, however, one way to detect such an "invisible" ship. In its path it will, at various times, occlude stars. If these occlusions are recorded and correlated, the path of such a weapon can be determined. Because the warhead moves predictably, it can easily be shot down if detected. Because of this such weapons need to be as small as possible, which of course limits the payload.
Planets
As already stated in the last paragraph, planets are very vulnerable, because their movements are entirely predictable - even space stations can be moved slightly. They are also easy to hit, being several orders of magnitude larger than space ships or stations.
However, planets are impossible to invade. On earth there are currently several million soldiers - therefore, several million troops and equipment would have to be transported through space to the planet. That does sound rather incredible.
Still, planetary surfaces can be easily bombed into oblivion with a few nuclear bombs. Therefore, they can also be easily held hostage. The conquerors need never touch the planet's surface - only destroy all anti-space defenses and then demand tribute - or else. Tribute, of course, would have to be compact and lightweight. Art, high-tech devices, medicine, rare elements like uranium 235 and particularily information.
The result of this might be war-fleets wandering about and demanding tribute. Others fleets would perhaps offer their services to protect the planet from such blackmailers. The line between pirate and defender would probably blur around the ambiguous word "protection fee".
War
...is perfectly horrible and rather pointless (except World War II, which was just and glorious and totally morally non-ambiguous), and war in space is doubly so. It is hard to imagine any reason why people would build interplanetary or even interstellar spaceships in order to attack somebody so far away that there couldn't possibly be any benefit from it. However, in order for there to be war in space, there would have to be several colonised planets around - which suggests that space travel would need be easy. So perhaps benefits could be gotten. And there are different reasons for war than benefits like riches or land. Ideological or religious reasons, which can inspire people to start utterly stupid things - like war.
Now about the whole bullet shape versus sphere thing: yitz says that I ignore momentum. Well, I don't. The question of whether the shape a spaceship could be in more resembles a sphere or a bullet is not a question of momentum. (All inertial systems are created equal and from the POV of the spaceship it's standing still at the beginning of the evasion maneuver.) It's simply a question of where the spaceship is pointing at the beginning of the maneuver. If the ship takes half a minute to turn 180 degrees, then in a minute it can go a lot farther in the direction it's already pointing at than into the opposite one. If that is the case then the area of space the ship could be in within one minute is indeed more or less the shape of a bullet. If the ship takes even longer to turn compared to the time available, the bullet becomes more pointy. (If the ship can't turn at all the bullet becomes so pointy it's a straight line beginning at the ship's present position.) But if the time the ship takes to turn is small compared to the time available for the maneuver, then the shape of that space will look spherical.
Actually, what would really interest me is writing a simulator for this, using real physics and approximations of real technology. That way such questions as "Are railguns any good?" or "Is the offensive or the defensive in the advantage, and how much?" could be answered. But I lack technical data about missile speeds and lasers and railguns and stuff, and I have no idea how to model a realistic damage system, nor how the simulation should be displayed and controlled. If you do, /msg me.
I've chosen not to extend the scope of the writeup into the tactical / strategical questions SandThatWasARock and yitz have discussed so well below. I think it's better that this writeup focuses on providing a technical basis upon which to found those discussions.