If there is not enough room to swing a cat, it means that there is a lack of room in a given area.
This expression is often thought to be related not to live cats, but a cat o'nine tails - a vicious type of whip, typically used to discipline sailors.
Boats aren't known for being roomy, and a sailor complaining about living in close quarters might be heard to remark "Shiver me timbers! There's not room to swing a cat in here. Aar." Or something of the sort.
The punishment for crimes was not decided solely by the captain, presumably as such tyrannical power would soon lead to mutiny. Fellow sailors would be able to decide whether a miscreant would be whipped, and could decide the number of lashes.
However, the captain did retain the power to make one sailor whip another: it could be an enemy to ensure the punishment was brutal, or a friend to add a bit of psychological trauma to the physical pain.
Sailors could agree in advance to go easy on each other, should they be put in that position. Hence, "if you scratch my back, I'll scratch yours."