The novel's setting is within the Culture's sector of the galaxy, it just isn't as obvious as most of Iain's other sci-fi books, and has hence led to quite a lot of discussion as to wether it is or isn't a real Culture novel. Being the first Iain Banks novel that I read, it certainly isn't at all obvious where it's set, but if you look at the book after reading other Culture novels, it becomes obvious.
(Warning: there may be spoilers below)
Firstly, Iain has said in several interviews that it is a Culture novel.
This quote from the second chapter of Use of Weapons that seems to sum up the doctor pretty well:
"Many of their [the Culture's] people become physicians to great leaders, and with medicines and treatments that seem like magic to the comparatively primitive people they're dealing with, ensure that a great and good leader has a better chance of survival."
And to quote from another discussion on the same subject:
"Early on the doctor is said to be a stranger from another Culture (spelled with a capital C) and near the end she is said to decline dining with the ships captain due to "special circumstances". I love that line. The knife missile is another hint [In the torture chamber, it seems that the doctor is saved by a knife missile], but these seem to just come right out and say it."
Finally, it can also be said that the title, Inversions suggests that this is a look at the Culture from the outside (i.e. from the PoV of a civilisation they are controlling), rather than a view from inside the Culture/Special Circumstances like most of the other Culture novels.