Routing is, generically, done in two levels: within an
autonomous system and between them. Routing within an autonomous system is done using an
IGP (interior gateway protocol). Routing between autonomous systems uses
EGP (exterior gateway protocol).
The two most common protocols for intra-AS routing are OSPF and RIP, although IS-IS also exists. These do the routing very differently.
OSPF (as well as IS-IS) is known as a link state routing protocol, which means that each router has a picture of what it thinks the network looks like right now, and decides which of its neighbors is on the best path to a destination.
RIP (routing information protocol) is called a distance vector protocol. Here, a router gets from its neighbors the length of the best path they have to every destination. Then, the router can decide among them. While this might more efficient than the global knowledge OSPF requires, it has other issues.
BGP is the most common exterior gateway protocol. It is somewhat similar to RIP, but instead of getting just the length of the best path, BGP tells the path that it has as well. This helps alleviate some of the routing loop problems that RIP has.
Generally, each border router (usually at an peering point) participates in both the IGP and EGP for that network, announcing paths internally for all the networks that its external neighbors have told, as well as taking the information internally and deciding what routes to announce to its external neighbors.