Adjacency is being next to each other. A graph (in the graph theoretical sense) describes an adjacency relation. If the term is used with routers it's because graph theory has useful applications in that area.