IP is based on the principle of getting from point A to point B. It is an
unreliable,
connectionless protocol, meaning that no attempt is made to keep the
packet from getting lost, and no packet is set up to go through the same set of routers (which is commonly known as a
circuit or a virtual circuit) but is bounced through each router in turn (known as next-hop) and may be reassembled at the destination out of order.
The real meat of the IP address is a set of four numbers. You have the source IP address and port number, and the destination IP address and port number. You know from the IP addresses that they exist on a certain network through the subnet mask, so you don't have to keep track of every host individually. If you're a router and you get a packet, you check it against a table of hosts you know about, and if so, send it to that host. If not, then you check networks you know about, and forward the packet to that network. If no tables match, you send it to a default router and let THAT worry about it.
That's it.... everything else is just a small matter of programming.