Well, I don't know exactly how the netfilter architecture works, so I'm not sure if all architectural aspect are 100% correct, but I'll give you three examples that should pretty well describes at least what appears to happen:
1:
A packet is received on an interface that is actually intended for the machine. As soon as it has traversed the first code of receiving it from the interface and verifying its MAC address, matching it with an established TCP connection (note that at this stage, this is matched against the ip_conntrack tables, not by the socket table as returned by netstat) and so on, the INPUT chain is called to check it out. The netfilter code calls each relevant match module and if they match, it passes the packet on to the match handler, or stops if the target was ACCEPT. If the packet was accepted, the netfilter code returns and the packet is passed on to the actual protocol stack, which matches it against a socket and passes it on to userspace, to the process that owned the socket (or returns an ICMP or TCP error if there was no socket that wanted the packet).
2:
Some process wants to send data. The protocol stack handles the packet and slaps on a TCP or UDP and an IP header on the data. After that, the netfilter code is called upon to check the packet against the OUTPUT chain. If the result is ACCEPT, then the packet is handed on to the interface device driver, which transmits it on the wire.
3:
The machine is a router and receives a packet which is intended for another network which the machine routes to. Again, the packet's MAC address is checked and the conntrack tables checked and so on, and the packet passed through the INPUT chain. Only this time, if the packet was ACCEPTed, it is not passed to the protocol stack, but to the forwarding code, that matches it against the routing tables (as returned by route(8) or by "ip route ls"). If it cannot match the packet against a routing entry, it returns an ICMP error to the sender. If it does match the packet, though, it sends it to the netfilter code which (here it comes =) ) passes it through the FORWARD chain. If the packet was ACCEPTed by the FORWARD chain, the forwarding code passes the packet on to the output code, which passes it through the OUTPUT chain and sends it as in example 2.
So, you might think of something like this "pseudo-machine":
Code:
----- ----- ----- ----- ----- ----- ----- -----
1 -> ---| 2 |--| 3 |--| 4 |--| 6 |--| 7 |---| 9 |--| A |--| B |--- -> C
----- ----- ----- ----- ----- ----- ----- -----
| |
----- ------------- -----
| 5 |--| Userspace |--| 8 |
----- ------------- -----
The numbering is arbitrary, I just wanted to save space so that it wouldn't wrap for those with lower vertical resolutions than what I have by not typing out the actual text.
Anyway, think of the lines between the boxes as "pipes", through which the packets flow. Think of the boxes as small "machines" that do stuff with the packets.
Here is what it should say instead of the numbers:
1: Physical wire; packets come in from here.
2: NIC device driver and initial matching code, that checks the MAC address and conntrack tables and so on.
3:
iptables INPUT chain; packets are only passed on if the final resolution of the chain is ACCEPT. Note that some iptables targets don't terminate the chain, but alter the packet and passes it on to later rules. See the iptables manpage for more on this.
4: "IP Valve"; If the packet is intended for the system's own IP address, it is passed downwards, otherwise to the right.
5: The protocol stack, input half; IP and TCP/UDP headers are decoded, the packet is matched against the socket table and so on.
6: Forwarding code; checks so that /proc/sys/net/ip_forward is on, matches the packet against the routing tables, etc.
7:
iptables FORWARD chain; packets are only passed on if the final resolution is ACCEPT.
8: The protocol stack, output half; TCP/UDP and IP headers are prepended to the packet, etc.
9: NOOP; does nothing, just combines the packets coming from below and from the left and passes them on.
A:
iptables OUTPUT chain; packets are only passed on if the final resolution is ACCEPT.
B: Output code and NIC device driver; build the ethernet header and sends the packet to the NIC.
C: Physical wire; packets are transmitted here.
Note that the Userspace box should not be considered as a "pass-through" box like the others. Packets coming into it do usually not return on the right (of course, that's not necessary, userspace processes might of course forward the packets themselves, as is the case with SOCKS servers... =) ).
Note also, that this is extremely simplified. Not only is the IP code much more advanced, but there are also the other iptables tables: "nat" and "mangle". This only describes the workings of the "filter" table.