So both machines are currently connected to the router and have ip addresses on the same subnet (preferably assigned by the router)?
If so, your network is running. Can you ping each machine successfully?
Assuming Windows XP, run the network wizard on each machine, assign the same workgroup name (default MSHOME for XP) and set Internet for LAN.
Once you have done this, you should be able to see the machines in network neighborhood. Also, if you have a Windows firewall on your network adaptor, you will probably have to turn it off as it will prevent file sharing. Hopefully, your router has a good firewall.